Author: Christian Eder,
Software-Engineer
Munich, Germany
The human brain is a complex structure consisting of billions of cells, the neurons. These neurons are connected by synapses. The neurons communicate with each other by sending electric pulses to their neighbouring neurons. Since the synapses can have different sizes, these electric pulses can be of varying power. The electricity floating through our brain makes us remember things, react on outside influences or makes our muscles move.
This already simplified image of our brain has been adopted by artificial neural networks. They are a kind of abstract, "mathematical" brain.
Neural networks could be formally described as a weighted graph G = (V, E). The set of vertices V can be seen as the neurons in the network. The edges E, that connect the vertices V are the synapses connecting the neurons. And the weights W of the edges describe the thickness or size of the synapses. A subset of vertices are called input neurons, and another subset of V is called output neurons. All vertices that are not part of the input or output subsets, are called "hidden" neurons.
The picture shows a simple, so called feedforward network - the neurons are only connected from one layer to the next. There exist more complex architectures for neural networks, e.g. with recurrent connections going from the hidden layer back to the input layer, but these networks are not discussed by this article, which wants to give an overview about how artificial neural networks work generally.
The electric pulses which are generated by the neurons in our brain are represented as numbers in neural networks - the higher the number, the higher the voltage.
The input layer of a neural network is where the electric pulses enter the network. In nature, an example is the set of neurons connected to our eyes through the optic nerve. The output layer can be seen as a set of neurons connected to actors like our legs - the eyes send an electric pulse to the input neurons, and our legs move because the output neurons tell them that there is a wall in front of us.
Artificial neural net works can be used to learn nonlinear functions - in other words, they can learn a mapping
where n is the number of input neurons and m the number of output neurons. Each of the neurons has an input and an output value. The values in an n-dimensional input vector are handed over to the input layer, and the output values of the i-th input layer neuron is set to the value in the i-th dimension of the input vector. The neurons in the hidden layer compute their input as the weighted sum of all ingoing connections from the input layer: they sum up all output values of the input neurons they are connected with, and multiply each of these values with the weight of the connecting synapse. After computing their input, the hidden layer neurons compute their output by applying an function to their input. The neurons in the output layer compute their input and output analog.
Formally, each of the neurons works as follows:
Formally, each of the neurons works as follows:
The weights of the synapses, combined with the structure of the net (number of neuons, number of neurons per layer, ...) determines the mapping that the neural network represents. Thus, learning a specific mapping means learning the correct weights of the synapses. There are several algorithms that can be used to accomplish this. One of them is called "error backpropagation".
When using error backpropagation, teaching the network works as follows:
An input vector is given to the net, and the correct output vector expected from the net is known. The actual output vector of the net is compared with the expected, and the difference vector between both defines the error vector. This error vector is set to the corresponding output neurons, and the error is propagated backwards through the net, in order to find out, how the synapse's weights would have to be changed to minimize the error. This gradient based approach needs the activation functions of all neurons to be differentiable. A concrete implementation of the backpropagation algorithm can be found on Wikipedia.
Source Knol: http://knol.google.com/k/introduction-to-the-basics-of-neural-networks#
When using error backpropagation, teaching the network works as follows:
An input vector is given to the net, and the correct output vector expected from the net is known. The actual output vector of the net is compared with the expected, and the difference vector between both defines the error vector. This error vector is set to the corresponding output neurons, and the error is propagated backwards through the net, in order to find out, how the synapse's weights would have to be changed to minimize the error. This gradient based approach needs the activation functions of all neurons to be differentiable. A concrete implementation of the backpropagation algorithm can be found on Wikipedia.
Source Knol: http://knol.google.com/k/introduction-to-the-basics-of-neural-networks#
Ud. 16.1.2024
Pub. 29.4.2012
No comments:
Post a Comment