The development of ANN (Artificial Neural Networks) has started back in the 1940s, but due to some publications showed deficiencies of the early perceptron models, funding was withdrawn and many scientists left the field. The interest in neural networks re-emerged only after some important theoretical results were attained in the early eighties (most notably the discovery of error back-propagation), and new hardware developments increased the processing capacities. Today, ANN are successfully employed in the fields of identification and control systems, game-playing AI and decision making, pattern recognition (face recognition, voice recognition, object recognition), sequence recognition, health diagnosis, stock trade applications (automated trading systems) and even…e-mail spam filtering.

The theory behind ANN is full of abstraction. There are several new notions for many of us, as they are not really the field of engineers, but rather mathematicians or theorists. But in many cases the networks finalize in a software or hardware implementation, which we can touch or see.

Example 1
An ANN can be implemented as software on the PC, and its purpose could be to recognize a specific face. The input data could consist of an image of a human face taken by a webcam, and the output of the ANN could be a yes/no answer regarding the permission of that person to access a particular part of a building.

Example 2
An ANN can be hardware implemented in an FPGA, which could act together with a microphone and ADC in order to recognize a specific voice command. If the network is correctly trained and implemented, it can also be used to recognize the command as being valid only coming from a particular person, recognizing the specific patterns of that voice.

Example 3
An ANN can be used as a hardware-software system to decide whether a specific product is “good” or “bad”. For instance a company manufacturing video cameras can use such a network to decide if the image from a specific camera is acceptable or not. Rather than measuring all kind of parameters as focus or color sharpness or hue etc, the image from each camera may be fed to a ANN which would decide for itself if it is acceptable or not, based on previous training. Simplistically speaking, the training is accomplished by feeding the ANN a reasonable number of good images (acceptable for a human judge) and informing the ANN that these images are good; then the ANN would be fed a similar number of unacceptable images (for a human judge) and it would be informed that these images are “bad”. If the proper training algorithm is used and if the number if images used to train the camera is also appropriate, the ANN will be able from then on to decide by itself if a new image is good or bad.

From an abstract point of view, the neural network is an interconnected group of nodes, similar to the huge networks of neurons in the human brain (although this comparison really simplifies the structure of the human brain).

The simplest possible “node” is a perceptron, which in the most basic case has only two inputs and an output:

Each ho the two inputs, x1 and x2 has a corresponding weight, so the output of the perceptron would be a function of the simple weighted sum of these inputs:

The function f is also defined exactly when the perceptron is actually implemented in software or hardware so you are really looking at a single layer neural network, which takes as inputs some numbers, and which returns also a number. The output of the network is thus dependent on the weights (w1 and w2). These should be adjusted so that the output of the perceptron suits the application, and the adjustment itself is done during the training phase.

The reason for which such a concept is considered an artificial neural network, is its similar aspect to a real neural network, which is also based around basic nodes called neurons. The natural structure of a neuron and its environment is shown below with the elements that compose it: a set of incoming fibers - the dendrites (corresponding to the input values x1, x2 of the perceptron), the synapses (and the synaptic efficiency corresponds to the weights of the perceptron), the soma (corresponding to the arithmetical summation of the perceptron) and one outgoing fiber – the axon (roughly, the function f).

Perceptrons may be grouped together in larger neural networks, just as the human brain consists of about 100.000.000.000 neuron (enough for 1000 years of life – these neurons degrade in time, and nature has not provided us yet with a mechanism to replace them). Thus complicated artificial neural network may be built, and these can be used for various tasks.

Of course the mathematical theory behind the ANN is stuffy enough to scare many people oriented to more practical purposes. But also these structures can have simple applications which may be considered the “hello world” of ANNs. I will suggest below one such application.

Character recognition application
A relatively simple ANN may be trained to recognize the characters from 0 to 9. The characters can be considered as being drawn on a grid of 10x14 pixels, each pixel having a value of 0 (for white) or 1 (for black).

Thus the image may be decomposed in a 140 elements vector, and this vector may be fed to such an ANN with 140 inputs (having a reasonable number of perceptrons in its first layer, for instance 12 perceptrons with 5 inputs each) and with ten perceptrons in the output layer. Depending on which of the 10 characters from 0 to 9 is represented by the bitmap, only one of the 10 outputs of the ANN should be activated as being logic 1, all the others should be logic 0.

Thus a structure for the ANN is established. By no means should this be considered the best structure for this application; however, it will get the job done. There is heavy literature out there (quite abstract, but very scientific and theoretical) that will teach you how to choose the structure for the ANN to best suit the application you have in mind.

OK, once the structure of the ANN is established, the next required job is to train it. Again, this is a science in itself, and training an ANN may involve quite complicated algorithms; luckily these are also somewhat off-the shelf and readily implemented in powerful development environments, like C or Matlab.

The training of the ANN is a process in which the ANN “learns”; it basically means adjusting the weights of each perceptron in the network so that when presented with a known pattern (like for instance a clear character “2”, with no noise involved and no similarity to other characters) the network will activate the proper output. Basically, it means that the network is presented with character “2” and its weights will be adjusted according to a particular algorithm so that mathematically speaking, the output of the third perceptron in the output layer will be close to 1 and the outputs of all other 9 perceptrons will be 0. Of course the proper adjustment of the weights cannot be achieved in a single iteration, and training a ANN involves presenting it several times with known patterns and also with patterns which are affected by noise, like for instance the figure below:

If trained properly, the ANN should be capable of recognizing a noisy image, just like the human brain would.

Due to their theoretical complexity, neural networks are not particularly popular with engineers; they are more the realm of mathematicians. However, if you are curious, give them a try. Imagine what complex algorithms would be involved if a conventional piece of software would have to be used in order to recognize the above characters. It might not be rocket science, but it complexity might grow exponentially with the resolution of the characters. ANNs provide a rather cheap (and even fast both in terms of implementation and in terms of execution) method for the problem
As I mentioned, Matlab has a pretty good toolbox regarding neural networks, which allow you to generate, train and even use an ANN with relative ease. The Matlab environment also includes some nice demos which are much more visual (and perhaps instructive) than the mathematical language conventionally employed to describe these networks.

Further reading:
http://www.gc.ssr.upm.es/inves/neural/ann1/anntutorial.html
http://www.learnartificialneuralnetworks.com/
http://en.wikipedia.org/wiki/Artificial_neural_network
http://en.wikipedia.org/wiki/Neural_network_software
http://neurondotnet.freehostia.com/index.html
http://www.patternrecognition.co.za/sourcecode.html