2.5 Multiple separations
How does a neural network with two inputs learn to make decisions?
Unlike in the previous examples, in which a single neuron successfully learnt to distinguish ladybirds from caterpillars, a slightly larger neural network will be used here (without explaining it further). It will learn a total of four classes of animals, which even overlap slightly.
In the examples so far, we have seen that a neuron can separate two classes from each other with a line. When several neurons are employed, as is the case here, several separation lines can be used to distinguish all classes from each other. Finding such separations between classes is one of the main tasks of neural networks. All this still happens (and will continue to happen) only by changing the weight and threshold values.
Instructions
- Click
Reset: Everything that the network has learnt is reset. - Hover the mouse over the animals in the figure: The type of animal and its data are displayed under the coordinate system.
- Click in the coordinate system to add new animals.
- Hover the mouse over the white dots in the image, which represent unknown animals: A prediction of which animal it could be and what its data are is displayed below the coordinate system.
- Click
Start. After the training, the colouring in the background of the figure indicates which animal is likely to be predicted based on the respective values for width and length.
Concluding remarks
Overlapping data, such as the “mini bumblebee” at position (7 / 7) and the large ladybird at position (7 / 7.5), does not lead to a problem for the network. Neural networks are very error-tolerant and are quite willing to accept such misclassifications. After all, the majority of animals are identified correctly. For this reason, it is rarely desirable or even possible to achieve 100 per cent accuracy. 80 to 90 per cent are often deemed sufficient. In nature or when dealing with real biological or real data in general, overlaps are very frequent, and a rough to near-perfect classification is sufficient. This is why neural networks are often assigned to a field called soft computing.
The neural network manages all this in an imprecise but ingenious way, as it makes errors right from the start. At the beginning (or after clicking Reset) it usually considers all fields to be the same animal. This initially very large error then diminishes as the training progresses. The example on this page stops after 500 training epochs. If the neural network consisted of more neurons and was trained for much longer, it would perhaps place new, additional, smaller separating lines around the overlaps so that the red and blue areas really would contain all the ladybirds and bumblebees, respectively. However, the dividing lines would no longer be so simple and straight. In technical terms, this would mean that the neural network has been overtrained; it would perhaps now identify 100 per cent of the training data but would no longer be able to generalise new data so well, i.e. classify it correctly. This is why the number of neurons is kept as low as possible, a large amount of training data is used, and training is stopped as early as possible. Furthermore, the performance of a neural network should always be evaluated with new, unknown test data.
Share this page
