UNDERSTANDING NEURAL NETWORK MATHEMATICS

Artificial Neural Networks "imitate" how real brain cells work by focusing on a few key aspects of how neurons behave and turning those behaviors into a computer program. However, ANNs don't try to simulate a biologically complete model of a neuron - only three key features are explicitly programmed into an artificial neural network.

1) Talking with many neighboring cells. Real brain cells receive electrical input signals from their neighboring cells and send out electrical signals to other cells. In essence, the cell listens to its neighbors and decides what to do next. A neural network is set up the same way: nodes in the artificial neural network listen to inputs, then respond with outputs.

2) Activation Threshold. Neurons must receive a minimum electrical input to start generating outputs. Essentially, there's a threshold effect - below the threshold, nothing happens. Above the threshold value, more input leads to more output. Artificial neural networks replicate this feature of real cells: each node will only generate output if receives input that exceeds a threshold value.

3) Weighted Listening. In the brian, not all electrical input signals are equal; input signals from some brain cells count more than others. For example, some inputs activate a given brain cell 3x more than others signals of the same magnitude. And some signals could be worth nothing at all (or even inhibitory - a negative weighting). In the artificial neural network, we will replicate this principle by weighting each node's inputs differently.

The equation for a neural network with five nodes is shown below (two inputs, two nodes in layer 1, and and output layer).

Click the example below to see a visual representation of how the neural network equation breaks apart into distinct pieces (circles represent nodes, bars represent the process of aggregating inputs to a node, w's are weights, and T and HR represent inputs to the first layer of nodes).