Weights and Connections in Neural Networks
Feedforward describes the connection pattern, not merely the fact that a network has an input and eventually produces an output.
The Loop Test
A neural network may have an input on one side and an output on the other, but that description alone does not tell you whether it is feedforward. The decisive question is structural: do the connections contain a loop? A feedforward artificial neural network has no loop in its connections. If even one loop exists, the network is classified as recurrent.
Forward Information Flow
In a feedforward artificial neural network, information moves through the network's connection pattern without returning through a loop to an earlier unit. The network is organized into an input layer, one or more hidden layers, and an output layer. The input layer is where the input enters, hidden layers are internal layers, and the output layer produces the output.
Feedforward describes the connection pattern, not merely the fact that a network has an input and eventually produces an output.
Layers and Internal Units
Each layer has a distinct position. The input layer receives the input. The output layer produces the output. Any layer that is neither the input layer nor the output layer is a hidden layer. A feedforward ANN may have one or more hidden layers.
Weights on Connections
The units in a feedforward ANN are connected by links. Every link has an associated real-valued weight. Therefore, a connection is not identified only by the units it joins; its associated weight is also part of the network's specification. In terminology used for biological neural networks, a weight roughly corresponds to the efficacy of a synaptic connection.
Classifying a layered network
Consider a network with an input layer, one hidden layer, and an output layer. Every link between its units has a real-valued weight, and no link returns to an earlier unit. Is it feedforward or recurrent?
- 1Locate the layers
The network has the input, hidden, and output layer roles expected in a feedforward ANN.
- 2Inspect the connections
The connections do not form a loop, so no unit's output can influence its own input through a returning path.
- 3Use the decisive test
The absence of loops is the feature that determines the classification. The real-valued weights describe the links but do not change this structural test.
The network is feedforward.
When a Loop Changes the Label
A network can still have recognizable input, hidden, and output layers while being recurrent. The classification changes when at least one loop exists. In particular, a feedforward ANN has no path through which a unit's output can influence its own input. One loop is enough to make the network recurrent.
| Feature | Feedforward ANN | Recurrent ANN |
|---|---|---|
| Connection pattern | No loops | At least one loop |
| Input, hidden, and output layers | Can be identified | Can still be identified |
| Unit output influencing its own input through a path | No | Possible because a loop exists |
The presence or absence of a loop determines the feedforward or recurrent classification.
Common mistake Calling every network with an input and an output feedforward.
What it looks likeA diagram shows information entering at one side and producing an output, but one connection returns to an earlier unit.
Why it failsHaving an input and an output is not sufficient. The decisive feature is whether the connections contain a loop.
Do this insteadInspect the complete connection pattern and classify the network as recurrent if even one loop exists.
Common mistake Assuming that a hidden layer is disconnected or inactive.
What it looks likeA learner treats the word hidden as meaning that the layer does not participate in the network.
Why it failsHidden means that the layer is internal rather than the input or output layer.
Do this insteadIdentify hidden layers by position: any layer that is neither input nor output is hidden.
Common mistake Ignoring weights when describing the network's connections.
What it looks likeA learner lists the links between units but does not mention that each link has an associated real-valued weight.
Why it failsWeights are an additional part of the network's specification.
Do this insteadDescribe each connection together with its associated real-valued weight.
Classify the Pattern
A diagram contains an input layer, two hidden layers, and an output layer. All connections move through the layers without returning to an earlier unit. Each connection has a real-valued weight. Classify the network and justify your answer using the connection pattern.
Hints
- Count the layer roles, then inspect whether any connection creates a loop.
- The number of hidden layers does not determine whether the network is feedforward.
- Use the absence or presence of loops as the decisive evidence.
What do you think happens?
Suppose the same network gains one connection from an output-side unit back to an earlier hidden unit. What is its classification?
Reveal answer
It is recurrent.
The presence of even one loop changes the classification from feedforward to recurrent, even though the input, hidden, and output layers can still be identified.
Key Takeaways
- 1A feedforward ANN is defined by the absence of loops in its connections.
- 2A feedforward ANN is organized into an input layer, one or more hidden layers, and an output layer.
- 3The input layer receives input, the output layer produces output, and hidden layers are internal layers.
- 4Every connection between units has an associated real-valued weight.
- 5The presence of even one loop makes the network recurrent.
At a glance
- Feedforward refers to a network's connection pattern, specifically the absence of loops.
- Input, hidden, and output layers describe the positions and roles of layers in the network.
- Every link between units has an associated real-valued weight.
- A single loop is enough to classify the network as recurrent rather than feedforward.