Recurrent Artificial Neural Networks
Feedforward describes the connection pattern of the network, not merely the fact that the network has an input and an output.
The Decisive Structural Question
A neural network may receive information at one side and eventually produce an output, but that description alone does not tell you whether it is feedforward. The decisive question is whether the network's connections contain a loop. A feedforward artificial neural network has no loop in its connections. If even one loop exists, the network is recurrent.
Feedforward describes the connection pattern of the network, not merely the fact that the network has an input and an output.
Layers and Information Flow
A feedforward ANN is organized into an input layer, one or more hidden layers, and an output layer. The input layer receives the input. The output layer produces the output. Any layer that is neither the input layer nor the output layer is a hidden layer. Hidden means that the layer is internal to the network; it does not mean that the layer is disconnected or inactive.
The layer arrangement helps you describe where units are positioned, but the absence of loops supplies the classification. A network can have identifiable input, hidden, and output layers and still be recurrent if its connections include a loop.
Connections and Weights
The units in an ANN are connected by links. Every connection has an associated real-valued weight. In terminology used for biological neural networks, a weight roughly corresponds to the efficacy of a synaptic connection. These weights are part of the network's specification, but they are not what determines whether the network is feedforward. That classification depends on whether the links form a loop.
| Feature | What it tells you |
|---|---|
| Layer organization | The network has input, hidden, and output positions. |
| Connection weight | Each link has an associated real-valued weight. |
| Loop structure | No loop means feedforward; at least one loop means recurrent. |
Separate the network's components from the structural test used for classification.
Classifying a Network Diagram
A loop changes the classification
Consider a network organized with an input layer, a hidden layer, and an output layer. Its connections go from the input layer to the hidden layer and from the hidden layer to the output layer. Then add one connection from a hidden unit back to an earlier unit.
- 1Identify the layers
The entrance is the input layer, the internal layer is the hidden layer, and the exit is the output layer.
- 2Inspect the connections
The forward connections describe movement through the layer arrangement, but the additional connection points back to an earlier unit.
- 3Check for a loop
The backward connection creates a loop through which a unit's output can influence its own input.
- 4Classify the network
The presence of even one loop means the network is recurrent rather than feedforward.
The network is recurrent. Its input, hidden, and output layers can still be identified, but the loop prevents it from being feedforward.
Common Classification Mistakes
Common mistake Calling every network with an input and an output feedforward.
What it looks likeThe network receives information on one side and eventually produces an output, so it is labeled feedforward without checking its links.
Why it failsHaving an input and an output is not sufficient. A recurrent network can also have input, hidden, and output layers.
Do this insteadInspect the connection pattern and determine whether it contains a loop.
Common mistake Assuming that a hidden layer is disconnected or inactive.
What it looks likeThe word hidden is interpreted as meaning that the layer does not participate in the network.
Why it failsHidden means that the layer is internal rather than the entrance or exit of the network.
Do this insteadIdentify a hidden layer as any layer that is neither the input layer nor the output layer.
Common mistake Using the weights to decide whether an ANN is feedforward.
What it looks likeA connection has a real-valued weight, so the network is called feedforward.
Why it failsWeights describe connections, but feedforward classification depends on whether the connections form a loop.
Do this insteadRecord that each link has a real-valued weight, then perform a separate loop check.
Common mistake Ignoring a single loop because most connections point toward the output.
What it looks likeNearly all connections move through the layers toward the output, but one connection points back to an earlier unit.
Why it failsThe presence of even one loop changes the classification to recurrent.
Do this insteadTreat any loop as decisive.
Practice the Loop Test
A network has an input layer, two hidden layers, and an output layer. Every connection proceeds through the arrangement toward the output except one connection that returns from the second hidden layer to the first hidden layer. Is the network feedforward or recurrent? Explain which structural feature determines your answer.
Hints
- First identify the input, hidden, and output layers.
- Then ignore the number of layers and inspect whether any connection creates a loop.
- A single loop is sufficient to determine the classification.
The practice network is recurrent because the connection returning to the first hidden layer creates a loop. The number of hidden layers does not determine the classification.
Summary
- 1A feedforward ANN is defined by the absence of loops in its connections. It is organized with an input layer, one or more hidden layers, and an output layer. The input layer receives input, the output layer produces output, and internal layers are hidden layers. Every connection between units has an associated real-valued weight. If even one connection creates a loop through which a unit's output can influence its own input, the network is recurrent rather than feedforward.
At a glance
- Feedforward describes the absence of loops in an ANN's connections.
- A feedforward ANN has an input layer, one or more hidden layers, and an output layer.
- Each connection between units has an associated real-valued weight.
- The presence of even one loop makes the network recurrent.
- Classify the network by inspecting its connection pattern, not merely by observing that it has inputs and outputs.