Deeply Layered Artificial Neural Networks

An ANN is a network of interconnected units, not a single isolated unit.

Programming

From Unit to Network

An artificial neural network, or ANN, is not one isolated computational unit. It is a network of interconnected units. Individual units have some properties similar to neurons in nervous systems, but the central idea is the complete connected system: many units working together to represent a relationship.

Key point

When you study an ANN, focus on the relationships among its units, not only on what one unit does by itself.

Connected Units as One System

The word network changes the way the system should be understood. A single unit can be considered in isolation, but an ANN is defined by interconnected units. The network provides the structure used to represent a relationship. Therefore, the relevant question is not simply whether one unit can express a rule. The more useful question is how the connected collection of units can represent a relationship as a system.

What contains what, and how are individual computational units connected to form a complete network?

Worked example

Reading the Network View

Suppose a description refers to three interconnected units as one artificial neural network. What should you identify as the main system?

1
Identify the units
The three units are the individual components of the system.
2
Identify the connections
The relationships among the units are essential because the system is described as interconnected.
3
Name the system
The complete collection of connected units is the artificial neural network, rather than any one unit considered alone.

Result

The ANN is the connected network of units. Treating one unit as if it were the whole ANN misses the central definition.

Representing Nonlinear Relationships

Many machine learning tasks involve relationships that are not simply a straight, easily described rule. Artificial neural networks are widely used for nonlinear function approximation: they use a network of connected units as the tool for representing such a relationship. The important contrast is between an isolated simple rule and a connected system intended to represent a relationship that does not have a simple straight description.

How can a connected collection of units be used when the relationship is not simply a straight, easily described rule?

In the real world

Consider a machine learning task whose relationship cannot be described by a simple straight rule. In the source description, this is the kind of task for which ANNs are widely used. The example does not require selecting one isolated unit; it requires viewing the connected network as the representation tool.

Information Through Deep Layers

A deeply layered ANN extends the network idea across many layers. The basic idea remains a network of interconnected units. What changes is the depth of the network: there are many layers of connected units between the beginning and the end of the system. This layered structure is a useful way to think about how a machine learning system can represent a relationship as a connected system rather than as one isolated rule.

How does information move through multiple layers of interconnected units from input to output?

Best practice

When explaining a deeply layered ANN, describe both parts of the idea: the units are interconnected, and the network contains many layers. Mentioning depth without mentioning connected units leaves out the basic ANN concept.

Training and Expanded Capabilities

Artificial neural networks have a long history. More recent progress in training deeply layered ANNs has contributed to some of the most impressive abilities of machine learning systems. This places training advances in context: the underlying concept is still a network of interconnected units, while improved training of networks with many layers has helped those systems achieve stronger capabilities.

What changes in the network's capabilities as progress is made in training deeply layered networks?

Mistakes in Interpretation

Common mistake
Treating an ANN as one isolated unit
What it looks like
An explanation discusses one unit but calls that unit the entire artificial neural network.
Why it fails
The defining description is a network of interconnected units, not a single isolated unit.
Do this instead
Describe the units together with their connections as the ANN.
Common mistake
Explaining ANN use without mentioning nonlinear relationships
What it looks like
An explanation says that ANNs are useful but gives no connection to nonlinear function approximation.
Why it fails
The source identifies nonlinear function approximation as a major reason ANNs are widely used.
Do this instead
Connect ANN use to representing relationships that are not simply straight, easily described rules.
Common mistake
Assuming that depth alone defines the whole concept
What it looks like
An explanation focuses on many layers but leaves out the interconnected units.
Why it fails
Deep layering is part of the concept, but the foundational ANN idea is still a network of interconnected units.
Do this instead
Explain depth as many layers within the connected network.
Common mistake
Separating training advances from machine learning abilities
What it looks like
An explanation mentions improved training but does not connect it to the expanded abilities of machine learning systems.
Why it fails
The source places progress in training deeply layered ANNs in the story of impressive machine learning capabilities.
Do this instead
State that advances in training have contributed to stronger machine learning abilities.

Check Your Understanding

MEDIUM

Explain, in your own words, why an artificial neural network should be described as a connected system rather than a single isolated rule. Then explain why nonlinear function approximation is an important use of ANNs.

Hints

Begin with the definition of an ANN as a network of interconnected units.
Contrast a simple straight, easily described rule with a relationship that is not simply straight.
End by connecting progress in training deeply layered ANNs to machine learning abilities.

Your attempt (not saved)

Key Takeaways

1
An artificial neural network is a network of interconnected units, not one isolated unit.
2
ANN units have some properties similar to neurons in nervous systems.
3
ANNs are widely used for nonlinear function approximation because many machine learning relationships are not simply straight, easily described rules.
4
A deeply layered ANN preserves the network idea while organizing interconnected units across many layers.
5
Progress in training deeply layered ANNs has contributed to impressive abilities in machine learning systems.

At a glance

An ANN is defined by interconnected units working as a network.
The network perspective is more important than treating one unit as the whole system.
ANNs are widely used to approximate nonlinear relationships that are not simply straight, easily described rules.
Deeply layered ANNs extend the connected-network idea across many layers.
Advances in training deeply layered networks have contributed to expanded machine learning capabilities.

More concepts starting with D