Data representation

Machine learning discovers data-processing rules from examples of inputs and expected outputs.

Programming

From Examples to Rules

A machine-learning system is not given a complete list of hand-written rules for every possible input. Instead, it is exposed to examples of inputs and expected outputs. From those examples, it discovers data-processing rules for carrying out a task.

Data representation is central to this process. A representation is a way to encode or view data. The system does not only need to produce an output; it also needs to find a way of viewing the input that makes producing the expected output easier.

How is raw input data transformed into a representation that a machine-learning system can process?

The Learning Feedback Loop

Learning requires more than input data alone. The system needs examples of inputs, expected outputs for those examples, a way to measure performance, and feedback that is used to adjust the algorithm. The examples provide material for discovering rules. The expected outputs provide a target for the task. The performance measurement indicates how well the current rules are doing, and feedback guides adjustment of those rules.

How do input examples and expected outputs become performance measurements and feedback that improve the learned data-processing rules?

Worked example

A Simplified Learning Cycle

Imagine a machine-learning system that must assign an input to an expected category.

1
1. Provide examples
The system receives input examples together with the expected output for each example.
2
2. Apply current rules
The system uses its current data-processing rules to produce an output.
3
3. Measure performance
The produced output is considered in relation to the expected output, creating a performance measurement.
4
4. Use feedback
Feedback from the measurement is used to adjust the algorithm and its data-processing rules.
5
5. Seek a more useful view
The system can move toward a representation in which producing the expected output is easier.

Result

The learning process uses examples, measurement, and feedback to discover rules and representations that support the task.

Why Representation Matters

The same data can be represented in different ways. Those representations do not necessarily support a task equally well. A task that is difficult with one representation may become easier with another because the useful information is presented in a form that is more directly usable for the task.

What changes when the same data is represented in a way that makes the desired prediction or decision easier?

In the real world

Generated example: Suppose a system receives descriptions of items and must assign each item to an expected category. One representation might preserve the descriptions in a form that does not make the category distinction direct. Another representation might organize the available information so that the distinction is easier for the task to use. The input data has not necessarily changed; the way it is encoded or viewed has changed.

Representation Learning and Prediction

How does learning a useful representation contribute to the central goal of making accurate predictions or decisions from data?

Learning representations means finding transformations of input data that move the model closer to the expected output. The model's task is to discover rules that turn available input into a form useful for the task, such as classification. This search for appropriate representations is therefore connected directly to the central goal of machine learning: discovering data-processing rules from examples of inputs and expected outputs.

Definition

Data representation is a way to encode or view data. In machine learning, representation learning is the search for transformations of input data that produce a form useful for carrying out the task and moving toward the expected output.

Question	Representation-focused answer	Why it matters
What is being changed?	The way input data is encoded or viewed	Different views can support different tasks
What guides the change?	Examples, expected outputs, performance measurement, and feedback	The system can adjust its data-processing rules
What is the goal?	A representation useful for the task	Producing the expected output becomes easier

Representation learning connects the form of the input with the expected result of the task.

Mistakes About Representation

Common mistake
Treating representation as unrelated to the task
What it looks like
Assuming that any way of encoding the input will support the task equally well.
Why it fails
Different representations of the same data can support different tasks more effectively.
Do this instead
Ask whether the representation makes the information needed for the expected output more directly usable.
Common mistake
Describing learning as memorizing one answer per example
What it looks like
Thinking that the system only stores an answer for every input it has already seen.
Why it fails
The system is described as discovering data-processing rules from examples, not merely keeping a list of answers.
Do this instead
Focus on the rules and transformations that turn available input into a form useful for the task.
Common mistake
Leaving feedback out of the learning process
What it looks like
Listing input examples but not including a performance measurement or adjustment step.
Why it fails
Learning requires performance measurement and feedback used to adjust the algorithm.
Do this instead
Trace the full cycle: examples and expected outputs, produced output, performance measurement, feedback, and adjusted rules.
Common mistake
Assuming that the output alone explains learning
What it looks like
Asking only whether the system produced an output.
Why it fails
The important question also includes how the system changes the way it views the input so that producing the expected output becomes easier.
Do this instead
Examine both the produced output and the representation used to reach it.

Best practice

When analyzing a machine-learning process, name both the task and the representation. Then ask how examples, performance measurement, and feedback connect the current representation and rules to the expected output.

Check Your Understanding

MEDIUM

A machine-learning system receives input examples and expected outputs. Its current rules produce outputs, and a performance measurement generates feedback that adjusts the algorithm. Explain where data representation appears in this process and why changing the representation could make the task easier.

Hints

Start by defining representation as a way to encode or view data.
Identify the transformation from input data to a form useful for the task.
Connect the useful form to the expected output and the feedback used for adjustment.

Your attempt (not saved)

Worked example

Tracing the Ingredients

Identify the role of each ingredient in a learning process: input examples, expected outputs, performance measurement, feedback, and representation.

1
Input examples
These give the system data from which it can discover data-processing rules.
2
Expected outputs
These provide the result toward which the task is directed.
3
Performance measurement
This indicates how the current produced output relates to the expected output.
4
Feedback
This is used to adjust the algorithm and its rules.
5
Representation
This is the way the input is encoded or viewed; a useful representation can make producing the expected output easier.

Result

All five ingredients fit together: examples and expected outputs define the learning situation, measurement and feedback guide adjustment, and representation determines how directly the input supports the task.

Key Takeaways

1
A data representation is a way to encode or view data.
2
Machine learning discovers data-processing rules from examples of inputs and expected outputs rather than beginning with a complete hand-written rule for every input.
3
Learning requires input data, performance measurement, and feedback used to adjust the algorithm.
4
Different representations of the same data can support different tasks more effectively.
5
Representation learning searches for transformations that make the input more useful for producing the expected output, making it central to machine learning and deep learning.

At a glance

Data representation describes how data is encoded or viewed.
Machine learning uses examples and expected outputs to discover data-processing rules.
Performance measurement and feedback guide adjustments to the algorithm.
A useful representation can make a task easier without reducing learning to memorizing answers.
Representation learning connects transformations of input data with the central goal of producing useful predictions or decisions.

Related concepts

More concepts starting with D