Classification tasks
The central idea is not simply producing an output. The system must discover rules that turn available input into a form useful for the task, such as classification.
From Examples to Classes
A classification task asks a machine-learning system to produce a class as the output for an input. The system is not given a complete list of hand-written rules for every possible input. Instead, machine learning discovers data-processing rules from examples of inputs and expected outputs.
The central idea is not simply producing an output. The system must discover rules that turn available input into a form useful for the task, such as classification.
The Learning Cycle
Learning requires three basic ingredients: input data, a way to measure performance, and feedback used to adjust the algorithm. Examples provide inputs together with expected outputs. The system can then compare its result with the expected output through a performance measurement. Feedback from that measurement is used to adjust the algorithm, helping it discover a more useful data-processing rule.
Tracing One Learning Cycle
A system receives examples of inputs and expected classes. Trace the role of each part of the learning process.
- 11. Receive examples
The system receives input data together with expected outputs.
- 22. Produce a result
The system uses its current data-processing rule to produce an output for an input.
- 33. Measure performance
The result is assessed using a performance measurement.
- 44. Use feedback
Feedback from the performance measurement is used to adjust the algorithm.
The rule changes through the learning cycle rather than being supplied as a complete hand-written list of rules.
What Representation Means
A representation is a way to encode or view data. It describes how the available input is presented so that a learning system can process it.
Representation does not mean that the underlying data has become a different real-world thing. It means that the data is encoded or viewed in a particular way. The choice of representation affects how directly a task can use the information.
Why Useful Representations Help
A representation is useful when it makes the information needed for a task more directly available to the learning process. For classification, a useful representation can make the expected classes easier to distinguish. The learning system is therefore concerned not only with which class it outputs, but also with how it changes the way it views the input.
Comparing Representations
Suppose the same input can be viewed in two different ways before a classification task. What should we ask about the two representations?
- 11. Identify the available information
Both representations describe the same available input data.
- 22. Examine task usefulness
Ask which representation presents information more directly for distinguishing the expected classes.
- 33. Connect usefulness to learning
A more useful representation can make discovering a rule for the classification task easier.
The better representation is not defined by being a different real-world object. It is useful because it makes the task easier for the learning process.
Representation Learning
Learning representations means finding transformations of input data that move the model closer to the expected output. The model is not merely memorizing an answer for each example. Its task is to discover rules that turn the available input into a form useful for the task, such as classification.
The search for appropriate representations is central to machine learning and deep learning because useful transformations help connect available inputs to expected outputs.
Mistakes About Classification
Common mistake Thinking that machine learning begins with a complete list of hand-written rules for every input.
What it looks likeTreating the system as if every possible input-to-class decision had already been specified.
Why it failsMachine learning discovers data-processing rules from examples of inputs and expected outputs.
Do this insteadFocus on the learning process: examples are used to discover a rule rather than supplying a complete list of rules.
Common mistake Treating representation as the final class output.
What it looks likeCalling the encoded or viewed form of an input the classification result.
Why it failsA representation is a way to encode or view data. It is an input form that can support the task, not necessarily the expected output.
Do this insteadSeparate the represented input from the class that the system is expected to produce.
Common mistake Ignoring performance measurement and feedback.
What it looks likeDescribing learning as receiving examples and producing outputs without any adjustment.
Why it failsLearning requires performance measurement and feedback used to adjust the algorithm.
Do this insteadInclude the full cycle: examples, output, performance measurement, feedback, and algorithm adjustment.
Common mistake Assuming that every representation makes a task equally easy.
What it looks likeTreating a change in representation as irrelevant to classification.
Why it failsDifferent representations of the same data can support different tasks more effectively.
Do this insteadAsk whether a representation makes the information needed by the classification task more directly available.
Common mistake Equating representation learning with memorizing an answer for each example.
What it looks likeAssuming the model only stores separate answers instead of discovering a rule.
Why it failsRepresentation learning searches for transformations that move the model closer to the expected output.
Do this insteadDescribe the goal as discovering useful transformations and rules for processing new available inputs.
Check Your Understanding
Explain, in your own words, why a classification system needs more than input data alone. Include the roles of expected outputs, performance measurement, feedback, and representation.
Hints
- Start with what examples provide.
- Explain how performance measurement produces useful feedback.
- End by describing how a representation can make classification easier.
A task seems difficult using one representation of its input. What question should you ask before concluding that the task itself cannot be learned?
Hints
- Compare the current representation with another possible way to encode or view the data.
- Ask which representation makes the task's information more directly usable.
Key Takeaways
- 1Machine learning discovers data-processing rules from examples of inputs and expected outputs.
- 2Learning requires input data, performance measurement, and feedback used to adjust the algorithm.
- 3A representation is a way to encode or view data.
- 4Different representations of the same data can make different tasks more or less effective.
- 5Representation learning searches for transformations that make the input more useful for tasks such as classification and is central to machine learning and deep learning.
At a glance
- Classification uses learned data-processing rules to connect inputs with expected classes.
- Examples, performance measurement, and feedback form the basic learning cycle.
- A representation is an encoding or view of data, and a useful representation can make a task easier.
- Representation learning seeks transformations that move input data closer to the expected output.
- Finding useful representations is central to machine learning and deep learning.