Deep Learning Basics
Machine learning does not begin by giving the computer a complete, explicitly ordered set of rules. It begins with examples containing data and expected answers.
The Programming Question
Most programming begins with a human deciding which rules the computer should follow. Machine learning begins with a different question: can a computer examine data, discover useful rules, and use those rules on information it has not seen before? The key change is where the rules come from. In classical programming, humans write them. In machine learning, the computer produces them from data and expected answers.
Machine learning does not begin by giving the computer a complete, explicitly ordered set of rules. It begins with examples containing data and expected answers.
From Examples to Rules
The machine learning process described here has three important stages. First, humans provide data together with the answers expected for that data. Second, the computer uses those paired examples to produce rules. Third, the learned rules are applied to new data, producing answers for cases that were not part of the original material.
Classifying Messages
A human provides many messages as data and supplies an expected category for each message. How can those examples support an answer for a new message?
- 1Provide paired examples
The computer receives messages together with the expected answer for each message, such as one category or another.
- 2Produce rules
The computer uses the paired data and answers to learn rules.
- 3Receive a new case
A new message arrives that was not part of the original material.
- 4Apply the rules
The learned rules are applied to the new message to produce an answer for that case.
The original examples lead to learned rules, and those rules generate an answer for a new message.
Classical and Machine Learning
The clearest comparison is to track what humans provide and what the computer produces. In classical programming, humans provide a program containing rules and data for those rules to process. The computer follows the supplied rules and produces answers. In machine learning, humans provide data together with the expected answers. The computer uses that material to produce rules, which can later be applied to new data.
| Approach | Humans provide | Computer produces |
|---|---|---|
| Classical programming | A program containing rules and data | Answers after following the supplied rules |
| Machine learning | Data together with expected answers | Rules that can be applied to new data |
The central difference is the direction of the relationship between rules, data, and answers.
Using Rules on New Data
Learning does not end when the computer produces rules. The purpose of those rules is to handle new data. A new case moves into the learned rules, and the rules produce an answer for that case. The new case was not part of the original examples, so this stage shows how machine learning changes the usual programming path: the computer first produces rules from examples, then uses those rules on additional information.
What do you think happens?
A computer has used paired messages and expected categories to produce rules. What should happen when a message arrives that was not in the original examples?
Reveal answer
The computer applies the learned rules to produce an answer.
The learned rules are intended to be used on new data, including cases that were not part of the original material.
Lady Lovelace's Objection
The historical question behind machine learning concerns whether a computer can do more than follow instructions explicitly ordered by people. In 1843, Ada Lovelace argued that Charles Babbage's Analytical Engine could perform whatever people knew how to order it to perform, but could not originate anything. Alan Turing later discussed this idea as Lady Lovelace's objection while considering whether general-purpose computers could learn and show originality.
Machine learning addresses the learning part of that historical question by changing where the rules come from. Rather than asking only what instructions a human should write, it asks whether a computer can examine data, discover useful rules, and then use those rules on information it has not seen before. This does not erase the role of humans: people provide the data and expected answers. It does change the computer's role from merely following human-written rules to producing rules from the supplied material.
The historical connection is about the origin of rules. Lady Lovelace's objection asks whether a computer can originate anything; machine learning explores whether a computer can learn rules from data instead of receiving every rule directly from a human.
Mistakes About Machine Learning
Common mistake Saying that machine learning starts with rules written by humans.
What it looks likeThe computer is given a complete set of rules and then learns from them.
Why it failsThat describes the classical programming arrangement. In the machine learning arrangement, humans provide data and expected answers, and the computer produces rules.
Do this insteadState that machine learning starts with data and expected answers from which the computer produces rules.
Common mistake Stopping after the computer produces rules.
What it looks likeThe learned rules are the final answer, so no new data is needed.
Why it failsThe learned rules are meant to be applied to new data to generate answers for new cases.
Do this insteadDescribe both stages: producing rules from examples and applying those rules to new data.
Common mistake Claiming that humans provide nothing in machine learning.
What it looks likeThe computer independently invents a solution without human-provided material.
Why it failsThe described machine learning process depends on humans providing data together with expected answers.
Do this insteadIdentify the human contribution as the data and expected answers, while identifying learned rules as the computer's production.
Common mistake Treating machine learning as unrelated to the historical question about computers and originality.
What it looks likeLady Lovelace's objection concerns only older mechanical engines and has no connection to machine learning.
Why it failsThe source connects the objection to the question of whether computers can learn rather than merely follow explicitly ordered instructions.
Do this insteadExplain that machine learning addresses the learning part of that question by changing where the rules come from.
Check Your Understanding
Compare the two programming approaches in your own words. For each approach, identify what humans provide, what the computer does with that material, and what the computer produces. Then describe how a new message would be handled after machine learning has produced rules.
Hints
- Use the sequence humans provide, computer processes or learns, and computer produces.
- For classical programming, focus on a human-written program containing rules.
- For machine learning, focus on data together with expected answers, followed by learned rules and their application to new data.
A Complete Trace
Trace the machine learning process for a new message.
- 1Human input
Humans provide many messages and the expected category for each message.
- 2Rule production
The computer uses the paired messages and categories to produce learned rules.
- 3New data
A message arrives that was not included in the original examples.
- 4Answer production
The learned rules are applied to the new message and produce an answer for that case.
Data and expected answers lead to rules, and the rules are then used to answer a new case.
What to Remember
- 1Classical programming uses human-written rules to process data and produce answers.
- 2Machine learning uses data and expected answers to produce rules.
- 3Learned rules can be applied to new data to generate answers for cases not included in the original examples.
- 4The historical connection to Lady Lovelace's objection concerns whether computers can learn rather than merely follow explicitly ordered instructions.
- 5Machine learning changes where the rules come from, while humans still provide the data and expected answers.
At a glance
- Classical programming begins with human-written rules and data; the computer follows those rules to produce answers.
- Machine learning begins with data and expected answers; the computer uses them to produce rules.
- Those learned rules can be applied to new data to produce answers for new cases.
- Lady Lovelace's objection provides the historical question of whether a computer can learn rather than merely follow explicitly ordered instructions.