Symbolic AI and Rule-Based Systems
The expected answers are important because they accompany the original data and give the computer the intended answer for each example. The result is not merely a collection of examples: it is a set of rules that can later be applied to new data.
Two Ways to Program
Programming does not always begin with a human writing every rule that a computer must follow. In classical programming, humans write the rules and provide data for those rules to process. In machine learning, humans provide data together with expected answers, and the computer produces rules from that material. Those learned rules can then be used on new data.
What do you think happens?
In which approach does the computer produce rules from examples: classical programming or machine learning?
Reveal answer
Machine learning
Classical programming starts with rules written by humans. Machine learning starts with data and expected answers, from which the computer produces rules.
The Classical Programming Path
The classical programming model has a direct arrangement. A human supplies a program containing rules and also supplies data. The computer follows those supplied rules while processing the data, then produces answers. The path from input to output is designed by the programmer before the computer runs the program.
| Question | Classical programming | Machine learning |
|---|---|---|
| What do humans provide? | A program containing rules and data | Data together with expected answers |
| What does the computer produce? | Answers after following the supplied rules | Rules learned from the provided material |
| What happens to new data? | The human-written rules process it | The learned rules are applied to it to produce answers |
The central difference is where the rules come from.
From Examples to Rules
Machine learning changes the usual starting point for programming. Instead of asking only what instructions a human should write, it asks whether a computer can examine data, discover useful rules, and use those rules on information it has not seen before. The human supplies examples in the form of data and expected answers. The computer uses those paired examples to produce rules. The rules are then available for processing new data.
The expected answers are important because they accompany the original data and give the computer the intended answer for each example. The result is not merely a collection of examples: it is a set of rules that can later be applied to new data.
A Message-Classification Trace
Generated teaching scenario: imagine that a human gives a computer many messages as data. For each message, the human also supplies an expected answer indicating which of two categories the message belongs to. The paired messages and answers provide the material from which the computer learns rules.
From Message Examples to a New Answer
Trace what happens when a computer receives message data together with expected category answers, followed later by an unfamiliar message.
- 1Provide examples
The human supplies many messages as data and gives the expected category answer for each message.
- 2Produce rules
The computer uses the paired messages and answers to produce rules.
- 3Receive new data
A new message arrives that was not part of the original material.
- 4Apply the rules
The learned rules are applied to the new message.
- 5Produce an answer
The rules produce an answer for the new case.
The computer moves from human-provided examples and expected answers to learned rules, then uses those rules to answer a case that was not in the original examples.
Lovelace and the Learning Question
The historical question behind machine learning concerns whether computers can do more than follow explicitly ordered instructions. In 1843, Ada Lovelace argued that Charles Babbage's Analytical Engine could perform whatever people knew how to order it to perform, but could not originate anything. Alan Turing later discussed this position as Lady Lovelace's objection while considering whether general-purpose computers could learn and show originality.
Machine learning addresses the learning part of that question by changing where the rules come from. In classical programming, the rules are explicitly supplied by a human. In machine learning, the computer examines data and expected answers and produces rules that can be used on new information. This does not erase the human contribution: humans still provide the data and expected answers. It changes the role of the computer from only following supplied rules to producing rules from the provided material.
Mistakes About Rule Sources
Common mistake Saying that classical programming learns rules from data.
What it looks likeTreating a human-written program as if the computer had produced its rules.
Why it failsClassical programming starts with rules written by humans, which the computer then uses to process data.
Do this insteadDescribe classical programming as human-supplied rules plus data producing answers.
Common mistake Saying that machine learning begins with rules written explicitly for every case.
What it looks likeDescribing the human as supplying the final rules before the computer examines examples.
Why it failsMachine learning starts with data and expected answers, from which the computer produces rules.
Do this insteadIdentify the data and expected answers as the starting material for learning.
Common mistake Stopping after the computer produces learned rules.
What it looks likeExplaining how rules are learned but not what happens when a new message arrives.
Why it failsThe learned rules can be applied to new data to generate answers.
Do this insteadTrace the complete path: examples and expected answers, learned rules, new data, and an answer.
Common mistake Claiming that humans provide nothing in machine learning.
What it looks likeDescribing the computer as learning without human-provided material.
Why it failsHumans provide the data together with the answers expected for that data.
Do this insteadSeparate the human contribution from the computer's contribution: humans provide examples and expected answers; the computer produces rules.
Check Your Understanding
A human provides a collection of data and an expected answer for each item. The computer produces rules and then receives new data. Explain the sequence from the original material to the answer for the new data. Then state whether this describes classical programming or machine learning, and explain why.
Hints
- Identify what the human provides before the computer works.
- Identify what the computer produces from the paired data and answers.
- Include the step in which the produced rules are applied to new data.
Compare the following two descriptions: one human writes rules and supplies data; another human supplies data with expected answers and the computer produces rules. For each description, name the programming approach and state what the computer produces.
Hints
- Classical programming begins with human-written rules.
- Machine learning begins with data and expected answers.
- The output of the machine-learning stage is rules that can later be applied to new data.
Key Takeaways
- 1Classical programming starts with rules written by humans and uses those rules to process data.
- 2Machine learning starts with data and expected answers, from which the computer produces rules.
- 3Learned rules can be applied to new data to generate answers for cases outside the original material.
- 4The distinction between the two approaches is mainly about what humans provide and where the rules come from.
- 5Lady Lovelace's objection provides a historical backdrop for asking whether computers can learn rather than merely follow explicitly ordered instructions.
At a glance
- Classical programming uses human-written rules to process data.
- Machine learning uses data and expected answers to produce rules.
- The learned rules can be applied to new data to produce answers.
- The key difference is the source of the rules and the role assigned to the computer.
- Lady Lovelace's objection connects machine learning to the historical question of whether computers can learn rather than only follow ordered instructions.