Writing Effective Tests
The goal is not to inspect more output. The goal is to ask smaller, more useful questions about the program's data and results.
From Overwhelming Output to Targeted Questions
When a program processes thousands or millions of rows, printing every value and checking the output by hand becomes impractical. Effective testing changes the question from “What does everything look like?” to “What specific fact do I need to verify?” You can make the dataset smaller, summarize important properties, inspect types, and add checks that identify impossible or inconsistent results.
The goal is not to inspect more output. The goal is to ask smaller, more useful questions about the program's data and results.
Shrinking the Debugging Cycle
Start by modifying the program so that it reads only the first n lines of the dataset. Use a small value of n while investigating the basic behavior. Once the program behaves correctly on that portion, increase n gradually and repeat the process. This keeps each debugging cycle fast and focused, and it helps isolate the dataset size at which the problem first appears.
Finding the First Failing Size
A program produces the expected result with a small input but fails when more data is included. How should the debugging process proceed?
- 1Begin with a small portion
Configure the program to read only the first n lines, using a small value so that each run remains quick to inspect.
- 2Increase gradually
Raise n in stages rather than jumping immediately to the full dataset.
- 3Record the change
When the behavior changes, note the dataset size at which the error first appears.
- 4Focus the investigation
Use the failing size as a smaller, more manageable target for further checks and debugging.
A gradual increase narrows the debugging problem to the point where the error becomes visible instead of requiring a full-dataset inspection.
Summaries That Answer Specific Questions
A full data dump often hides the information you actually need. Replace it with a compact summary chosen for a particular question. A length can show how many items are present. A sum can show the total of a list of values. A sample can give a quick look at representative values. A range can help reveal unexpectedly small or large values. A type check can show whether a value has the kind of data representation the program expects.
| Question | Useful inspection | What it tells you |
|---|---|---|
| How many items are present? | Length | The size of the collection |
| What is the total of the values? | Sum | The combined value of a list |
| What do some values look like? | Sample values | A limited view of the data |
| Are values unexpectedly small or large? | Range | The spread between observed values |
| Is a value represented as expected? | Type inspection | The type of the value |
Choose the smallest summary that answers the debugging question.
Suppose a generated debugging scenario produces an unexpectedly small result. Printing the entire dataset may create a large, difficult-to-scan output. Checking the dataset length, the total of its values, a few sample values, and the types of inspected values gives several focused ways to investigate the result.
Checks for Impossible and Inconsistent Results
A sanity check detects an impossible result. It asks whether an outcome makes sense on its own. A consistency check compares two computations to ensure that they agree. These checks turn logical questions into automatic signals instead of requiring you to discover every problem by manually reading output.
Choosing the Right Check
A program produces a result that needs to be examined automatically. Should the first check ask whether the result is possible or whether two computations agree?
- 1Ask whether the result is possible
Use a sanity check when the main question is whether the result violates an expectation about what could occur.
- 2Compare independent computations
Use a consistency check when the main question is whether two computations that should agree actually produce the same result.
- 3Respond to failure
Treat a failed check as a signal to investigate the relevant result instead of allowing the problem to remain hidden in ordinary output.
The choice depends on the question: sanity checks detect impossible results, while consistency checks compare computations for agreement.
Readable Debugging Output
Debugging output should make the important differences easy to scan. Add labels so that each value is identified, and use indentation to show related information. Pretty-printed output is easier to inspect for errors than raw, unformatted text because the reader can connect each value with its meaning and distinguish groups of information.
12 480 integer 3 7 9 14 510 integer 4 8 10
Values appear as an unstructured sequence.
batch: first
count: 12
total: 480
type: integer
batch: second
count: 14
total: 510
type: integerLabels and indentation make groups and meanings easier to scan.
Mistakes That Hide the Real Problem
Common mistake Printing the entire dataset for every debugging run
What it looks likeA large output is produced even when the question is only how many items were processed.
Why it failsLarge datasets cannot be meaningfully inspected line by line, and the relevant signal can disappear inside the output.
Do this insteadReduce the dataset first or print a targeted summary such as its length.
Common mistake Keeping the dataset at full size while trying to isolate an error
What it looks likeEvery run processes the complete input, even though the error may occur much earlier.
Why it failsThe debugging cycle becomes slower and less focused.
Do this insteadRead only the first n lines, then increase n gradually as the behavior becomes clearer.
Common mistake Using a summary without deciding what question it should answer
What it looks likeSeveral values are printed, but none shows the dataset size, total, sample, range, or type relevant to the suspected problem.
Why it failsOutput can remain large without providing useful evidence.
Do this insteadChoose the smallest summary that answers the current debugging question.
Common mistake Treating every check as the same kind of check
What it looks likeA comparison between two computations is described as checking whether a result is possible.
Why it failsSanity checks and consistency checks answer different logical questions.
Do this insteadUse a sanity check for impossible results and a consistency check when two computations should agree.
Common mistake Leaving debugging output unformatted
What it looks likeRelated values are printed in one unlabelled sequence.
Why it failsIt takes longer to determine what each value represents and which values belong together.
Do this insteadAdd labels and indentation to group related information.
Practice: Design a Debugging Pass
Imagine a program that processes a large dataset and produces an unexpected result. Design a short debugging plan. State how you would reduce the input size, which summaries or type checks you would inspect, which sanity or consistency check you would add, and how you would format the output.
Hints
- Begin with only the first n lines and identify how you will increase n.
- Choose summaries that answer specific questions rather than printing every value.
- Decide whether your logical check concerns an impossible result or agreement between computations.
- Use labels and indentation so that related values are easy to scan.
- 1Start with a small portion of the dataset.
- 2Increase the number of lines gradually as the behavior becomes clearer.
- 3Replace full output with targeted summaries and type inspection.
- 4Add sanity checks for impossible results and consistency checks for disagreements.
- 5Format the remaining debugging output with labels and indentation.
Key Takeaways
- 1Scale the dataset down first, then increase its size gradually to isolate when an error appears.
- 2Use lengths, sums, samples, ranges, and type checks to ask targeted questions without printing everything.
- 3Use sanity checks to detect impossible results and consistency checks to compare computations that should agree.
- 4Use labels and indentation to make debugging output easier to scan and interpret.
- 5Effective testing turns an overwhelming inspection task into a sequence of focused checks.
At a glance
- Start with a small dataset and increase its size gradually.
- Prefer targeted summaries and type checks to full data dumps.
- Use sanity checks for impossible results and consistency checks for disagreements.
- Format debugging output with labels and indentation.
- A focused debugging cycle makes large-data problems more manageable.