Sorting Dictionary Data

Use a dictionary to count word frequencies by checking if each word exists as a key and either initializing it to 1 or incrementing its count.

Programming

Interactive lab

Try it: Counting Words with a Dictionary

The py4e histogram pattern: split each line into words, count them with counts[word] = counts.get(word, 0) + 1, then find the most common word with a max loop or by sorting (count, word) tuples in reverse.

How it works

Optionally remove punctuation with str.translate and lower-case the line, then split() it into words.
For each word: counts.get(word, 0) returns the current count or 0 for a new word; store it plus one (or the if word not in counts form).
A dict remembers insertion order: a word's position is fixed the first time it is seen.
Max loop: keep the word whose count is strictly larger than the best so far (the first word wins a tie).
Or build [(count, word) …], sort with reverse=True (ties ordered by the word, reversed) and slice [:N].

Default run (25 steps): Start with an empty dict. The text has 3 lines. … [:3] keeps the top 3: [(5, 'the'), (2, 'sat'), (2, 'on')].

Simplified: Printable-ASCII text up to 320 characters, 8 lines and 60 words, typed in the lab instead of read from a file or URL.

Educational simulation

Loading the simulation…

Why Frequencies Matter

When you need to understand what is inside a file, one useful question is: how many times does each item appear? This is a frequency analysis problem. A dictionary is well suited to it because each unique item can be a key and its frequency can be the associated value. As the file is processed, the dictionary grows when a new item appears and changes an existing value when an item appears again.

First Dictionary Updates

Suppose a file contains the lines “the cat sat on the mat” and “the dog ran”. Split the lines into words and process the words from left to right. The first word, “the”, is not in the empty dictionary, so it becomes a key with value 1. When “the” appears again, it is already a key, so its value becomes 2. The same decision is made for every later word.

For each word, how does the program choose between adding a key and incrementing a count?

Worked example

Processing the first five words

Trace the dictionary while processing the words the, cat, sat, on, the.

1
Start
Begin with an empty dictionary: {}.
2
the
The key is new, so add the: 1.
3
cat
The key is new, so add cat: 1.
4
sat
The key is new, so add sat: 1.
5
on
The key is new, so add on: 1.
6
the again
The key already exists with value 1, so increase it to 2.

Result

{'the': 2, 'cat': 1, 'sat': 1, 'on': 1}

The Counting Algorithm

The algorithm starts with an empty dictionary. For each line in the file, split the line into words. Then process each word individually. Membership testing determines whether the word is already a dictionary key. A missing key receives value 1; an existing key has its value increased by 1. At the end, every unique word is paired with its frequency.

Code walkthrough

counts = {}

for line in file:
    words = line.split()
    for word in words:
        if word not in counts:
            counts[word] = 1
        else:
            counts[word] += 1

1
counts = {} creates the storage for unique words and their frequencies.
2
The outer loop supplies one line at a time from the file.
3
line.split() produces the words from the current line.
4
The inner loop handles each extracted word.
5
If the word is missing, counts[word] = 1 creates its first count.
6
Otherwise, counts[word] += 1 updates the existing frequency.

Create an empty dictionary for frequencies.
Read the file one line at a time.
Split each line into words.
Check whether the current word is already a key.
Initialize a new word with count 1.
Increment the count for a repeated word.

How does information move from an input file into a dictionary of counts?

Reading Dictionary State

Tracing means recording the dictionary after each item is processed. This makes the internal change visible: a first occurrence adds a key, while a repeated occurrence changes only that key's value. For the source example, the final dictionary contains each unique word and the number of times it appeared. The word “the” has value 2 because it occurs twice; the other words shown occur once.

How does the dictionary change after each successive word, including a repeated word?

Debugging

If a word has an unexpected count, print the dictionary inside the loop after each word is processed. The trace can show whether the word was never encountered, whether it was treated as a new key, or whether its existing value was incremented incorrectly. Once the logic is confirmed, remove the debug print statements.

Structured File Histograms

The same counting pattern works when the input is structured rather than ordinary prose. A mail log can contain lines beginning with “From”, followed by an email address, while the day appears in a predictable position. The program can filter for lines that start with “From”, split those lines into fields, extract the day from the third word, and count each day in a dictionary.

How are structured lines filtered, split, interpreted, and assigned to a category?

python

counts = {}

for line in file:
    if line.startswith('From'):
        words = line.split()
        day = words[2]
        if day not in counts:
            counts[day] = 1
        else:
            counts[day] += 1

counts = {}

for line in file:
    if line.startswith('From'):
        words = line.split()
        day = words[2]
        if day not in counts:
            counts[day] = 1
        else:
            counts[day] += 1

The dictionary logic has not changed. Only the item being counted has changed: instead of every word, the program counts extracted days. The source describes a result in which Friday has 20 messages, Thursday has 6, and Saturday has 1. The resulting dictionary is a histogram because its keys are categories and its values are frequencies.

Other Frequency Categories

A mail log can also be used to count email addresses. For each line beginning with “From”, extract the email address and use it as the dictionary key. The value records how many messages came from that sender. The source example reports that cwen@iupui.edu appears 5 times, while some other senders appear once.

How are dictionary keys and frequency values interpreted as histogram categories and bar heights?

Item being counted	Dictionary key	Dictionary value
Words	Each unique word	Number of appearances
Mail days	Each extracted day	Number of messages on that day
Email senders	Each extracted email address	Number of messages from that sender

The counting operation stays the same while the extracted item changes.

Mistakes in Frequency Counting

Common mistake
Incrementing a key before giving it an initial value
What it looks like
Using counts[word] += 1 when word is not yet in counts
Why it fails
A new word does not yet have a stored count to increment.
Do this instead
Check membership first. Add counts[word] = 1 for a new key, and increment only when the key already exists.
Common mistake
Counting every line in a structured log
What it looks like
Treating all mail-log lines as message records
Why it fails
Only lines with the required format should contribute to the selected category.
Do this instead
Filter lines by their format, such as checking whether a line starts with “From”, before extracting fields.
Common mistake
Extracting the wrong field
What it looks like
Using a field other than the predictable day position
Why it fails
The dictionary would count a different piece of text from the intended category.
Do this instead
Split the line and use the documented predictable position, such as words[2] for the day in the source example.
Common mistake
Skipping a state trace when the result looks wrong
What it looks like
Inspecting only the final dictionary
Why it fails
The final result does not reveal when a key was missed or incremented incorrectly.
Do this instead
Print the dictionary after each word or extracted item to locate the first incorrect update.

Practice the Pattern

MEDIUM

A file contains several lines of text. Write the counting logic that produces a dictionary of word frequencies. Then trace the dictionary after processing the words “red”, “blue”, “red”, and “red”.

Hints

Start with an empty dictionary.
Use a membership check for each word.
The first “red” creates a key with value 1.
The second and third “red” increment the existing value.

Your attempt (not saved)

Worked example

Practice trace result

Determine the final dictionary after processing red, blue, red, red.

1
red
The key is new, so its count is 1.
2
blue
The key is new, so its count is 1.
3
red again
The existing red count increases from 1 to 2.
4
red once more
The existing red count increases from 2 to 3.

Result

{'red': 3, 'blue': 1}

Key Takeaways

1
A frequency dictionary stores each unique item as a key and its occurrence count as the value.
2
The essential decision is whether the current item is already a key: initialize it to 1 if it is new, or increment it if it exists.
3
Tracing the dictionary after each update reveals when keys are added and when counts change.
4
Structured files can be counted by filtering lines, extracting a predictable field, and applying the same dictionary pattern.
5
Words, days, and email addresses use the same counting logic; only the extracted item changes.

At a glance

Use dictionaries to map each unique item to its frequency.
Initialize a missing key with 1 and increment an existing key.
Trace dictionary state to debug counting logic.
Filter and parse structured records before counting an extracted category.
The same pattern builds histograms for words, days, and email addresses.

Related concepts

More concepts starting with S