Concepts / Building Dictionaries from Text

Building Dictionaries from Text

The complete workflow begins with a text file.

  • Programming

From Words to Rankings

A word-frequency dictionary answers how often each word appears, but it does not directly answer which words are most common. The dictionary uses words as keys and counts as values. To rank the words by frequency, restructure each entry so the count comes first. This changes the sorting problem from comparing words first to comparing counts first.

word-frequencydictionaryword → countthe5(5, the)count → wordto5(5, to)count → wordentryrestructureentryrestructure
How does each word-frequency entry change when the count becomes the first tuple element?

Counting Words First

The complete workflow begins with a text file. As words are read, the program builds a dictionary: each word becomes a key, and its frequency becomes the associated value. The first encounter with a word establishes its count, while later encounters increase that count. After the dictionary is complete, the program creates a separate list of tuples, placing each count before its word. The original dictionary is not modified.

empty dictionary{}wordcount establishedsame wordcount increasedfrequency dictionarywords → countsfirst encounterencounter againcontinue reading
How does the dictionary change when a word appears for the first time and when it appears again?
Worked example

A Small Frequency Dictionary

Suppose the completed dictionary contains the entries the: 5, to: 5, and: 4.

  1. 1
    Read the dictionary

    The keys are words and the values are their counts.

  2. 2
    Create tuples

    Rearrange each entry so the count is first: (5, the), (5, to), and (4, and).

  3. 3
    Prepare for sorting

    Because the count is the first tuple element, it becomes the primary value used during sorting.

Result

The new list contains sortable count-and-word tuples while the original dictionary remains separate.

Reverse Sorting and Ties

Python compares tuple elements from left to right. With count-and-word tuples, the count is compared first. Calling sort with reverse=True places larger counts before smaller counts. If two tuples have the same count, Python compares their second elements, the words. Because the whole sort is reversed, the alphabetically later word comes first among tied entries.

(8, word)higher count(5, to)tie: word lateralphabetically(5, the)tie: word earlieralphabetically(4, word)lower countcount descendsequal count; word descendscount descends
After reverse sorting, how are equal-frequency words ordered?
Worked example

Resolving an Equal-Frequency Tie

Compare the tuples (5, the) and (5, to) when the list is sorted in reverse order.

  1. 1
    Compare the first elements

    Both first elements are 5, so neither tuple wins on frequency.

  2. 2
    Compare the second elements

    Python compares the words. The word to comes later alphabetically than the word the.

  3. 3
    Apply reverse ordering

    Reverse ordering places the alphabetically later tied word first.

Result

The tied entries appear as (5, to), then (5, the).

Selecting the Top Results

Once the tuple list is sorted, list slicing selects the results to display. The slice lst[:10] starts at index 0 and stops before index 10, selecting indices 0 through 9. In a loop, tuple unpacking assigns the first tuple element to count and the second to word. Changing the slice limit changes how many leading results are displayed without changing the sorted list itself.

index 0(12, alpha)index 1(10, beta)index 2(9, gamma)index 3(8, delta)slice boundaryN = 3next resultnext resultstop before index 3not selected
Which portion of the sorted tuple list is selected by a top-N slice?
(61, i)one selected resultcount61display resultcount and wordwordifirst elementsecond elementformatformat
How does each two-item tuple become separate count and word values?
Worked example

Changing N

A sorted list contains twelve count-and-word tuples. Compare a top-3 selection with a top-10 selection.

  1. 1
    Use the top-3 slice

    The first three tuples, at indices 0, 1, and 2, are selected.

  2. 2
    Use the top-10 slice

    The first ten tuples, at indices 0 through 9, are selected.

  3. 3
    Keep the ranking unchanged

    Both selections come from the same sorted list; only the endpoint of the slice changes.

Result

The slice controls the number of displayed leading results.

Following the Full Workflow

text fileinput wordsfrequency dictionaryword → countcount-word tuples(count, word)reverse-sorted listhighest counts firsttop-N sliceleading resultsranked outputcount and wordcount wordsrestructuresort reverseselect Nunpack and display
How does data move through the complete word-frequency workflow?

The full program combines several separate operations. It reads a text file, counts the words into a dictionary, converts the dictionary entries into count-and-word tuples, sorts those tuples in reverse order, selects the leading results, and displays each selected count with its word. The sorting stage ranks by frequency, while the tuple's second element provides deterministic ordering for ties.

In the real world

In the Romeo and Juliet text example, the output lists the ten most frequent words with their counts. The word i appears 61 times and and appears 42 times. These results are ordered from the highest frequency downward because the count was placed first in each tuple before reverse sorting.

Best practice

Keep the stages conceptually separate: build the dictionary, create the sortable tuple list, sort it, select the desired prefix, and unpack the selected tuples for display. This makes the transformation easy to trace and leaves the original dictionary unchanged.

Mistakes to Avoid

  • Common mistake

    Leaving the word first in each tuple.

    What it looks like
    (word, count)
    Why it fails

    Python compares the first tuple element first, so the words become the primary sorting values instead of the frequencies.

    Do this instead

    Use (count, word) so frequency controls the main ordering.

  • Common mistake

    Expecting equal-frequency words to keep an arbitrary order.

    What it looks like
    (5, the) and (5, to)
    Why it fails

    Python compares the second tuple elements when the counts match.

    Do this instead

    Expect reverse alphabetical ordering for tied words when reverse sorting is used.

  • Common mistake

    Reading lst[:10] as ten through twenty.

    What it looks like
    lst[:10]
    Why it fails

    The slice begins at index 0 and stops before index 10.

    Do this instead

    Interpret it as indices 0 through 9.

  • Common mistake

    Reversing the unpacking order.

    What it looks like

    Assigning the word to count and the count to word.

    Why it fails

    The tuple is organized as count first and word second.

    Do this instead

    Assign the first element to count and the second element to word.

  • Common mistake

    Assuming sorting changes the original dictionary into a ranked dictionary.

    What it looks like

    Treating the tuple list as the original dictionary.

    Why it fails

    The tuple list is a new sortable data structure.

    Do this instead

    Remember that the dictionary and the sorted list are separate structures.

Apply the Pattern

MEDIUM

A frequency dictionary contains the entries alpha: 7, beta: 7, gamma: 10, and delta: 3. Describe the count-and-word tuple list, its order after reverse sorting, and the entries selected by a top-2 slice.

Hints
  • Put each count before its word.
  • Place the largest count first.
  • For the two entries with count 7, compare alpha and beta as the secondary values.
  • A top-2 slice selects indices 0 and 1.

What do you think happens?

Which tied entry appears first after reverse sorting: (5, the) or (5, to)?

  • (5, the)

  • (5, to)

  • They remain in dictionary order

Reveal answer
Answer:

(5, to)

The counts tie, so Python compares the words. Since to comes later alphabetically than the, reverse sorting places to first.

Key Takeaways

  1. 1
    A word-frequency dictionary stores words as keys and counts as values.
  2. 2
    Convert entries to (count, word) tuples so sorting uses frequency as the primary criterion.
  3. 3
    Reverse tuple sorting places larger counts first and uses the word as a reverse-alphabetical tiebreaker.
  4. 4
    The slice lst[:10] selects the first ten sorted tuples, while tuple unpacking separates each count from its word.
  5. 5
    The complete workflow moves from file input to counting, tuple conversion, sorting, slicing, and formatted output.

At a glance

  • Restructure dictionary entries as (value, key), or specifically (count, word), before sorting.
  • Reverse sorting compares counts first and uses words to resolve equal-count ties.
  • Use a top-N slice to select only the leading results.
  • Use tuple unpacking to handle each count and word separately during output.
  • The pattern provides deterministic word-frequency rankings without changing the original dictionary.

Related Topics

Related concepts

More concepts starting with B