Sorting and Ordering Data

The central pattern is counting first and sorting second. A dictionary provides the word-to-frequency map, while a list of tuples provides a sequence that can be ordered by frequency.

Programming

Interactive lab

Try it: Sorting with key= and reverse=

How sorted() and list.sort() compute key(item) once per item, order the items by those keys with Python's comparison rules, and keep equal keys in their original order — including py4e's sort-a-dict-by-value pattern.

How it works

Decorate: call the key function once on every item (no key means the item itself is compared).
Compare keys with Python's rules: ints by value, strs by code point ('Z' < 'a'), tuples item by item.
Order the items by key; reverse=True puts the largest key first.
Stable: items with equal keys keep their original relative order, also with reverse=True.
sorted() returns a new list; list.sort() rearranges the list in place and returns None; [:N] keeps the first N.

Default run (12 steps): 5 items in their original order. Python will call lambda t: t[1] once per item. … sorted() returned a new list: result = [('fig', 5), ('kiwi', 5), ('pear', 3), ('apple', 3), ('date', 1)].

Simplified: Up to 10 items of one kind (ints, short ASCII words, (word, count) tuples or a small dict) and a fixed menu of key functions. The sorting algorithm itself (Timsort) is not shown — only what it guarantees: the order by key and stability.

Educational simulation

Loading the simulation…

Interactive lab

Try it: Counting Words with a Dictionary

The py4e histogram pattern: split each line into words, count them with counts[word] = counts.get(word, 0) + 1, then find the most common word with a max loop or by sorting (count, word) tuples in reverse.

How it works

Optionally remove punctuation with str.translate and lower-case the line, then split() it into words.
For each word: counts.get(word, 0) returns the current count or 0 for a new word; store it plus one (or the if word not in counts form).
A dict remembers insertion order: a word's position is fixed the first time it is seen.
Max loop: keep the word whose count is strictly larger than the best so far (the first word wins a tie).
Or build [(count, word) …], sort with reverse=True (ties ordered by the word, reversed) and slice [:N].

Default run (25 steps): Start with an empty dict. The text has 3 lines. … [:3] keeps the top 3: [(5, 'the'), (2, 'sat'), (2, 'on')].

Simplified: Printable-ASCII text up to 320 characters, 8 lines and 60 words, typed in the lab instead of read from a file or URL.

Educational simulation

Loading the simulation…

From Text to Top Words

Imagine a text file containing thousands of words. Printing every word count would produce a long, unsorted result, so the useful question is not only how often each word appears, but which words appear most often. The solution has two phases: first count each word, then organize the counts so the largest frequencies rise to the top.

Key point

The central pattern is counting first and sorting second. A dictionary provides the word-to-frequency map, while a list of tuples provides a sequence that can be ordered by frequency.

What does each word represent in a word-frequency dictionary, and where is its changing frequency stored?

Building the Frequency Dictionary

The counting phase reads the file line by line. Each line is processed by removing punctuation and converting it to lowercase, and then the program examines each word. For a word that is not yet in the dictionary, the program creates an entry with a count of 1. For a word that is already present, the program increases its existing count by 1.

What happens to the dictionary when a word is new or when it has already been counted?

Code walkthrough

counts = {}

for word in words:
    if word not in counts:
        counts[word] = 1
    else:
        counts[word] = counts[word] + 1

1
Start with an empty dictionary named counts.
2
Examine each word in the sequence named words.
3
If the word is not already a key, store the word with a frequency of 1.
4
If the word is already a key, replace its frequency with the old frequency plus 1.

Worked example

Tracing Repeated Words

Track the dictionary while the words love, hope, love are read.

1
Read love
The word is new, so add love with a frequency of 1.
2
Read hope
The word is new, so add hope with a frequency of 1.
3
Read love again
The word already exists, so increase its frequency from 1 to 2.

Result

The resulting frequency map is love mapped to 2 and hope mapped to 1.

Reversing the Pair for Sorting

After counting is complete, the dictionary contains each unique word as a key and its frequency as the value. To sort by frequency, take each word-count pair and create a tuple with the count first and the word second. This changes a pair shaped like word, count into count, word.

How does data change from dictionary entries shaped like word and count into tuples shaped like count and word?

python

items = []

for key, val in counts.items():
    items.append((val, key))

items = []

for key, val in counts.items():
    items.append((val, key))

Tuples are ordered and immutable sequences. They are suitable here because each result has a fixed two-part structure: frequency followed by word. When tuples are compared for sorting, the first element is considered first, and the second element can be considered when the first elements tie.

Ordering by Frequency

How does sorting by the frequency value rearrange the tuples compared with sorting alphabetically by the word?

Sort the list of count-word tuples in reverse order. Because the frequency is the first element, reverse sorting places the largest frequencies at the beginning. The list now represents the document ordered from the most frequent words to the least frequent words.

python

items.sort(reverse=True)

items.sort(reverse=True)

Worked example

Sorting Three Word Counts

Order the tuples (2, love), (5, hope), and (3, peace) by frequency from highest to lowest.

1
Inspect the first elements
The frequencies are 2, 5, and 3. Each frequency is first in its tuple.
2
Sort in reverse order
The largest first element is 5, followed by 3, then 2.
3
Keep each pair together
The word remains attached to its frequency while the tuples move.

Result

The ordered list is (5, hope), (3, peace), (2, love).

Selecting the Most Common Words

Once the tuple list has been sorted, select the first N entries with a slice. For the ten most common words, use lst[:10]. The resulting slice contains up to ten tuples, and a loop can print each tuple's frequency and word.

How do the highest-frequency tuples move to the beginning of the sorted list, and how are they selected?

Code walkthrough

for val, key in items[:10]:
    print(key, val)

1
Take the first ten tuples from the sorted list.
2
Unpack each tuple into val, the frequency, and key, the word.
3
Print the word and its frequency.

Complete Two-Phase Pattern

The complete solution keeps the jobs separate. The first phase reads and counts words. The second phase changes the dictionary entries into frequency-first tuples, sorts them in reverse order, and prints the requested number of results.

python

counts = {}

for line in file:
    line = line.lower()
    for word in line.split():
        if word not in counts:
            counts[word] = 1
        else:
            counts[word] = counts[word] + 1

items = []
for key, val in counts.items():
    items.append((val, key))

items.sort(reverse=True)

for val, key in items[:10]:
    print(key, val)

counts = {}

for line in file:
    line = line.lower()
    for word in line.split():
        if word not in counts:
            counts[word] = 1
        else:
            counts[word] = counts[word] + 1

items = []
for key, val in counts.items():
    items.append((val, key))

items.sort(reverse=True)

for val, key in items[:10]:
    print(key, val)

Phase	Data structure	Purpose
Counting	Dictionary	Map each word to its frequency
Transformation	List of tuples	Place frequency first so sorting uses it
Ordering	Sorted list of tuples	Place the largest frequencies first
Selection	Slice of the sorted list	Extract the top N results

The responsibilities of each stage in the word-frequency pipeline.

Mistakes with Frequency Sorting

Common mistake
Keeping the word first in each tuple
What it looks like
```
(word, count)
```
Why it fails
Sorting compares the first tuple element first, so the word becomes the primary sorting value rather than the frequency.
Do this instead
Create tuples in the order (count, word).
Common mistake
Sorting without reverse order
What it looks like
```
items.sort()
```
Why it fails
The highest frequencies will not be placed at the beginning by the required descending-frequency process.
Do this instead
Sort the frequency-first tuples in reverse order.
Common mistake
Reading the tuple fields in the wrong order
What it looks like
```
for key, val in items[:10]
```
Why it fails
The transformed tuple stores the frequency first and the word second.
Do this instead
Unpack as val, key and print the word together with its frequency.
Common mistake
Assuming the top-ten slice always contains ten entries
What it looks like
items[:10] on a document with fewer than ten unique words
Why it fails
A short document may produce fewer than ten tuples.
Do this instead
Use the slice directly; it returns all available tuples without error.

Practice the Pipeline

EASY

Suppose the frequency dictionary contains the entries love mapped to 47, hope mapped to 31, and peace mapped to 18. Write the list of tuples produced when each entry is changed to frequency-first order, then write the result after reverse sorting. Finally, identify the result of taking the first two entries.

Hints

Put each frequency before its word.
Reverse sorting places 47 before 31 and 18.
The first two entries are selected with a slice equivalent to the first two positions.

Your attempt (not saved)

1
A correct solution should produce the frequency-first tuples (47, love), (31, hope), and (18, peace). They are already ordered from highest to lowest frequency, so selecting the first two returns (47, love) and (31, hope).

Key Takeaways

1
Use a dictionary with words as keys and frequencies as values to count occurrences.
2
For every word, initialize its count to 1 when it is new and increment the count when it already exists.
3
Convert each word-count pair into a frequency-word tuple so sorting compares the frequency first.
4
Sort the tuple list in reverse order to place the most frequent words at the beginning.
5
Slice the sorted list to select the top N words, including fewer than N when fewer results are available.

At a glance

Word-frequency analysis separates counting from ordering.
A dictionary efficiently maps each word to its changing frequency.
Frequency-first tuples make the value, rather than the key, the primary sorting criterion.
Reverse sorting puts the most common words at the beginning of the list.
Slicing the sorted list extracts the requested top results safely.

Related concepts

More concepts starting with S

Try it: Sorting with key= and reverse=

Try it: Counting Words with a Dictionary

From Text to Top Words

Building the Frequency Dictionary

Reversing the Pair for Sorting

Ordering by Frequency

Selecting the Most Common Words

Complete Two-Phase Pattern

Mistakes with Frequency Sorting

Practice the Pipeline

Key Takeaways

At a glance

Related Topics

Related concepts