Sorting and Ordering Data Structures
Sorting makes the result easier to search, verify, and understand because the output follows a consistent alphabetical arrangement.
From Text to Vocabulary
Imagine trying to discover every distinct word used across thousands of pages by reading the pages and tracking each new word manually. A program can perform this task as a sequence of transformations: read the file, split each line into words, keep only words that have not already been collected, and sort the final list alphabetically. The important idea is not one isolated list operation. It is the movement of data through a pipeline, where the data changes form at each stage.
Tracking the Growing List
The unique-word list grows one word at a time. When a word is extracted from a line, the program checks whether that word is already in the list. If the word is not present, it is added. If the word is already present, the program skips it. This check-before-adding pattern is the exact point where duplicates are filtered.
A Small Membership Trace
Trace the unique list while processing the words the, sun, the, moon.
- 1Process the
The list is empty, so the is not in it. Add the.
- 2Process sun
sun is not in the current list. Add sun.
- 3Process the again
the is already in the list. Skip it, so the list does not gain another copy.
- 4Process moon
moon is not in the current list. Add moon.
The resulting unique list is [the, sun, moon] before sorting.
Reading and Splitting Lines
Before membership testing can occur, the program must turn each line into separate word strings. The split() function performs this transformation. When called on a string without arguments, split() breaks the string at whitespace boundaries such as spaces, tabs, and newlines, returning a list of the resulting pieces. This changes the program's view of the data from one complete line to several individual words that can be processed one by one.
line = "the sun rises"
words = line.split()
print(words)- 1The string assigned to line contains three words separated by whitespace.
- 2Calling
split()without arguments separates the line at its whitespace boundaries. - 3The value assigned to words is a list containing the three resulting strings.
- line stores one complete line as a string.
split()creates a list of separate word strings.- The resulting list can be processed word by word.
["the", "sun", "rises"]Sorting the Final Collection
After every line has been read and every extracted word has passed through the membership check, the program sorts the remaining list alphabetically. Without this final operation, the list reflects the order in which words were first encountered. Sorting makes the result easier to search, verify, and understand because the output follows a consistent alphabetical arrangement.
["moon", "sun", "the"]Complete File Workflow
unique_words = []
with open("words.txt") as file:
for line in file:
words = line.split()
for word in words:
if word not in unique_words:
unique_words.append(word)
unique_words.sort()
print(unique_words)- 1Create an empty list for unique words.
- 2Open the text file and iterate through its lines.
- 3Split the current line into individual word strings.
- 4Check each word against the collection already built.
- 5Append only words that are not already present.
- 6After the entire file has been processed, sort the list.
- 7Display the sorted list.
unique_wordsstarts as the collection that will hold one copy of each word.- The file is processed line by line.
split()changes each line into separate word strings.- The membership test filters out words already in
unique_words. sort()orders the completed collection before it is displayed.
The code follows the conceptual sequence directly. Reading happens before splitting, splitting happens before membership testing, and sorting happens only after the file's lines have been processed. This order matters: sorting an incomplete collection would not produce the vocabulary of the entire file, and adding every extracted word without a membership check would preserve duplicates.
For romeo.txt, the expected result is a list of 26 unique words sorted alphabetically: Arise, But, It, Juliet, Who, already, and, breaks, east, envious, fair, grief, is, kill, light, moon, pale, sick, soft, sun, the, through, what, window, with, yonder. The placement of Arise and But before already reflects the ordering of uppercase letters before lowercase letters.
Debugging Divergent Output
When the output differs from what you expect, trace the collection after each stage rather than looking only at the final print statement. Ask which lines were read, what split() produced, which words passed the membership test, and whether sorting occurred after the complete collection was built. The first stage where the state differs from the expected state identifies the likely source of the problem.
Common mistake Adding every extracted word without checking membership
What it looks likeAppending word immediately instead of testing whether word is already in the unique list.
Why it failsRepeated words enter the collection multiple times, so the result is not a list of unique words.
Do this insteadCheck if the word is not in the list before appending it.
Common mistake Forgetting to sort the completed list
What it looks likePrinting the collection immediately after reading the file.
Why it failsThe output reflects first-appearance order rather than alphabetical order.
Do this insteadSort the final list after all lines and words have been processed.
Common mistake Not reading the entire file
What it looks likeProcessing only part of the file before sorting.
Why it failsWords from unread lines cannot appear in the result.
Do this insteadTrace the file-reading loop and confirm that all lines are processed.
Common mistake Ignoring case sensitivity or punctuation
What it looks likeExpecting uppercase and lowercase words to be treated or ordered identically without addressing that behavior.
Why it failsCase affects the displayed order, and punctuation can affect the extracted strings.
Do this insteadInspect the actual word strings produced during extraction and compare them with the expected output.
for word in words:
unique_words.append(word)Every extracted word is appended, including repeated words.
for word in words:
if word not in unique_words:
unique_words.append(word)Only a word not already in the collection is appended.
Practice the Trace
Suppose two lines produce these word sequences: ["bright", "moon"] and ["moon", "bright", "night"]. Trace the unique list after each word, then determine the final sorted list.
Hints
- Start with an empty list.
- For each word, decide whether it is already present before adding it.
- Apply sorting only after all words from both lines have been processed.
Practice Check
Find the unique sorted result for bright, moon, moon, bright, night.
- 1Filter duplicates
The first bright and moon are added. The second moon and second bright are already present and are skipped. night is added.
- 2Sort
Arrange the remaining words alphabetically after the membership checks are complete.
The final sorted list is [bright, moon, night].
Pipeline Takeaways
- 1Read the file and process its lines.
- 2Use
split()to transform each line into separate word strings. - 3Use membership testing before appending so duplicates are excluded.
- 4Sort the completed unique list alphabetically.
- 5Debug by comparing the collection's state after reading, splitting, filtering, and sorting.
At a glance
- Unique-word extraction is a pipeline that moves from file text to lines, words, unique words, and finally sorted output.
split()creates a list of separate word strings from a line.- The condition word not in the unique list is the point where duplicates are filtered.
- Sorting after the complete file has been processed makes the vocabulary easier to search, verify, and understand.
- Tracing the collection after each stage reveals whether a problem comes from extraction, duplicate filtering, incomplete reading, case, punctuation, or sorting.