Introduction to gmodel.py and Data Processing
The goal is not to erase historical addresses. The goal is to make them equivalent during processing by directing them to one chosen destination address.
One Person, Several Addresses
A person's email address can change when they move between organizations. If an archive treats every historical address as unrelated, one contributor may appear to be several different people. gmodel.py addresses this problem through mappings stored in mapping.sqlite. The Mapping table handles individual email addresses, while the DNSMapping table handles domain-name changes.
The goal is not to erase historical addresses. The goal is to make them equivalent during processing by directing them to one chosen destination address.
Following a Career Across Addresses
Consider the source example of Steve Githens. His archive contains three email addresses associated with different stages of his career. The mapping design chooses swgithen@mtu.edu, his most recent address, as the canonical destination. The other two addresses point toward it.
Steve Githens' Three Addresses
Consolidate three historical email addresses into the canonical address swgithen@mtu.edu.
- 1Choose the destination
Use swgithen@mtu.edu as the canonical destination because it is the most recent address in the source example.
- 2Add the first mapping
Create an entry directing the Northwestern address to swgithen@mtu.edu.
- 3Add the second mapping
Create an entry directing the Cambridge address to swgithen@mtu.edu.
- 4Leave the destination unmapped
Do not add an entry directing swgithen@mtu.edu to itself. It is already in its final form.
Two Mapping table entries consolidate the three addresses. Whenever gmodel.py encounters either historical address, it uses swgithen@mtu.edu instead; an occurrence of the canonical address is already usable as-is.
Mapping Table Rules
The Mapping table stores individual email mappings using arrow notation: source address to destination address. The source is the address that may appear in historical data. The destination is the canonical address that gmodel.py should use when it encounters that source. Multiple source addresses can point to one destination.
Common mistake Adding a mapping entry for the canonical address pointing to itself.
What it looks likeswgithen@mtu.edu to swgithen@mtu.edu
Why it failsThe canonical address is already in its final form and does not require a mapping entry.
Do this insteadAdd entries only for the historical source addresses that should point to the canonical destination.
Common mistake Adding only one historical address when two historical addresses must be consolidated.
What it looks likeMapping the Northwestern address but omitting the Cambridge address.
Why it failsThe omitted address will not match a Mapping table entry and will be used as-is.
Do this insteadCreate one entry for each noncanonical address that should be consolidated.
Processing Order
gmodel.py applies mappings while processing the archive. It reads each message, extracts sender and recipient addresses, and checks each address against the Mapping table. When a mapping exists, it uses the destination address. When no mapping exists, it keeps the address as-is. A single mapping can therefore affect many messages if the address appears repeatedly.
The mapping is a transformation rule, not an edit to each stored message. The same rule is checked each time the relevant address is encountered during processing.
Domain-Level Normalization
The DNSMapping table works at the domain level rather than the individual-address level. For example, when gmodel.py processes someone@iupui.edu, it can first check whether iupui.edu maps to indiana.edu. If that DNS mapping exists, the address becomes someone@indiana.edu before individual email address mappings are checked.
| Mapping system | What it handles | When it operates |
|---|---|---|
| DNSMapping | Domain-name variations | First |
| Mapping | Individual email-address variations | After DNS mapping |
The processing order matters: domain mapping happens before individual address mapping.
Persisting a Mapping Entry
A mapping entry moves through three related places: the mapping decision is defined for the application, it is stored in mapping.sqlite, and gmodel.py consults it during archive processing. The important distinction is that storage of the rule and transformation of archive values occur during data processing; the original archive remains unchanged.
Treat mappings as adjustable transformation rules. If later review shows that a source address was assigned to the wrong destination, change or remove the mapping rather than altering the original archive.
Archive Cleanup Practice
An archive contains three addresses for one contributor: a Northwestern address, a Cambridge address, and swgithen@mtu.edu. The goal is to treat all three as one canonical address. Decide which address should be the destination and determine how many Mapping table entries are needed.
Hints
- Use the most recent address named in the source example as the destination.
- Count the historical addresses that are not already the destination.
- The destination does not need a self-mapping.
An address appears as someone@iupui.edu, and the DNSMapping table contains a rule mapping iupui.edu to indiana.edu. Describe the processing order, including when an individual Mapping table lookup occurs.
Hints
- Start with the domain portion of the address.
- DNSMapping operates before individual email-address mappings.
- After the domain is normalized,
gmodel.pycan check the resulting address against Mapping.
Key Takeaways
- 1The Mapping table connects historical individual email addresses to one canonical destination.
- 2For three addresses consolidated into one destination, only the two noncanonical addresses need mapping entries.
- 3DNSMapping handles domain-level normalization before individual email-address mappings are applied.
- 4
gmodel.pyapplies mappings during processing and leaves the original archive unchanged. - 5Together, email and DNS mappings create a more unified view of contributors across organizational and career changes.
At a glance
- Use the Mapping table to direct multiple historical email addresses to one canonical address.
- Do not create a self-mapping for the canonical address.
- Use DNSMapping first when the inconsistency is at the domain level.
- Mappings are non-destructive rules applied by
gmodel.pyduring data processing. - Reviewing and adjusting mappings improves the consistency of large email archives without changing their original data.