How I consolidated 736,000 family emails from 236 backup files

By Michael, founder of EmailAttic · September 2026 · 12 min read

Twenty-five years of family email doesn't live anywhere. It lives everywhere: five email accounts, every computer we'd owned since the late nineties, and a network drive quietly hoarding backups of backups. This is the story of merging all of it into one clean archive — and the trap along the way that would have silently thrown away thirteen thousand messages.

The starting point: a drawer, a NAS, and dread

Like most families that lived through the Outlook era, ours "backed up email" by copying PST files somewhere and hoping. When I finally decided to deal with it, the inventory was sobering: eighteen PST backups in one folder, a couple of zipped mail archives from an old server, some loose exports — and, on the family NAS, 218 more PST files, many of them backups of the same mailbox taken years apart, some over 40 GB, one of them damaged.

Every file was a question mark. Which ones overlapped? Which held mail that existed nowhere else? Which were safe to delete? Nobody deletes a file called terri-mail-backup-FINAL2.pst without an answer, which is why the drawer only ever grows.

Step 1: Find everything, and copy it local first

The first real task is a full inventory: every .pst, .mbox, .eml folder, mail-shaped .zip, and forgotten export, across every drive. Two hard-learned rules:

• Scan the whole drive, then verify against a second scan. The files you forget exist are exactly the ones holding the mail nothing else has.
• Copy to a local disk before opening anything. Reading a PST is random access, and random reads over a network share are catastrophically slow. Copy first, extract locally. This one rule saved days.

Step 2: Deduplicate the containers

236 files sounds impossible until you realize most are copies of each other. Hashing the files by content collapsed the 218 NAS files to 57 byte-distinct ones. That's the first big win: a third of the problem evaporates before you open a single mailbox.

Step 3: Extract everything to a common format

PSTs come in two incompatible flavors — ANSI (older Outlook) and Unicode — and plenty of tools quietly handle only one. Some files needed one extractor, some another, and Outlook's own repair tool (scanpst) had to rescue a damaged 49 GB monster (it turned out to be essentially lossless — the repaired copy gave up only a few dozen messages). Everything got extracted to individual .eml files: the plainest, most future-proof format email has.

Step 4: The trap — "redundant" files that weren't

If I had trusted appearances and deleted everything that looked redundant, thirteen thousand family emails would have vanished without any error, warning, or way to know. The rule that follows: container-level dedupe is for skipping identical files only. Anything merely similar gets opened and extracted anyway. Let the message-level dedupe sort it out.

Step 5: Deduplicate the messages

With everything as .eml, deduplication is honest work: match on the Message-ID header when it exists, fall back to a content hash when it doesn't (some old mail and calendar items have no Message-ID). 736,312 extracted messages collapsed to roughly half — every message kept exactly once, attachments intact, sent and received together.

Step 6: Organize by person, not by source file

Nobody browsing the archive cares which backup a message came from. The structure that works is by person, then by address: each family member's folder holding all their mail across every era, with business and personal tiers separated. Contacts, calendars, and notes extracted from the PSTs went into a separate non-mail folder per person, so nothing was lost but nothing clutters the mail.

Step 7: Prove it, then (and only then) clean up

The point of the whole exercise is being allowed to delete the drawer. That takes receipts:

• Per-source counts: how many messages each file contained, how many were unique, how many were duplicates of what.
• A list of anything unreadable, with a reason.
• Source files zipped with verification (and, for the irreplaceable ones, archives with recovery records), tested before any original was touched.

Only after the numbers reconciled did the loose copies get deleted. The originals on the NAS were never deleted at all — just backed up properly. Paranoia is the correct disposition around family history.

What I'd tell anyone starting this

1. Inventory first; open nothing over the network.
2. Hash-dedupe the files, but only skip exact duplicates.
3. Extract everything — especially the files that look redundant.
4. Dedupe at the message level, by Message-ID with a content-hash fallback.
5. Organize by person. Plain .eml files in plain folders outlive every app.
6. Demand a reconciliation report from yourself before deleting anything.