Email Service System Design

An email service stores, sends, receives, indexes, and presents electronic mail messages on behalf of users. Despite popular forecasts of its demise, email remains the universal asynchronous-communication substrate of the internet: roughly 4.4 billion users globally and on the order of 360 billion messages sent per day as of 2024 (Radicati Group figures via Statista), with tens of years of accumulated mailbox history per heavy user. The dominant industrial deployments are Gmail (~1.8B accounts, per Google’s public statements), Outlook / Microsoft 365 (~400M), Yahoo Mail (declining but still significant), ProtonMail (privacy-focused, end-to-end encrypted within its network, 100M+ Proton accounts as of 2024), Hey (Basecamp’s opinionated reinvention), and tens of thousands of self-hosted Postfix/Sendmail/Exim deployments running on private and corporate domains. Architecturally, email is unique among messaging systems because it is a federated, store-and-forward protocol — your mail server must accept mail from arbitrary servers run by arbitrary entities, deal with their varying reliability and trustworthiness, fight spam from a global open network, and present years of historical content with sub-second search across hundreds of thousands of messages. The system spans five distinct algorithmic problems: SMTP receiving with anti-spam, full-text search at scale, threading conversations into trees, attachment deduplication, and (for E2E-encrypted services like ProtonMail) the impossible-on-its-face problem of search over encrypted data.

1. Why This System Is Distinctive

Three structural properties differentiate email from other messaging systems:

Federated by design. Unlike Slack/WhatsApp/Telegram (each a closed ecosystem), email is the SMTP-based open federation of every mail server on the internet. Anyone can run a mail server; anyone can send to anyone. This brings the upside (universal connectivity) and the central downside: every message arriving at your server is potentially adversarial (spam, phishing, malware), and you cannot trust the sender’s IP, the headers, or even the claimed sender domain without explicit cryptographic verification.
Permanent and searchable. Unlike chat (where most users skim recent context), email is treated as long-term archival. A user with a 10-year-old Gmail account expects to search for an email from 2014 and find it in milliseconds. The full-text index over multi-decade history is an enormous data-management problem.
Attachment heavy. Single messages can carry tens of MB of attachments; large mailboxes are gigabytes; aggregate storage at platform scale is exabytes. Naive per-message storage of attachments multiplies storage by recipient count — a 25 MB JPEG sent to a million-person mailing list would store 25 TB if naively stored. Content-hash deduplication is non-optional.

Add the cross-cutting requirement that email delivery must be > 99.99% reliable (a single dropped message can have legal consequences in some jurisdictions, can cost a deal, can leak a security alert) and the design goal becomes: extremely conservative, cryptographically authenticated, redundantly stored, asynchronously queued, and aggressively spam-filtered.

2. Requirements

2.1 Functional Requirements

Send mail. A user composes a message (subject, body, attachments, recipients) and submits it. The system delivers the message to each recipient’s mail server.
Receive mail. The system accepts incoming mail from external servers via SMTP, runs it through spam/malware/policy filters, and deposits it in the appropriate user mailbox.
Read mail. The user opens a mailbox via web, mobile app, or third-party client (which talks IMAP or POP3 — see §4). Lists folders/labels, reads messages, marks read/unread.
Threading. The conversation view groups messages by reply-chain into threads, displaying the chronological history.
Search. Full-text search across the user’s entire mailbox, including subject, body, sender, attachment names, and (for some services) attachment content.
Compose features. Rich text formatting, signatures, autocomplete recipients, drafts saved automatically, scheduled send, undo send within a few seconds.
Attachments. Files up to 25 MB inline (Gmail limit, public docs); larger via Drive/OneDrive integration.
Labels / folders. Gmail-style labels (a message can have multiple) or Outlook-style folders (a message has one location). Built-in labels: Inbox, Sent, Drafts, Spam, Trash, Important, Starred.
Filters and rules. User-defined rules (“from boss@example.com → label Important”). Auto-applied on receive.
Auto-reply (vacation). Generated reply on each incoming message during vacation periods, with rate-limiting to avoid loops.
Forwarding and aliases. A user can configure “all mail to alice+work@example.com forwards to alice@personal.com”, or aliases like “shop@example.com points to alice@example.com”.
Signing and encryption. PGP / S/MIME for encrypted messages between consenting senders/receivers. Some services (ProtonMail) E2E-encrypt by default within their network.
Push notifications. Mobile devices receive push notifications when new mail arrives.
Calendar / contacts integration. Tightly tied to email in productivity suites — reservations parsed from emails added to calendar; sender name/photo from contacts.

2.2 Non-Functional Requirements

Delivery reliability. ≥ 99.99% of legitimate email accepted by the system reaches its destination’s inbox (not just “delivered to the destination server” but actually visible to the user, not silently spam-foldered). Industry benchmark; very hard to verify externally.
Search latency. < 1 second at p99 for full-text queries across a 100 GB+ mailbox.
Read latency. < 100 ms to open and render a message.
Send latency. Asynchronous; the sender sees “sending…” within seconds, but actual delivery to destination MTAs is queued and may take seconds to minutes.
Attachment limit. 25 MB per message (Gmail), or larger via integrated cloud storage.
Spam precision. False-positive rate (legit mail incorrectly flagged as spam) < 0.1% — this is the user-killer metric, much more than false-negatives.
Privacy. Treat user content as confidential; for E2E-encrypted services, the platform itself cannot read message bodies.
Compliance. GDPR (right to deletion, data portability), HIPAA (for healthcare), eDiscovery (legal hold), data residency.

3. Capacity Estimation

Gmail-scale (illustrative, public statements vary):

active accounts                ~ 1.8 × 10^9
average emails received/day    ~ 100 (legit + spam combined; spam is filtered)
total ingest emails/day        ~ 1.8 × 10^11 = 180 billion / day
sustained ingest rate          ~ 2 × 10^6 emails/sec
peak ingest rate (spam waves)  ~ 10× sustained
average email size             ~ 75 KB (after attachments factored in)
ingest bandwidth               ~ 2 × 10^6 × 75 KB/s = 150 GB/s

Storage:

per user mailbox average       ~ 5–15 GB; heavy users 100+ GB
total user storage             ~ 10^9 × ~10 GB = 10^10 GB = 10 EB (10 exabytes)
attachments share (with dedup) ~ 70% of bytes; dedup savings ~ 30–50%
total deduplicated storage     ~ 5 EB
3-replica durability           ~ 15 EB

Search index:

total messages stored          ~ 10^14 (10 trillion)
per-user inverted index        ~ 1–10 GB for heavy mailboxes
total index size               ~ 0.5–1 PB after compression

Bandwidth for serving reads:

peak read QPS (open inbox)     ~ 5 × 10^6 / sec at peak
per-read avg payload           ~ 50 KB (rendered HTML + threading metadata)
read bandwidth                 ~ 250 GB/s peak

These numbers are why email is one of the largest computing infrastructures on the planet. The design must shard everything by user.

4. API and Protocols

4.1 The Three Classic Protocols

SMTP (Simple Mail Transfer Protocol) RFC 5321 — the protocol mail servers use to relay mail to each other, on port 25, generally unauthenticated (a destination MX must accept mail from arbitrary senders). A separate but closely related role, message submission, is where a mail user agent (client) hands an outgoing message to its own provider; that runs on port 587 and is specified by RFC 6409 (Message Submission for Mail), which mandates that the submission server (MSA) by default reject the MAIL command unless the session has authenticated via SMTP-AUTH. The split matters: port-25 relay is the open-federation surface that fights spam; port-587 submission is the authenticated front door that lets a provider attribute, rate-limit, and DKIM-sign outbound mail. (Port 465, “submissions”, is the implicit-TLS variant.) The legacy practice of clients sending via port 25 is now largely closed off precisely because it bypassed authenticated submission.
IMAP (Internet Message Access Protocol) — the protocol clients use to access server-stored mailboxes; supports folders, search, and partial fetches (headers first, body on demand). The long-deployed version is IMAP4rev1, RFC 3501 (2003); it was obsoleted by IMAP4rev2, RFC 9051 (2021), which is “largely compatible” with rev1 but removed facilities that proved problematic. In practice RFC 3501 remains the dominant deployed dialect; servers that support both advertise capabilities and enable rev2 only on explicit client request (per RFC 9051). It is the dominant standardized client-server protocol where a native IMAP client is used.
POP3 (Post Office Protocol) RFC 1939 — older download-and-delete protocol; mostly obsolete except for legacy desktop clients.

4.2 SMTP Sender → MX Server (Receiving)

When alice@example.com sends to bob@destination.com:

Alice’s MTA (Mail Transfer Agent) looks up destination.com’s MX (Mail eXchanger) DNS records to find the destination mail server.
Connects to the destination’s port 25 over TCP, ideally with STARTTLS upgrade to TLS.
Issues SMTP commands:

220 mx.destination.com ESMTP Postfix
HELO sender.example.com
250 mx.destination.com Hello sender.example.com
MAIL FROM:<alice@example.com>
250 OK
RCPT TO:<bob@destination.com>
250 OK
DATA
354 End data with <CR><LF>.<CR><LF>
From: alice@example.com
To: bob@destination.com
Subject: Hello
Message-ID: <ID-12345@example.com>
In-Reply-To: <ID-prev@example.com>
References: <ID-thread@example.com>
Date: Thu, 8 May 2026 09:00:00 -0400

Body of the message...
.
250 OK queued as 7F8B3
QUIT

Once the destination has acknowledged (250 OK queued), the sender’s responsibility is discharged.

4.3 Modern HTTP/JSON APIs (for client UIs)

Web clients don’t speak IMAP directly anymore — Gmail’s web app talks to internal HTTP APIs. The functional surface:

POST /api/v1/messages                  # send
GET  /api/v1/messages?label=INBOX&q=&page_token=
GET  /api/v1/messages/{id}
PATCH /api/v1/messages/{id}            # mark read, label, archive, delete
GET  /api/v1/threads
POST /api/v1/drafts
POST /api/v1/filters                   # rule definitions
POST /api/v1/labels
WS   /api/v1/notifications/stream      # real-time mailbox updates

The web API is much richer than IMAP: it supports threading natively, has tighter integration with calendars/contacts, and can return rich metadata (auto-detected travel dates, entities) that IMAP doesn’t surface.

5. Data Model

5.1 Messages

A message is immutable after delivery. Mutability lives in user-specific metadata (read/unread flag, labels, deletion).

messages (one shard per user)
  message_id     UUID                   -- internal
  external_id    string                 -- Message-ID header from RFC 5322
  user_id        bigint
  in_reply_to    string NULL            -- references the parent's external_id
  references     string[]               -- chain of prior external_ids
  thread_id      bigint                 -- computed from in-reply-to / references
  subject        text
  from_addr      text
  to_addrs       text[]
  cc_addrs       text[]
  bcc_addrs      text[]
  date           timestamp
  body_blob_url  text                   -- pointer to immutable body in object store
  body_size      int
  attachments    JSON [{name, content_hash, size}]
  spf_pass       boolean
  dkim_pass      boolean
  dmarc_pass     boolean
  spam_score     float
  labels         text[]                 -- per-user labels (Gmail) or folder (Outlook)
  read           boolean
  starred        boolean
  archived       boolean
  trashed_at     timestamp NULL         -- soft delete

The split between immutable body (in object store, deduplicatable across recipients) and mutable metadata (in a database) is essential for scale. A 25 MB attachment sent to a million-person list is one object in the blob store, referenced by a million metadata rows.

5.2 Threads

Threading is computed from email headers per RFC 5322 §3.6.4:

Message-ID: a globally unique ID for this message (e.g., <abc123@example.com>)
In-Reply-To: the Message-ID this message is responding to
References: chain of Message-IDs back to the conversation root

The threading algorithm (Jamie Zawinski’s original threading paper from 1995, now canonical):

Build a graph of messages where edges are In-Reply-To/References.
Resolve to trees (one thread per connected component rooted at a message with no In-Reply-To, or stitched by Subject when In-Reply-To is missing).
Sort children of each node by date.
Display linearly (Gmail-style) or as an indented tree (Outlook-style).

Real implementations also fallback to subject-based stitching: if In-Reply-To is missing (some clients drop it), match on normalized Subject (strip “Re:”, “Fwd:”) within a recent time window. Gmail famously aggressive at threading; Apple Mail more conservative.

5.3 Labels and Folders

Gmail labels: a many-to-many mapping (message_id, label); labels are user-defined or system (Inbox, Sent, Drafts, Trash, Spam, Important, Starred). A message can have multiple labels.
Outlook folders: a strict tree; a message lives in exactly one folder. Moving from one folder to another is a database update.

The Gmail model is more flexible (the same message visible under Work, Project-X, Team-meeting); the Outlook model is more familiar to filesystem-trained users. Internally both are similar — Gmail just relaxes the tree constraint.

5.4 Attachment Storage with Content-Hash Deduplication

The brilliant trick of large mail providers: store attachments by content hash, not by message.

attachments_blobs (object store, e.g., S3-equivalent)
  content_hash (SHA-256) → blob bytes

attachment_refs (one shard per user)
  user_id, message_id, attachment_idx, content_hash, original_filename, size

When IMG_1234.jpg is sent to a million recipients:

SHA-256 hashes the bytes once at ingest. Suppose hash is abc123....
Look up abc123... in the attachment-blob store. If present, increment a refcount; if absent, write the bytes once.
For each recipient, write a row in attachment_refs mapping (recipient, message_id) → abc123....

Storage saved: ~25 MB × 1,000,000 = 25 TB → 25 MB. Six orders of magnitude. This is the reason a giant email service can have any economic prayer of profitability.

Garbage collection: when the last attachment_refs row for a content hash is deleted, the blob is reaped. In practice deferred and amortized, with per-blob refcount or scheduled scan.

5.5 Per-User Search Index

Each user has their own Inverted Index over their mailbox content. Sharded by user_id because:

Users cannot search each other’s mail (privacy / compliance).
A user’s index size is bounded by their mailbox; few are huge but most are small.
Per-user shards parallelize across the fleet trivially.

Index entries: tokens from subject, body, sender names/addresses, attachment filenames. Usually segmented by time (recent N months in a hot index; older in a cold index).

6. High-Level Architecture

flowchart TB
    subgraph External
        SenderMTA[External Sender MTA]
    end
    subgraph EdgeIn["Inbound Edge"]
        DNS[MX Records]
        SMTPin[SMTP Receivers]
        SPFCheck[SPF / DKIM / DMARC Verifier]
        Spam[Spam / Malware Filter]
    end
    subgraph Storage
        Mailbox[(Mailbox Metadata DB<br/>per-user shards)]
        BodyBlob[(Body + Attachment<br/>Object Store, content-hash)]
        Index[(Per-User Inverted Index)]
    end
    subgraph EdgeOut["Outbound Edge"]
        SMTPout[SMTP Senders]
        ReputationMgr[IP/Domain Reputation Manager]
    end
    subgraph Clients
        WebUI[Web Client]
        MobileApp[Mobile App]
        IMAPClient[IMAP / POP3 Clients]
    end
    subgraph Server["Server-Side Logic"]
        APIGW[API Gateway]
        ReadSvc[Read / Render Service]
        SendSvc[Send Service]
        SearchSvc[Search Service]
        Threading[Threading Service]
        Filters[Filters / Rules Engine]
        Push[Push Notif Service]
    end

    SenderMTA --> DNS
    DNS --> SMTPin
    SMTPin --> SPFCheck
    SPFCheck --> Spam
    Spam --> Filters
    Filters --> Mailbox
    SMTPin --> BodyBlob
    Mailbox --> Index

    WebUI --> APIGW
    MobileApp --> APIGW
    IMAPClient -.IMAP/POP3.-> APIGW
    APIGW --> ReadSvc
    APIGW --> SearchSvc
    APIGW --> SendSvc
    SendSvc --> SMTPout
    SendSvc --> BodyBlob
    SMTPout --> ReputationMgr
    SearchSvc --> Index
    ReadSvc --> Mailbox
    ReadSvc --> BodyBlob
    ReadSvc --> Threading
    Mailbox --> Push
    Push --> MobileApp

What this diagram shows. The system is essentially symmetric: an inbound edge (left) accepts mail from the open internet, runs it through SPF/DKIM/DMARC verification and spam/malware filtering, applies user-defined filters, and writes to per-user mailbox storage. The outbound edge (right) accepts user-composed messages, signs them with DKIM, queues them for delivery, and tracks our IP/domain sender reputation. Storage is split three ways: per-user metadata in a sharded database, immutable body and attachment blobs in a content-hashed object store (the deduplication win), and per-user inverted indexes for search. Clients (web, mobile, third-party IMAP) connect through an API gateway that routes to read, send, search, threading, filter, and push services. The architectural principle: every user’s mail lives in their own shard; cross-user operations are extraordinarily rare; the system scales by partitioning user_id space across servers.

7. Request Flow / Sequence Diagrams

7.1 Receiving a Message

sequenceDiagram
    participant Sender as External Sender MTA
    participant DNS as DNS
    participant MX as Our SMTP Receiver
    participant Auth as SPF/DKIM/DMARC
    participant Spam as Spam Filter
    participant Filt as User Filters
    participant Blob as Object Store
    participant DB as Mailbox DB
    participant Idx as Search Indexer
    participant Push as Push

    Sender->>DNS: MX query for destination.com
    DNS-->>Sender: mx1.destination.com priority 10
    Sender->>MX: SMTP HELO + MAIL FROM + RCPT TO
    MX->>Auth: verify SPF (sender IP allowed for sender domain?)
    Auth-->>MX: pass / soft-fail / fail
    MX->>Auth: verify DKIM signature
    Auth-->>MX: pass / fail
    MX->>Auth: check DMARC policy
    Auth-->>MX: align ok? policy = none/quarantine/reject
    alt DMARC reject and aligned fail
        MX-->>Sender: 5xx reject
    else accept (with possible spam-folder hint)
        MX->>Blob: store body + attachments by content hash
        MX->>Spam: score message (Bayesian + ML + IP reputation)
        Spam-->>MX: spam_score, classification
        MX->>Filt: apply user-defined rules
        Filt->>DB: insert message row with labels
        DB->>Idx: enqueue for indexing
        Idx->>Idx: tokenize, update inverted index
        DB->>Push: notify mobile devices
        Push->>Push: APNS / FCM push
        MX-->>Sender: 250 OK queued
    end

7.2 Sending a Message

sequenceDiagram
    participant U as User Web Client
    participant API as API Gateway
    participant Send as Send Service
    participant Sign as DKIM Signer
    participant Q as Outbound Queue
    participant SMTP as SMTP Sender
    participant ExtMX as Recipient's MX

    U->>API: POST /api/v1/messages (recipient, subject, body, attachments)
    API->>Send: forward
    Send->>Send: persist as Sent, store body
    Send->>Sign: sign with DKIM (private key for our domain)
    Send->>Q: enqueue for delivery
    Send-->>U: 200 (sending in background)

    Q->>SMTP: dequeue
    SMTP->>SMTP: DNS MX lookup for recipient.com
    SMTP->>ExtMX: SMTP transaction
    alt successful 250
        ExtMX-->>SMTP: 250 accepted
        SMTP->>Q: ack delivered
    else temporary failure (4xx)
        SMTP->>Q: retry with exponential backoff
    else permanent failure (5xx)
        SMTP->>U: bounce notification (return-to-sender NDR)
    end

Reading these flows. Receive path is almost entirely server-driven; the user just sees new mail appear via push. Send path defers actual delivery to a queue: the user’s 200 response is “we have safely captured your message”; actual delivery to remote MX may take seconds (immediate) to days (the receiving server is down, our queue retries up to a configured time). Bounces (NDR — Non-Delivery Reports) come back asynchronously as new emails to the sender.

8. Deep Dive 1 — Anti-Spam: SPF, DKIM, DMARC, Bayesian Filtering, ML

Spam is the existential pressure on email; without aggressive filtering, every inbox would be 95% junk. The defensive stack has both cryptographic authentication (does this email really come from where it claims?) and content-and-reputation analysis (does this email look like spam?).

8.1 SPF (Sender Policy Framework) — RFC 7208

A sender domain publishes a DNS TXT record listing the IPs authorized to send mail for that domain:

example.com.   IN TXT "v=spf1 ip4:198.51.100.0/24 ip4:203.0.113.5 -all"

Meaning: “any IP in 198.51.100.0/24 or the single IP 203.0.113.5 may send mail with MAIL FROM: of example.com; all others fail (-all is hard fail).”

Receiving server checks: when an SMTP transaction arrives, look up the sender domain’s SPF record and verify the connecting IP is in the allow-list. If not, soft-fail or hard-fail per the record’s policy.

Limitation: SPF authenticates the envelope sender (MAIL FROM), not the visible From: header. Many phishing emails have a legitimate MAIL FROM (forwarder) but a forged From:. SPF alone is insufficient.

8.2 DKIM (DomainKeys Identified Mail) — RFC 6376

The sending server cryptographically signs the message with a private key; the public key is published in DNS. Receiver fetches the public key and verifies the signature.

DKIM-Signature: v=1; a=rsa-sha256; d=example.com; s=mail2024;
   c=relaxed/relaxed; q=dns/txt; t=1715000000;
   h=from:to:subject:date:message-id;
   bh=base64(hash of body);
   b=base64(signature over headers + body hash)

Public key in DNS:

mail2024._domainkey.example.com  IN TXT "v=DKIM1; k=rsa; p=MIIBIj..."

Reading the tags (per RFC 6376): a= is the signing algorithm — signers MUST implement and SHOULD sign with rsa-sha256, and verifiers MUST support both rsa-sha1 and rsa-sha256 (Ed25519 was added later by RFC 8463); d= is the signing domain (the “SDID” claiming responsibility); s= is the selector, which subdivides a domain’s key namespace so multiple keys can coexist and rotate; h= is the colon-separated list of header fields covered by the signature; bh= is the hash of the canonicalized body; b= is the actual base64 signature over the signed headers plus body hash; and c= is the canonicalization (simple or relaxed, separately for headers and body) that normalizes whitespace before hashing. The verifier computes two hashes — one over the body, one over the selected header fields — and checks both against the published public key, which lives in a DNS TXT record under the _domainkey subtree at <selector>._domainkey.<domain>.

DKIM authenticates the message body and selected headers, not the connection IP. It survives forwarding only to the extent the relevant content is preserved: because the signature verifies a canonicalized hash, any intermediary that modifies the signed headers or body (e.g., a list-server that appends a footer) invalidates the signature (per RFC 6376). This is exactly the forwarding-breakage problem that ARC (Authenticated Received Chain, RFC 8617) was later designed to mitigate, and the reason relaxed canonicalization (which tolerates whitespace fixups) is usually preferred over simple.

8.3 DMARC (Domain-based Message Authentication, Reporting, and Conformance) — RFC 7489

DMARC ties SPF and DKIM together with identifier alignment — the domain authenticated by SPF (the MAIL FROM/RFC5321.MailFrom domain) or by DKIM (the d= signing domain) must align with the domain in the visible From: header (RFC5322.From) — and tells receivers what to do if neither aligns. Crucially, DMARC passes if at least one of SPF-with-alignment or DKIM-with-alignment passes; it does not require both (per RFC 7489). Alignment can be relaxed (the two domains share an organizational domain) or strict (exact match).

_dmarc.example.com.  IN TXT "v=DMARC1; p=reject; rua=mailto:dmarc-reports@example.com; pct=100;"

p=reject means: messages claiming to be from example.com for which neither SPF-aligned nor DKIM-aligned authentication passes should be rejected (ideally during the SMTP transaction). p=quarantine means the receiver should treat failing mail as suspicious (spam-folder it). p=none means observe/report only — take no action, just send reports. The rua= tag names the address that receives aggregate feedback reports on pass/fail across the domain’s mail streams. This From:-header alignment is exactly what plain SPF cannot give you — SPF authenticates only the envelope sender, so a message with a legitimate MAIL FROM but a forged From: passes SPF yet fails DMARC.

DMARC was the breakthrough that made phishing-from-major-brands measurably harder. After Gmail and Yahoo aggressively rejected DMARC-failing messages claiming to be from major banks (~2014–2016), bank-impersonation phishing dropped sharply.

The three together (SPF + DKIM + DMARC) form the modern email authentication baseline; legitimate large senders all configure them.

8.4 Bayesian Filtering and Modern ML

Even authenticated email can be spam (a legitimate domain compromised, or just spammy promotional content). Content-based filtering is the second layer.

Bayesian filtering (Sahami et al. 1998; popularized by Paul Graham’s 2002 essay “A Plan for Spam”): score each token in the message by its prior probability of appearing in spam vs ham (legitimate). Multiply per-token probabilities (with assumptions about independence) into a final spam score. Train per-user (each user’s spam differs from another’s) on user-marked spam folder.

Mathematically (simplified):

P(spam | token) = P(token | spam) × P(spam) / [ P(token | spam) × P(spam) + P(token | ham) × P(ham) ]

where:

P(token | spam) is the probability the token appears in spam, estimated from training data.
P(token | ham) is the same for ham.
P(spam) / P(ham) is the prior probability of spam vs ham.

Final score combines top-N most-discriminating tokens (typically the 15–20 with the most extreme P(spam | token) away from 0.5). Threshold (e.g., 0.9) declares spam.

Modern ML. Production systems augment classical token-probability scoring with deep models — encoders over message text, header analysis, image classification of inline images (for image-based spam that evades text classifiers), and URL classification (does this link go to a known phishing site?). Training labels come from explicit user “Mark as Spam” actions and from large global datasets. Google states publicly that Gmail’s ML models help block more than 99.9% of spam, phishing, and malware, and that adding TensorFlow-based models let it block roughly 100 million additional spam messages per day — specifically targeting image-based messages, mail with hidden embedded content, and low-volume spam hidden inside legitimate traffic from newly created domains (per Google Workspace’s announcement). Importantly, Google describes these deep models as complementing “existing ML and rules-based protections,” not replacing them — so the modern pipeline is explicitly a hybrid of classical signals (heuristics, token statistics, reputation rules) and learned models, not a wholesale move to neural nets. Later work (RETVec, a robustness-oriented text vectorizer) is reported to have detected ~38% more spam while cutting false positives ~19%.

8.5 IP and Domain Reputation

Aggregate signal beyond the message: a particular sender IP’s recent spam ratio. Maintain a Bloom Filter (or several, with different time windows) of “known spammy IPs” and “known legit IPs”; check connecting IP at SMTP time. Real implementations use commercial reputation services (Spamhaus, Cisco/IronPort, Barracuda) and internal aggregation across the platform’s traffic.

A Bloom filter is the right structure for the spammy-IP set because a false positive (a non-spammer briefly classified as spammy) is recoverable; a false negative (a real spammer missed) is what the Bayesian + ML layer catches afterward. Memory savings are enormous: tens of millions of IPs fit in a few hundred MB Bloom filter; a hash table would need many GB.

8.6 Greylisting

A defense against amateur spam bots: when a previously-unseen sender connects, return a temporary 4xx failure code. Real MTAs retry after a few minutes; spam bots often don’t bother. Once the IP has been seen retrying (i.e., it’s a real MTA), accept future connections from it. Adds a few minutes’ latency to first-time legitimate mail but knocks out a substantial fraction of cheap spam.

9. Deep Dive 2 — Search Over Mailboxes at Scale

A user’s mailbox can be 100 GB; they expect search across decades of mail to return in milliseconds. The engineering corner where email becomes a search-engine problem.

9.1 Per-User Inverted Index

Each user has a private Inverted Index over their messages. Tokens come from subject, body, sender name, sender address, recipient addresses, attachment filenames, and (selectively) attachment content (PDF text, image OCR). Posting lists map token → message_ids along with positional info for phrase queries.

Index sharding by user, not by token, is essential because:

Privacy: a single token like “weekend” should not have a posting list mixing different users’ messages.
Authorization is implicit: querying a user’s index returns only their messages.
Per-user index is small enough to fit on one machine for most users; heavy users (ProtonMail mentions 100GB+ mailboxes) get multi-shard per-user splits.

9.2 Time-Tiered Storage

Most search queries are for recent mail (last 30 days, 90 days). Build the index in tiers:

Hot tier (last 90 days): in-memory or on fast SSD; fully-fresh index; sub-millisecond lookups.
Warm tier (last 2 years): SSD; queryable but slower; index compactions less frequent.
Cold tier (older): spinning disk or object store; queries acceptable up to seconds; archival.

Most queries hit only the hot tier and are fast. Queries marked “older” or full-mailbox queries fan out across tiers.

9.3 Real-Time Indexing

When a new message arrives, the search index must be updated within seconds (the user expects a just-arrived message to be searchable). Two patterns:

Synchronous index update. The receive pipeline writes to the inverted index inline. Higher latency for receive; low search lag.
Asynchronous via Kafka. Receive pipeline emits an indexing event; an indexer service consumes and updates. Receive latency low; search lag a few seconds.

Production systems use async with the indexer running close to the storage layer.

9.4 Query Parsing

Gmail-style queries: from:alice subject:(quarterly review) has:attachment after:2024-01-01. Each clause is converted to a posting-list lookup (or filter on metadata) and the results intersected.

9.5 Ranking

Most search queries should return chronologically reverse by default — the user is usually looking for “the most recent message about X.” Some queries need relevance ranking (TF-IDF over body content), particularly when the query is broad. A blended ranker: recency primary; relevance secondary; the user’s explicit interactions (read / starred / replied) tertiary.

10. Deep Dive 3 — End-to-End Encryption (ProtonMail) and the Search Problem

Most email is encrypted in transit (TLS between MTAs) but not encrypted at rest from the platform’s perspective: Gmail can read your mail, scan it for spam, and run features on it. End-to-end encrypted (E2EE) services like ProtonMail and Tutanota flip this: the platform cannot read message bodies even with full server access.

10.1 ProtonMail’s Model

Per Proton’s encryption documentation:

Each user has an OpenPGP key pair. The private key is stored on Proton’s servers under zero-access encryption — encrypted with a key derived from the user’s password, which never reaches Proton — so Proton cannot decrypt it.
When sending Proton-to-Proton, the message is encrypted with the recipient’s public key and is end-to-end encrypted both in transit and at rest. Proton’s servers see only ciphertext.
When sending Proton-to-non-Proton, the default is TLS-only in transit (no E2E). For E2E to an outside recipient, the sender uses the password-protected emails feature: the recipient receives a notification with a link to a Proton-hosted page and decrypts using a password the sender shares out-of-band.
Inbound from a non-Proton sender arrives over TLS in plaintext to Proton, which then stores it under zero-access encryption with the recipient’s public key — so once stored, even Proton cannot read it (it is “E2E from that point forward” in the sense that only the user’s password-protected private key can decrypt it).
One honest caveat Proton documents: subject lines are not end-to-end encrypted, an OpenPGP-standard limitation (headers are not part of the PGP-encrypted body).

10.2 The Search-Over-Encrypted-Data Problem

The platform cannot read the bodies, so the platform cannot index them. ProtonMail’s solution: client-side search. The user’s app downloads message bodies, decrypts them locally, builds an inverted index on the device, and queries the local index.

Engineering implications:

Mobile devices have limited storage; full-mailbox download for E2EE search is expensive. ProtonMail offers progressive download for older mail.
Multi-device sync: each device builds its own index; the index isn’t shared (it would have to be encrypted with a key only the user has, and synced).
Query performance is bounded by device hardware.

Searchable encryption (homomorphic-encryption-style techniques where the server can run search over encrypted indexes without decrypting) is an active research area but not yet practical at email scale; the academic prototypes have substantial overhead and have not been deployed at consumer scale.

This is the fundamental architectural tradeoff: the user-experience features (server-side search, server-side spam ML, server-side travel-receipt extraction) require the server to read the content. Pure E2EE forces those features to run client-side, with concomitant device-resource costs. ProtonMail accepts the tradeoff; Gmail does not.

11. Scaling Strategy

11.1 Shard Mailboxes by User

The natural and dominant sharding axis. Each user’s mailbox + index lives on one shard. Cross-user operations (forwarding, sharing) are explicit and rare.

11.2 Outbound Reputation Management

When sending to external MTAs, our IPs and our domain accumulate reputation. To prevent one user’s spam from poisoning the reputation of all senders on a given IP:

Assign separate IP pools for different sender categories (transactional, marketing, regular user, suspicious).
Throttle per-user outbound rate (Token Bucket).
Detect abnormal sending (a freshly-created account sending to 10K addresses = bot).
Outbound mail goes through the “reputation manager” service that schedules deliveries to maximize legitimate-IP hit rates and avoid blacklisting.

11.3 Geo-Replication

User mailbox shards are replicated across regions for durability and disaster recovery. Reads typically served from the user’s home region; cross-region reads in failure scenarios.

11.4 Mailing Lists at Scale

A 1M-recipient mailing list is a write-amplification disaster if naively implemented (insert a copy of the message into 1M mailboxes). Optimizations:

The body and attachments are content-hash-deduplicated (single blob).
Metadata rows still inserted per recipient (1M rows), but rows are small and batchable.
Delivery to external recipients (people not on our platform) goes through optimized SMTP delivery batching to common recipient domains.

11.5 Async Delivery Queue

The outbound queue is the place where reliability is preserved. RabbitMQ or Kafka holds outbound deliveries; retry workers pull and retry per the per-receiver rules (exponential backoff, distinct retry intervals per error class, send-after-delay for greylisted destinations). Messages can sit in the queue for days during destination outages.

11.6 Push Notifications

When a new message arrives, push to mobile devices via APNS (Apple Push Notification Service) or FCM (Firebase Cloud Messaging). See Notification Service System Design for the patterns.

12. Real-World Example

Gmail

Serves on the order of 1.8 billion users (the figure Google’s spam announcement and contemporaneous reporting both use). Massive deduplication and per-user sharding.
Indexing built on internal Google search infrastructure (likely related to or descended from Google’s web search index implementations). Per-user inverted indexes with time-tiered storage.
Spam filtering is the hybrid of cryptographic authentication (SPF/DKIM/DMARC), rules-based protections, and ML/deep-learning classifiers described in §8.4 — Google’s own post confirms TensorFlow models complement its existing ML and rules-based filters and that the stack blocks >99.9% of spam/phishing/malware (Google Workspace blog).
Smart Compose, Smart Reply, etc. are built on the unencrypted message data — the explicit business choice of Gmail vs ProtonMail.

Microsoft Outlook / Exchange / Microsoft 365

~400M consumer + business accounts (estimated; Microsoft does not break out exact numbers).
Outlook for desktop talks Exchange Web Services or MAPI; Outlook.com and Microsoft 365 web talk REST APIs on Exchange Online.
Exchange Server is the on-premises product; Microsoft 365 is the hosted version. Both share the database engine (ESE / “Jet Blue”). Migrations from on-prem to cloud have been a multi-year effort.

ProtonMail

E2EE by default for Proton-to-Proton mail. Public whitepaper documents PGP-based encryption.
Switzerland-based; positioning around privacy.
Search runs client-side because servers cannot read the content. Mobile app indexes downloaded mail.
100M+ Proton accounts as of 2024, per Proton’s own milestone announcement (note: this counts all Proton accounts — Mail, VPN, Drive, etc. — not Mail mailboxes alone; Proton does not break out Mail-only figures).

Hey

Basecamp’s opinionated email reinvention; charges a flat fee instead of ad-supported.
Public engineering posts describe the “screener” model (first-time senders quarantined for explicit user approval), reduced label/folder usage, focus on “clipping” parts of messages.
Smaller scale than Gmail/Outlook but a useful sanity check on what email looks like with a cleaner slate.

Postfix / Sendmail / Exim (open source)

Powering tens of thousands of corporate and ISP mail servers. Postfix in particular is highly modular and well-documented (Wietse Venema’s architecture overview). Rather than one monolithic daemon, Postfix is a set of cooperating processes: network mail enters through smtpd (or qmqpd), is normalized by the cleanup daemon (which adds missing From:/other headers and computes the queue entry), and is then handed to the queue manager (qmgr). Mail flows through distinct on-disk queues — incoming → active → deferred — where qmgr keeps only a bounded active queue open for delivery to avoid exhausting memory under load, and parks messages awaiting retry in the deferred queue (per the Postfix overview). This is the open-source embodiment of the store-and-forward, bounded-active-set queueing the rest of this note describes abstractly.
Supports SPF/DKIM/DMARC via plug-in milters (mail filters).
Spam filtering via SpamAssassin, rspamd, or ClamAV for malware.
Single-server deployments handle thousands of accounts; clustering for larger scale.

Uncertain

Verify: Gmail’s internal architecture specifics — mailbox sharding factor, replica count, and the exact hot/warm/cold search-index tier breakdown. Reason: not publicly documented; the figures here are extrapolations from generic large-scale-systems patterns plus limited public Google blog posts, not fetched primary disclosures. To resolve: a Google engineering paper or talk on Gmail storage/indexing internals. uncertain

13. Tradeoffs

Decision	Option A	Option B	When A wins	When B wins
Server vs E2E encryption	Server can read (Gmail)	E2E (ProtonMail)	Server-side ML features matter	Privacy paramount
Storage	Body in DB	Body in object store, metadata in DB	Tiny scale	Real scale (always B)
Threading	RFC 5322 In-Reply-To	Subject-based	Explicit chain	Robust to dropped headers
Client protocol	IMAP only	HTTP API + IMAP	Legacy clients	Web/mobile (B for modern)
Spam filter	Bayesian only	Multi-layer (SPF/DKIM/DMARC + Bayesian + ML)	Self-host small	Production scale (always B)
Attachment storage	Per-message copy	Content-hash dedup	Tiny scale	Production (always B)
Search	Server-side index	Client-side	Privacy not required	E2EE constraint
Push	Polling	APNS/FCM push	Niche desktop	Mobile (B)
Outbound delivery	Synchronous SMTP from web	Queued + async	Toy	Production (always B)
Folders vs labels	Folders (Outlook)	Labels (Gmail)	Familiar	More flexible
Folder model	Hierarchical	Flat tags	Filesystem mental model	Multi-context messages

14. Pitfalls

Forgetting DMARC / SPF / DKIM on the outbound side. Your domain’s mail will increasingly be rejected as spam by major receivers if you don’t configure all three.
Treating bounce messages as just regular mail. They carry codes (5.1.1 user unknown, 4.4.7 message expired, etc.) the system must parse to update bounce-suppression lists; otherwise sending to a known-dead address forever destroys your sender reputation.
Not respecting the Message-ID. Generating duplicate Message-IDs across replies breaks threading at receivers. Always use <UUID@your.domain>.
Reading user mail without auth. A misconfigured admin tool dumped Yahoo Mail in the 2014 Yahoo breach. All access to mailboxes must go through audited, role-restricted paths.
Naive search with no per-user shard. A single global index that all users query is impossible to authorize and impossible to scale. Per-user from day one.
Storing attachments per-recipient. Storage explodes for large recipient lists. Always content-hash dedup.
Fanout-on-write with no backpressure. Sending a 1M-recipient mailing list through a synchronous loop blocks the sender’s UI. Always queue.
Spam filter trained only on user “Mark as Spam”. Users mark legitimate things as spam (it’s just a “delete” for them). Train on global signals, IP reputation, content classifiers, and user marks — not only marks.
No rate limiting on outbound. A compromised account sends millions of messages in an hour, blacklisting our IP. Per-user outbound rate limits via Token Bucket are mandatory.
Forgetting MX backup records. A primary MX outage with no backup means inbound email bounces (4xx-retried for hours/days, but eventually fails). DNS MX records support priority — always have a secondary.
Loop in auto-reply. Vacation auto-reply replies to a vacation auto-reply, infinitely. Standard guards: only reply once per sender per N days; never reply to mailing-list Precedence: bulk headers; never reply to MAIL FROM: <> (null sender, indicating an automated message).
Open relay. A misconfigured SMTP server that accepts mail from any sender for any recipient is exploited within hours by spammers. Close down: only relay for authenticated senders or for our own domains’ inbound mail.
DNS being SPOF (single point of failure). Email is built on DNS — MX records, SPF/DKIM/DMARC TXT records. The Mailgun “MX outage” of 2016 (DNS provider compromise) cut off mail delivery for thousands of customers. DNS redundancy is critical.

15. Common Interview Variants

“Design Gmail.” The flagship version covering all of the above.
“Design a corporate email server.” Same shape but smaller scale; focus on compliance (legal hold, retention), federation, calendar integration.
“Design ProtonMail (E2EE email).” Adds the encryption + key management + client-side search angle.
“Design a transactional email service (SendGrid / Postmark).” Focused on sending, not receiving — emphasis on outbound IP/domain reputation, deliverability metrics, bounce handling.
“Design a mailing list system (Mailman, Substack).” Adds list management, subscriber unsubscribe, list moderation.
“Design a spam filter.” Deep dive on the filtering pipeline: SPF/DKIM/DMARC + Bayesian + ML + reputation.
“Design Slack but with email-style threading.” Hybrid; explore which email properties (rich text, attachments, archival) translate to a chat product.
“Design email search across a 100GB inbox.” Focused on indexing strategy.

16. Open Questions / Uncertain

Uncertain

Verify: Gmail’s internal spam-model architectures, training-data sources, and exact false-positive-rate targets. Reason: proprietary — Google publicly states the headline outcomes (>99.9% blocked; TensorFlow + RETVec; hybrid of ML and rules-based filters — now cited inline in §8.4) but not the model internals. The high-level “authentication + reputation + classical signals + deep ML” layering is confirmed; the specifics are not. To resolve: would require non-public Google documentation. uncertain

(Resolved separately: the earlier doubt over whether Gmail still uses any classical/Bayesian signals is settled — Google explicitly describes its deep models as complementing “existing ML and rules-based protections,” so the pipeline is a confirmed hybrid, not a pure-neural replacement.)

Uncertain

Verify: the exact attachment-deduplication ratio in production (the note uses 30–50% storage savings). Reason: plausible given attachments dominate storage and duplicate heavily, but no provider publishes a precise figure; this is an estimate, not a measured published number. To resolve: a provider storage-engineering disclosure. uncertain

17. See Also

Major System Designs MOC — parent
SWE Interview Preparation MOC — grandparent
Inverted Index — per-user mailbox search
Bloom Filter — spam IP reputation tables
Token Bucket — outbound rate limiting per user
Notification Service System Design — push to mobile on new mail
Distributed Log System Design — async indexer / outbound queue
Consistent Hashing — sharding mailbox storage by user
Content Delivery Network System Design — for static asset and avatar caching
Slack Team Messaging System Design — sibling messaging system; instructive contrast
WhatsApp Messenger System Design — sibling E2EE messaging system
Distributed Counter System Design — unread counts per folder
Calendar System Design — tightly integrated sibling
Fraud Detection Pipeline System Design — phishing detection
Back-of-Envelope Estimation — capacity math primitives

vault

Explorer

Email Service System Design

Email Service System Design

1. Why This System Is Distinctive

2. Requirements

2.1 Functional Requirements

2.2 Non-Functional Requirements

3. Capacity Estimation

4. API and Protocols

4.1 The Three Classic Protocols

4.2 SMTP Sender → MX Server (Receiving)

4.3 Modern HTTP/JSON APIs (for client UIs)

5. Data Model

5.1 Messages

5.2 Threads

5.3 Labels and Folders

5.4 Attachment Storage with Content-Hash Deduplication

5.5 Per-User Search Index

6. High-Level Architecture

7. Request Flow / Sequence Diagrams

7.1 Receiving a Message

7.2 Sending a Message

8. Deep Dive 1 — Anti-Spam: SPF, DKIM, DMARC, Bayesian Filtering, ML

8.1 SPF (Sender Policy Framework) — RFC 7208

8.2 DKIM (DomainKeys Identified Mail) — RFC 6376

8.3 DMARC (Domain-based Message Authentication, Reporting, and Conformance) — RFC 7489

8.4 Bayesian Filtering and Modern ML

8.5 IP and Domain Reputation

8.6 Greylisting

9. Deep Dive 2 — Search Over Mailboxes at Scale

9.1 Per-User Inverted Index

9.2 Time-Tiered Storage

9.3 Real-Time Indexing

9.4 Query Parsing

9.5 Ranking

10. Deep Dive 3 — End-to-End Encryption (ProtonMail) and the Search Problem

10.1 ProtonMail’s Model

10.2 The Search-Over-Encrypted-Data Problem

11. Scaling Strategy

11.1 Shard Mailboxes by User

11.2 Outbound Reputation Management

11.3 Geo-Replication

11.4 Mailing Lists at Scale

11.5 Async Delivery Queue

11.6 Push Notifications

12. Real-World Example

Gmail

Microsoft Outlook / Exchange / Microsoft 365

ProtonMail

Hey

Postfix / Sendmail / Exim (open source)

13. Tradeoffs

14. Pitfalls

15. Common Interview Variants

16. Open Questions / Uncertain

17. See Also

Graph View

Table of Contents