vault

Click Tracking System Design

30 min read

Click Tracking System Design

The click tracking subsystem is the data pipeline behind Google Analytics, Adobe Analytics, Snowplow, every ad network’s billing infrastructure, and every marketing-attribution platform (Branch, Adjust, AppsFlyer). Its job is deceptively narrow on the surface: when a user clicks an ad or a tracked link, record the click and route the user to the destination. In practice the design problem expands into a full discipline because (a) the click is monetized — ad networks bill advertisers per click, so double-counting or under-counting directly translates to overcharge or revenue loss, demanding exact-once accounting on the billing path; (b) the user is waiting — the redirect must complete in less than ~50ms, which constrains where the logging can happen; (c) fraud is endemic — bots and click farms generate a meaningful fraction of all “clicks” globally, and filtering them is the difference between a working business and a closed one; (d) attribution is contested — when a user sees five ads and then converts, which ad gets credit (first-touch, last-touch, linear, time-decay, data-driven)? attribution model choice changes payouts; (e) privacy regulations (GDPR, CCPA, Apple’s App Tracking Transparency, the deprecation of third-party cookies, Google’s Privacy Sandbox) have rendered the historically-standard tracking pixel and 3rd-party-cookie pattern partially or wholly unusable, forcing the industry into server-side tracking and aggregation-only telemetry. The interview value of “design click tracking” is that it forces all five of these constraints onto the table at once, and the candidate has to keep them coherent: a redirect that’s “fast” but unreliable loses billing data; a billing pipeline that’s “exact” but synchronous slows the redirect; a tracking pixel that “just works” violates GDPR. The right architecture is a layered pipeline — edge collector → durable log → enrichment → fraud filter → attribution → reporting — borrowing heavily from Real Time Analytics System Design (analytics counterpart) and Distributed Log System Design (the durable transport).

1. Why This System Appears in Interviews

The interview tests whether the candidate sees the problem holistically rather than as “log a click to a database.” Strong answers cover:

  1. Two fundamentally different click types. A page-load tracking pixel (no redirect; user is on the publisher’s page) vs a click-through link (user clicks a link, gets redirected to the advertiser’s destination). Different mechanics, different latency budgets.
  2. The exactly-once accounting problem. Network retries happen; the same click event must not be billed twice. Resolved by client-generated click_id tokens and a dedup window.
  3. Fraud detection. Industry-published estimates put invalid traffic (IVT) at 5–25% of total ad clicks. Removing it without losing legitimate traffic requires a multi-layer detector (heuristics + ML).
  4. Attribution model choices. Last-touch is simplest; data-driven (Shapley values) is most accurate but expensive; the choice affects billing.
  5. Privacy compliance. Server-side tracking, consent management, IDFA opt-out, third-party cookie deprecation. Modern systems cannot blindly capture every interaction.
  6. Real-time reporting. Advertisers refresh dashboards every minute checking if their campaign is “spending OK”; the click pipeline must surface metrics to a Real Time Analytics System Design backend within a minute.

The sloppy answer ignores fraud and privacy. The strong answer reasons through all six concerns and explains how the components fit together.

2. Requirements

2.1 Functional Requirements

  1. Track impression. When an ad is displayed, log an impression event (publisher, slot, creative, viewer fingerprint, timestamp).
  2. Track click. When a user clicks an ad or a tracked link, log a click event AND redirect the user to the destination URL.
  3. Track conversion. When the user takes a downstream action (signup, purchase) on the advertiser’s site, log a conversion event and attribute it to a prior click/impression.
  4. Attribution. Given a user’s click and conversion stream, assign credit (full or partial) to one or more clicks per the configured attribution model.
  5. Aggregation for reports. Per-(advertiser, campaign, day, dimension), counts of impressions, clicks, conversions, spend, attributed revenue.
  6. Fraud filtering. Suppress clicks from bots, click farms, datacenter IPs, abnormal clickers; do not bill advertisers for filtered clicks.
  7. Real-time billing feed. Per-advertiser running spend total, updated within seconds, used to enforce daily budget caps.

2.2 Non-Functional Requirements

  1. Click throughput. A large ad network (Google Ads, Facebook Ads) handles ~10⁵–10⁶ clicks/sec at peak globally; impressions are ~100× clicks.
  2. Redirect latency. p99 < 50ms from click to 302 issued. The user is waiting — slow redirects feel broken.
  3. Billing accuracy. Clicks counted toward billing must be exact-once: no double-bill (overcharge), no miss (revenue loss). Tolerance: 0% double-bills; ≤ 0.01% loss is the IAB-standard internal SLA.
  4. Conversion-attribution latency. A purchase 30 days after click must still attribute correctly. 30+ day attribution windows are common.
  5. Fraud filter accuracy. False-positive rate (legitimate clicks filtered) < 0.1%; false-negative rate (fraud not caught) target < 5% but in practice 5–25% slips through.
  6. Privacy compliance. GDPR consent verified before storing PII; CCPA opt-out honored; ATT for iOS apps; cookie consent for web.
  7. Storage retention. Raw events retained for the attribution window (30–90 days typically); aggregated reports for years.

Uncertain

Verify: the specific peak click/impression QPS of named ad networks (Google Ads, Meta Ads) and the precise invalid-traffic (IVT) percentage. Reason: ad networks do not publish raw per-second click volumes or audited internal fraud rates; the orders of magnitude here are inferred from public DAU/impression figures and the structure of the IAB/MRC IVT framework, not from a primary disclosure. To resolve: a network’s published transparency report with audited IVT figures, or an MRC accreditation audit, would settle the exact numbers. The categories of IVT, by contrast, are precisely defined by the IAB/MRC IVT guidelines (see §8.7), so the qualitative model is firm even where the percentage is not. #uncertain

3. Capacity Estimation

3.1 Click QPS

clicks per day (large network)         = 5 × 10^10
seconds per day                        = 86,400
average click QPS                      ≈ 5.8 × 10^5 clicks/s
peak (3×)                              ≈ 1.7 × 10^6 clicks/s

About 1.7 million clicks per second at peak. Each click is a redirect + log; the log is the durable event.

3.2 Impression QPS

Impressions are ~100× clicks (typical CTR ~ 1%):

impression QPS average                 ≈ 5.8 × 10^7 events/s
impression QPS peak                    ≈ 1.7 × 10^8 events/s

170 million impression events/sec at peak. Most of these flow as low-value telemetry; only the click-converting fraction matters for billing.

3.3 Storage

event size (raw)                       ≈ 1 KB (URL, headers, geo, device, etc.)
events/day (impressions + clicks)      ≈ 5 × 10^12
storage/day (raw)                      ≈ 5 × 10^12 × 1 KB = 5 PB/day
compressed (10×)                       ≈ 500 TB/day
30-day retention                       ≈ 15 PB
3× replication                         ≈ 45 PB

Petabyte-scale daily ingestion. Tiered to cold storage after the attribution window.

3.4 Billing-Critical Path QPS

Of the click QPS, only the fraud-filtered clicks become billable events:

billable rate = click_rate × (1 - fraud_fraction) × (advertiser_subset)
              ≈ 5.8 × 10^5 × 0.85 × 1.0 ≈ 5 × 10^5 billable clicks/s

Half a million billable transactions per second — these need exact-once accounting. The pipeline below shows how this is achieved without putting that pressure on the redirect path.

4. API Design

4.1 Tracking Pixel (Impression)

GET https://track.example.com/impression?
        ad_id=ad42
        &publisher=pub7
        &campaign=cam99
        &slot=slot_top
        &cb=<cache_buster_random>

The pixel is a 1×1 transparent GIF. The browser renders it as part of page load; the GET to /impression is the logging side-effect. Response: 200 with the GIF bytes.

4.2 Click Tracking + Redirect

GET https://track.example.com/click?
        click_id=<uuid>             (or generated server-side if missing)
        &ad_id=ad42
        &campaign=cam99
        &dest=https%3A%2F%2Fadvertiser.example.com%2Flanding
→ 302 Found
Location: https://advertiser.example.com/landing?utm_source=...&click_id=<uuid>

The browser issues this GET when the user clicks; server logs the click and replies with a 302 redirect. Latency must be sub-50ms because the user sees nothing until the destination loads.

4.3 Conversion Postback

POST https://track.example.com/conversion
{
  "click_id":      "<uuid>",                  -- from the click cookie or URL parameter
  "advertiser_id": "advertiser_42",
  "value":         29.99,
  "currency":      "USD",
  "event_name":    "purchase",
  "external_id":   "order_123",                -- advertiser's order ID for dedup
  "user_id":       "<advertiser-side ID>"
}
→ 200 OK
{"attributed_to": "<click_id>", "model": "last_touch"}

The conversion call is from the advertiser’s server (server-to-server, S2S) or from a JS pixel on their thank-you page. The click_id ties back to the original click for attribution.

4.4 Beacon API (Browser Unload)

For events fired during page unload (the user is leaving), use the Beacon API:

window.addEventListener('beforeunload', () => {
  navigator.sendBeacon('https://track.example.com/event',
                       JSON.stringify({event: 'page_unload', ...}));
});

sendBeacon is fire-and-forget but the browser guarantees delivery even if the page navigates away. Critical for events that need to fire on exit.

5. Data Model

5.1 Click Events (Raw, Immutable)

Table:           click_events
Partition key:   click_id (UUID, client- or edge-generated)
Sort key:        timestamp
Cols:
  ad_id           STRING
  campaign_id     STRING
  publisher_id    STRING
  user_fp         STRING        -- user fingerprint (cookie / hashed device ID)
  ip              STRING        -- the source IP
  geo_country     STRING
  geo_city        STRING
  user_agent      STRING
  referrer        STRING
  destination_url STRING
  is_fraud        BOOLEAN       -- populated asynchronously by fraud detector
  fraud_score     FLOAT         -- 0..1 confidence
  billed_at       TIMESTAMP NULL -- set when committed to billing

Stored in a wide-column or columnar store (Cassandra, ClickHouse, or BigQuery). Indexed by click_id for dedup lookup; partitioned by date for time-range queries.

5.2 Click Dedup (Idempotency)

Table:           click_dedup
Partition key:   click_id
TTL:             attribution_window (30 days typical)
Value:           {first_seen_at, count}

Used to dedupe identical click_id retries within the attribution window. Stored in a Redis cluster for low-latency lookup or a Bloom filter (Bloom Filter) for ultra-cheap probabilistic dedup with negligible false-positive rate.

5.3 Attribution Records

Table:           attribution_records
Partition key:   conversion_id
Sort key:        attributed_click_id
Cols:
  weight                FLOAT      -- 1.0 for last-touch, fractional for multi-touch
  attribution_model     STRING
  attributed_at         TIMESTAMP

For multi-touch attribution models, one conversion can attribute weight to multiple prior clicks; the table is many-to-many.

5.4 Aggregations

Per-advertiser-campaign-day:
  impressions, clicks, conversions, attributed_revenue, billed_amount

Stored as in Real Time Analytics System Design — a columnar warehouse with rollups and HLL sketches for distinct-user counts.

6. High-Level Architecture

flowchart TB
    Client[User browser/app] -->|impression pixel| EdgeImpr[Edge Impression Collector]
    Client -->|click + redirect| EdgeClick[Edge Click Collector]
    Client -->|conversion S2S/JS| EdgeConv[Edge Conversion Collector]

    EdgeClick -->|302 immediately| Client
    EdgeClick -->|async log| Kafka[(Kafka<br/>events stream)]
    EdgeImpr --> Kafka
    EdgeConv --> Kafka

    Kafka --> Enrich[Enrichment Service<br/>geo, UA, device]
    Kafka --> Dedup[Dedup Service<br/>Bloom + Redis]
    Kafka --> Fraud[Fraud Detector<br/>heuristics + ML]

    Dedup --> Clean[(Clean Event Stream)]
    Enrich --> Clean
    Fraud --> Clean

    Clean --> Attrib[Attribution Engine]
    Attrib --> AttribStore[(Attribution Records)]
    Clean --> Billing[Billing Aggregator<br/>per-advertiser counters]
    Billing --> Spend[(Real-Time Spend Counters)]
    Clean --> AnalyticsLoad[Analytics Loader]
    AnalyticsLoad --> Analytics[(Real-Time Analytics Backend)]

    Spend --> BudgetEnforcer[Budget Enforcer<br/>caps active campaigns]
    Spend --> Reports[Reporting]

What this diagram shows. The architecture has three intake paths — impression pixel, click redirect, and conversion postback — each with its own latency profile but all converging on the same durable Kafka event stream. The click path is the most latency-critical: the edge collector immediately issues the 302 redirect to the user (sub-50ms), then asynchronously publishes the click event to Kafka. From the unified event stream, four parallel processing tracks run: (1) enrichment (geo lookup, user-agent parsing, device fingerprinting), (2) dedup (Bloom-filter check + Redis dedup store keyed by click_id to suppress retries), (3) fraud detection (heuristic + ML scoring; flags suspicious events), and (4) attribution (matches conversions to prior clicks per the configured model). The cleaned event stream feeds three downstream consumers: the billing aggregator (maintains per-advertiser real-time spend counters and feeds the budget enforcer that disables campaigns when daily budgets are hit), the attribution store (records of which click(s) earned credit for which conversion), and the analytics loader that pushes events into the Real Time Analytics System Design backend for dashboards and ad-hoc queries. The crucial separation is between the redirect-path latency budget (50ms, single hop) and everything downstream (seconds to minutes, durable, exactly-once-on-billing). The single point that breaks if Kafka is down is the durability of the event stream — but the redirect itself still issues, so users continue to land on advertiser pages even if telemetry is degraded; the cost is a window of un-attributable clicks until Kafka recovers.

7. Request Flow

7.1 Impression Path (Tracking Pixel)

sequenceDiagram
    participant B as Browser
    participant E as Edge Collector
    participant K as Kafka

    B->>E: GET /impression?ad_id=42&...
    E->>E: parse params, capture headers
    E->>K: publish ImpressionEvent (async)
    E-->>B: 200 + 1×1 GIF (cache headers)

The browser is satisfied when the GIF returns; impressions are not latency-blocking (the user already sees the ad).

7.2 Click Path (Tracking Redirect)

sequenceDiagram
    participant B as Browser
    participant E as Edge Click Collector
    participant K as Kafka
    participant DEST as Advertiser Server

    B->>E: GET /click?click_id=abc&dest=https://...
    E->>E: validate click_id, sign destination URL
    E-->>B: 302 Found, Location: dest URL+click_id
    Note over E: redirect fires immediately,<br/>publish to Kafka happens<br/>after the wire is freed
    E->>K: publish ClickEvent {click_id, ad_id, ts, ip, ua, ...}
    B->>DEST: GET dest URL+click_id
    DEST-->>B: 200 (advertiser page)

The 302 must precede the Kafka publish to keep latency tight. If the Kafka publish fails after the 302, the click is logged via a fallback (local disk WAL on the collector, replayed when Kafka recovers).

7.3 Conversion Path (Server-to-Server Postback)

sequenceDiagram
    participant ADV as Advertiser Server
    participant E as Edge Conv Collector
    participant K as Kafka
    participant ATT as Attribution Engine
    participant CDB as Click Database

    ADV->>E: POST /conversion {click_id, value, ...}
    E->>K: publish ConversionEvent
    E-->>ADV: 200
    K->>ATT: consume ConversionEvent
    ATT->>CDB: lookup click_id, get prior clicks for user
    CDB-->>ATT: click history
    ATT->>ATT: apply attribution model
    ATT->>K: publish AttributionEvent {conversion -> click(s)}

Conversion is async — the advertiser’s server gets a 200 immediately, attribution happens downstream.

8. Deep Dive

8.1 Tracking Pixel Mechanics

A tracking pixel is a 1×1 transparent GIF embedded in a publisher’s webpage:

<img src="https://track.example.com/impression?ad_id=42&pub=p7" width="1" height="1" />

When the browser renders the page, it issues the GET. The server records the request (URL params + HTTP headers carry the ad ID, campaign, publisher; the headers carry user-agent, referrer, IP, cookies). The response is a tiny GIF (43 bytes minimum); the browser displays it invisibly.

The pixel is the canonical web-tracking primitive because:

  • Works without JavaScript (some publishers disabled JS for ads).
  • Requires no consent dialog beyond the publisher’s existing cookie banner.
  • Universal browser support since 1995.

It is also the canonical privacy attack surface — every page-loaded pixel is a third-party connection that captures the user’s IP, the page they’re on (via referrer), their device fingerprint (via headers), and any third-party cookies the ad domain has set. GDPR and the third-party cookie deprecation are largely about reducing this surface.

8.2 Click-Through Pattern

A click-through link goes through the tracker before the destination:

<a href="https://track.example.com/click?cid=xyz&dest=https%3A%2F%2Fadvertiser.com%2Flanding">
  Buy now
</a>

When clicked, the browser GETs the tracker URL; the tracker logs and replies with 302 Location: https://advertiser.com/landing. The browser follows the redirect.

The latency budget is brutal: the user sees nothing until the destination loads. Slow click-trackers translate to perceived “broken ads.” Production click-trackers run at the edge (Anycast / CDN POPs) to keep round-trip latency under 50ms regardless of user location.

The destination URL typically gets the click_id appended as a query parameter so the advertiser’s site can correlate the conversion. The click_id is also dropped as a cookie on the advertiser’s domain (if cross-origin tracking is permitted) for cookie-based attribution.

8.3 Beacon API for Unload Events

Browsers traditionally cancel pending HTTP requests when the page navigates away. For events fired in beforeunload (e.g., “user closed tab after viewing page for 47 seconds”), this means losing the event.

The Beacon API (cited above) is navigator.sendBeacon(url, data) — fire-and-forget, browser guarantees delivery (queued in the network stack and sent even after page navigation), no response. Replaces the awkward synchronous-XHR-on-unload pattern that used to block page navigation.

8.4 Deduplication

The same logical click can produce multiple HTTP requests for several reasons:

  1. Network retries. Client side: aggressive retry on 5xx. Server side: at-least-once Kafka publish.
  2. Bot replays. A poorly-designed bot reposts the click URL.
  3. Browser back-forward navigation. User clicks back, then forward; depending on caching, the click URL may re-fire.
  4. Prefetchers. Browsers and search engines prefetch destination URLs as users hover; if prefetch hits the click URL, it logs a phantom click.

Dedup key: click_id. Generated client-side (in the link template, server-side rendered), or edge-side as a UUID if missing. The dedup store is checked early in the pipeline; duplicate click_ids are dropped (not even published to Kafka) or marked as duplicates and excluded from billing.

A Bloom Filter makes the dedup check ultra-cheap: keep recent click_ids in a Bloom filter; check membership on each click. False positives (Bloom claims “seen” for an unseen click_id, ~0.1% rate) drop a real click — acceptable noise. False negatives are impossible — if Bloom says “not seen” the click is new. Combine with a Redis dedup cache for the small set of likely-duplicates.

For exactly-once billing, the dedup must be backed by a durable store (Redis with AOF + replication, or a Cassandra/Manhattan key-value store keyed by click_id). The dedup window is the attribution window: 30 days typical.

8.5 Attribution Models

When a user has multiple touchpoints (saw 3 ads, clicked 2, then converted), which click gets credit?

8.5.1 First-Touch

Attribute 100% to the first click. Rewards top-of-funnel awareness ads.

8.5.2 Last-Touch

Attribute 100% to the last click before conversion. Industry default historically; rewards bottom-of-funnel conversion-driving ads. Easy to compute (constant time per conversion).

8.5.3 Linear

Attribute equal weight to every click in the path. weight = 1/N for N clicks.

8.5.4 Time-Decay

Weight = e^(-λ·age) where age is “time before conversion.” Recent clicks get more credit; old clicks fade.

8.5.5 Position-Based (U-Shaped)

40% to first click, 40% to last click, 20% spread across middle. A heuristic that combines first-touch and last-touch.

8.5.6 Data-Driven (Shapley Values)

Train a model on conversion outcomes to estimate the marginal contribution of each click in the path. Shapley value from cooperative game theory: the average marginal contribution of click c across all subsets of the path that don’t include c.

Computationally expensive (O(2^N) per path naively); practical implementations use sampling. Google’s “Data-Driven Attribution” in Google Ads is a productionized version. The most accurate model in principle; the most complex to implement and explain.

The choice affects payouts: a publisher whose ad usually appears first in the funnel earns more with first-touch than with last-touch. Advertisers who want to value upper-funnel ads often switch from last-touch to data-driven.

8.6 Session Reconstruction

Events are grouped by session (a user’s contiguous activity window). A session breaks after 30 minutes of inactivity (Google Analytics’ default). Within a session, events are ordered by timestamp — the user’s journey from landing to conversion or exit.

Implementation: events flow into a per-(user_id, session_window) aggregator that ends the session on the inactivity timeout, persisting a session record with (start, end, page_views, events). The user-event store schema in Real Time Analytics System Design §5.3 supports this.

Bots and bot-detection: real users have varied click intervals (5s, 50s, 3 minutes); bots often have eerily uniform intervals (every 1.0s ± 0.1s). Session-shape analysis is a fraud signal.

8.7 Fraud and Bot Filtering

The industry’s reference framework for invalid traffic is the joint IAB / Media Rating Council (MRC) Invalid Traffic (IVT) Detection and Filtration Guidelines (IAB/MRC addendum; MRC standards PDF). It splits invalid traffic into two tiers, and the distinction maps directly onto the filtering layers below. General Invalid Traffic (GIVT) is the unsophisticated, list-detectable kind: known spiders and robots identified from published bot lists, data-center / private-IP traffic, “auto-reloaders” that generate impressions on a periodic clock, IAB “dummy” bots, and fast-clickers — clicks that originate less than one second after the impression was served (a human cannot have read and decided to click that fast). Sophisticated Invalid Traffic (SIVT) is everything that evades the lists: traffic that requires advanced analytics, multi-point corroboration, and significant human intervention to identify — hijacked devices, click farms with rotating residential IPs, and adversaries that mimic human session shapes. The industry-standard rule is that any GIVT/SIVT impression or click must be removed from billable reporting and the advertiser not charged for it. Industry estimates (which are not audited public figures — see §2.2) put total IVT in the rough band of 5–25% of unfiltered ad clicks. The filtering layers below correspond to the GIVT/SIVT split: layers 1–4 are largely GIVT (lists, IP ranges, rate, fast-clickers), while layers 5–8 (click-farm correlation, reach analytics, ML scoring, honeypots) target SIVT:

  1. Bot user-agent blocklist. Known crawlers (Googlebot, Bingbot, etc.) and known headless-browser signatures.
  2. Datacenter IP detection. Clicks from AWS / GCP / Azure / DigitalOcean IP ranges are usually bots. Maintained as a CIDR blocklist.
  3. Geographic / time anomalies. A click from a country that’s never seen for this advertiser, at 3am local time, is suspicious.
  4. Rate per IP / per user-fingerprint. A single IP issuing 100 clicks/sec on different ads is a click farm.
  5. Click farm patterns. Multiple users on different IPs producing identical click patterns — coordinated activity.
  6. Reach analytics. A user-fingerprint that suddenly visits 100 advertiser pages in a row, none for more than 2 seconds, is bot.
  7. ML scoring. Random forest or gradient boosting model trained on labeled fraud data; outputs fraud_score ∈ [0, 1]. Above threshold (e.g., 0.7), the click is suppressed.
  8. Honeypot ads. Invisible ad slots that real users never click (because they’re not visible). Any click is by definition automated.

The fraud detector typically runs asynchronously downstream of the redirect (the user already got their 302), in a Fraud Detection Pipeline System Design sibling system. Suppressed clicks are excluded from billing; advertisers are not charged.

A real-world dynamic: ad networks have legal obligations (and audit requirements) to refund invalid traffic. The IAB / MRC accreditation requires demonstrable IVT filtering; networks publish quarterly IVT rates in their financial reports.

8.8 GeoIP Enrichment

Per-click enrichment from IP to (country, city, ISP, organization) using a GeoIP database (MaxMind or Google’s). Useful for fraud (datacenter IP detection), targeting (localized ads), and reporting (clicks-per-country dashboards). Done in the enrichment service downstream of ingest.

8.9 Exactly-Once Billing via At-Least-Once + Dedup

Kafka and most distributed systems give at-least-once delivery (a message may be delivered twice on consumer failure-and-restart). Billing must not double-charge. Solution: at the billing aggregator, before incrementing the per-advertiser spend counter, check the click_id against a dedup store. If already counted, skip; else increment and record click_id as counted.

The dedup store is the canonical bottleneck. Pattern: Redis with persistence + cross-shard replication for the recent (last hour) click_ids; a Cassandra table for the long-tail (older than 1 hour, within attribution window).

This is the “transactional write” pattern: the increment of the spend counter and the addition of click_id to the dedup store must be atomic. On most KV stores, this is a conditional write: SET click_id IF NOT EXISTS followed by INCR spend; if the SET fails (already exists), don’t INCR.

8.10 Privacy Regulations and Their Operational Impact

8.10.1 GDPR (EU)

Pixel-based tracking on EU users requires explicit consent. Most publishers display a cookie banner; tracking is disabled until consent. Click trackers must check a consent flag before logging; events without consent are dropped or anonymized (drop IP, hash user_fp).

8.10.2 CCPA (California)

Similar to GDPR but with opt-out rather than opt-in. The “Do Not Sell My Personal Information” link must suppress data sharing.

8.10.3 Apple ATT (App Tracking Transparency)

Since iOS 14.5 (released April 2021), apps must request explicit user permission via the App Tracking Transparency framework before accessing the IDFA (Identifier for Advertisers); if permission is denied the app receives an all-zeros IDFA. Opt-in rates are low — and notably volatile across measurement vendors and over time. Early post-launch tracking put global opt-in at roughly 15–25% (so 75–85% opting out), but later snapshots are higher: Statista reported ~46% global opt-in as of March 2022 (Statista), with finance/utility apps higher (~53%) and education/health lower (~41–42%). The takeaway for the architect is directional rather than a fixed number: a large fraction of iOS users withhold the IDFA, so ad networks lose deterministic per-device tracking for that segment and fall back to Apple’s SKAdNetwork (now StoreKit Ad Network) for aggregated, delayed, k-anonymous conversion reporting.

The browser landscape here changed materially and is widely misremembered. Safari has blocked third-party cookies by default since 2020 (Intelligent Tracking Prevention, building on work from 2017), and Firefox has blocked them by default since 2019 (Enhanced Tracking Protection). Google Chrome — which holds roughly two-thirds of the global browser market and was therefore the decisive actor — announced in 2020 a plan to deprecate third-party cookies “within two years,” then slipped the deadline repeatedly (2022 → 2023 → 2024 → early 2025).

That plan was abandoned. In July 2024 Google publicly ended its cookie-deprecation plan for Chrome, opting instead for a “user-choice” model layered onto Chrome’s existing privacy controls (Digital Commerce 360, 2024-07-24); in April 2025 it confirmed it would not ship the one-time deprecation prompt and would keep today’s cookie controls. A major driver was regulatory: the UK Competition and Markets Authority (CMA) was concerned that unilateral deprecation by the owner of Chrome, Google Ads, and a vast first-party data trove would entrench rather than reduce Google’s advantage. So as of writing (May 2026), third-party cookies still work in Chrome — but Safari and Firefox already block them, and the strategic direction remains away from cross-site cookie tracking. The engineering implication is unchanged even though the timeline reversed: a click-tracking system cannot assume a third-party cookie reaches a meaningful share of users. The workarounds remain the load-bearing patterns:

  • Server-side tracking. Publisher’s first-party cookie identifies the user; the publisher’s server forwards events to the ad network’s API. Privacy-preserving in name only — the same data is still collected, just routed through the first party.
  • Aggregation-only telemetry. Google’s Privacy Sandbox APIs (Topics, Protected Audiences — formerly FLEDGE, and the Attribution Reporting API) return aggregated, k-anonymous data without per-user identifiers. Note these now coexist alongside third-party cookies rather than replacing them.
  • Hashed deterministic identifiers. Email-hash or phone-hash matching (“hashed PII”) to identify the same user across publishers.

This is an area of acute regulatory and engineering churn; the correct architecture in 2026 differs from both 2018 (cookie-everywhere) and the 2022 expectation (imminent cookieless world that never arrived in Chrome).

8.11 Real-Time Spend Counter and Budget Enforcement

Per-advertiser daily / monthly budgets. The billing aggregator increments the spend counter (in a Distributed Counter System Design-style sharded setup) on each billable click. A separate budget-enforcer service monitors the counter; when spend approaches the cap (e.g., 95% of daily), it throttles or stops serving the advertiser’s campaigns.

Critical timing: the budget enforcer must react within seconds. If a campaign overspends because the enforcer is lagged by 5 minutes, the ad network eats the difference (advertisers refuse to pay over budget). The path from click → spend counter → enforcer must be < 30s end-to-end.

9. Scaling Strategy

9.1 What Breaks First

  1. Edge collector latency under load. Solved by horizontal scaling at the CDN edge.
  2. Kafka publish throughput. Solved by partitioning Kafka topics by customer_id or region.
  3. Dedup store at peak rates. Solved by sharding Redis dedup cluster + Bloom filter pre-check.
  4. Fraud detector ML latency. Async; doesn’t block the ingestion path.
  5. Spend counter contention at hot advertiser. Solved per Distributed Counter System Design §8.2 (sharded counters).

9.2 Sharding

  • Click events by click_id (UUID) — uniformly distributed.
  • Conversions by click_id (joined with the corresponding click).
  • Spend counters by (advertiser_id, shard) (sharded counter pattern).

9.3 Geo-Replication

Per-region edge collectors + per-region Kafka clusters. Events replicate cross-region for the attribution engine (which needs the global view of a user’s clicks across regions if the user travels) and for global reporting.

10. Real-World Example

Snowplow (open-source, cited above). Architecture: Snowplow JS / mobile / server SDK → Stream Collector (cloud-native, runs on AWS/GCP) → Stream Enrich (geo, UA, custom enrichments) → Loader to a warehouse (Redshift, Snowflake, BigQuery, Postgres, ClickHouse, Databricks). Snowplow open-sources its tracker SDKs and pipeline; customers run their own instance. Attribution and dashboards are downstream tools (Snowplow itself does not implement attribution models; users SQL-query the loaded events). Crucially for an interview, Snowplow demonstrably operates at the billions-per-day scale this design targets, not merely millions: Snowplow’s own customer materials describe Strava processing on the order of 3 billion events per day (peaking ~4.4 billion) into its warehouse via Snowplow (Snowplow — Strava), and a published community case study covers a single Snowplow + Snowflake deployment that handled 268 billion events (Snowplow community). So the reference architecture is a credible answer to “show me one that scales,” subject to the chosen warehouse’s own limits.

Google Analytics 4 (GA4 cited above). The successor to Universal Analytics. Event-based model (vs UA’s pageview/session model). Measurement Protocol API for server-to-server tracking. Default integration uses the gtag.js library; first-party cookies preferred over third-party. Backend architecture is undocumented but presumably built on Google’s BigQuery-class warehouse and ad-internal pipelines.

Adobe Analytics. Similar architecture; tracking pixel + JS library + S2S API. Workspace product layered on top for analyst queries.

Branch / Adjust / AppsFlyer (mobile attribution). Specialized in mobile install attribution — the click happens in a mobile browser ad; the conversion happens in an installed app. They reconcile the two using device fingerprinting and probabilistic matching when deterministic identifiers (IDFA) are unavailable.

Major ad networks (Google Ads, Meta Ads, TikTok Ads). Internal click pipelines mirror this architecture; details are largely proprietary. Public information includes IAB-disclosed IVT rates and high-level papers on auction theory (Mehta et al. 2007 cited above) but not implementation details.

Uncertain

Verify: the internal pipeline architecture of the major ad networks (Google Ads, Meta Ads, TikTok Ads). Reason: these are proprietary; public documentation covers only the API surface (Google’s Measurement Protocol, Meta’s Conversions API) and the attribution-model menu, not the ingest/dedup/billing internals. To resolve: an engineering blog or conference talk from the relevant team would be needed. Snowplow remains the most-documented open-source point of reference and is the safe one to cite concretely. #uncertain

11. Tradeoffs

Design ChoiceOption AOption BWhen A winsWhen B wins
Tracking methodPixel (1×1 GIF)JS event libraryUniversal browser supportRich client-side capture
Click loggingSynchronous redirectAsync after 302Strict accountingUser latency priority
AttributionLast-touchData-driven (Shapley)Simple, fast, transparentAccurate, expensive
DedupBloom filter onlyBloom + Redis storeBest-effort dedupBilling-grade exactly-once
CookiesThird-partyFirst-party + server-sideCross-site tracking (legacy)Privacy-compliant (modern)
IdentifierIDFA / cookieHashed PII or anonymousHigh match rate (legacy)Privacy-compliant
Fraud filterHeuristicsML modelExplainable, fastHigher accuracy
Real-time pipelineKafka + custom consumersStreaming SQL (Materialize)Mature, debuggedModern, less code
StorageSingle warehouseMulti-tier (hot+cold)Modest scalePetabyte scale

12. Pitfalls

  1. Synchronous click logging in the redirect path. Adds tens of milliseconds; users notice. Always async after the 302.

  2. No click_id token. Without a stable client-generated token, retries double-count or create phantom clicks. Always require a token; generate edge-side if missing.

  3. Logging IP without consent in the EU. GDPR violation; fines are real. Always check consent before storing IP; truncate or hash if not consented.

  4. Trusting the destination URL parameter. A malicious user can craft &dest=https://malicious.example.com and use the tracker as an open redirector for phishing. Always validate the destination against an allowlist (per advertiser or per campaign).

  5. Counting bot clicks as billable. Industry-standard fraud-filter SLA: less than 5% IVT in billed traffic. Without filtering, the ad network defrauds its advertisers — leads to refunds and lost contracts.

  6. Attribution model mismatch. Reporting dashboards using last-touch while billing uses data-driven causes inconsistency between what advertisers see and what they pay. Single source of truth for the model.

  7. Failing to handle the 30-day attribution window. A click followed by a conversion 31 days later is not attributed. If the advertiser sees the conversion in their CRM and complains, the discrepancy is the window. Document and configure.

  8. Pixel cache hits. Browsers cache the 1×1 GIF response; subsequent impressions for the same ad don’t re-fire. Always include a cache-buster query parameter (cb=<random>) and Cache-Control: no-store.

  9. Cookie domain mismatch. Setting a cookie on the tracker domain and trying to read it on the advertiser domain doesn’t work (different domain). Use first-party cookie strategies on the advertiser’s domain.

  10. Forgetting bot user-agents. A click from “Googlebot/2.1” should not be billable. Maintain and update the bot user-agent list.

  11. Race condition on conversion attribution. If conversion arrives before the corresponding click event is processed (out-of-order Kafka), the attribution lookup fails. Buffer conversions for a few seconds before attempting attribution; or maintain a “delayed attribution” queue.

  12. Privacy compliance bolted on as an afterthought. Architectures designed without consent / opt-out / privacy-budget hooks are extremely expensive to retrofit. Bake compliance into the schema (every event carries a consent_state field).

13. Common Interview Variants and Follow-Ups

  • “Attribute conversions across devices (web → mobile).” Cross-device attribution requires deterministic ID (logged-in user) or probabilistic matching (graph-based ID resolution). Discuss FB’s “People-Based Marketing” architecture.
  • “Build the budget enforcer that stops a campaign within seconds of hitting cap.” Real-time spend counter + watcher service + push to ad-server cache to stop serving the ad.
  • “Design for SOX / GDPR audit (every click traceable).” Append-only event log with immutable tombstones; cryptographic hash chain for tamper-evidence.
  • “Detect a coordinated click-farm campaign.” Graph anomaly detection: clusters of user-fingerprints with synchronous click patterns and shared destination targets.
  • “Build the conversion postback retry mechanism.” Exponential backoff; idempotent on the receiving side via conversion_id + external_id dedup.
  • “What if the third-party cookie is permanently disallowed (Privacy Sandbox future)?” Aggregation-only telemetry via Attribution Reporting API; deterministic only via logged-in identifiers; probabilistic matching for the unidentified.
  • “Aggregate conversion latency: when does today’s data become ‘final’?” End-of-attribution-window + reconciliation pass. Most ad networks report final numbers 24–48h after the day in question.
  • “Apple SKAdNetwork integration.” A different paradigm — the OS reports conversions, not the network; conversions are aggregated and delayed (24–48h) for k-anonymity. Architecturally a separate ingestion path.

14. Open Questions / Uncertain

Uncertain

Verify: the exact fraud-detection algorithms (specifically the ML model families and features) used internally by Google Ads / Meta Ads. Reason: these are proprietary, undisclosed, and change over time; public material is heuristic-level only. To resolve: nothing short of an internal disclosure would settle it — treat the §8.7 layering as the publicly documentable model (grounded in the IAB/MRC IVT taxonomy), not a claim about any one network’s internals. #uncertain

The future of click tracking in a post-third-party-cookie world is no longer the “imminent cookieless” frontier it was framed as in 2022–2023. As corrected in §8.10.4, Google reversed Chrome’s deprecation plan in July 2024 and confirmed in April 2025 it would keep third-party cookies under a user-choice model (Digital Commerce 360). The Privacy Sandbox APIs now coexist with cookies rather than replacing them. The architecture is unsettled by policy churn, not by an inevitable cookie removal — a meaningful distinction this note now reflects rather than flags.

Snowplow’s reference architecture does scale to billions of events per day in documented production (see §10: Strava at ~3 billion/day, a 268-billion-event Snowplow + Snowflake case study) — the earlier flag claiming “published case studies are at the millions-per-day scale” was simply wrong and has been corrected. The remaining genuine variable is the warehouse destination’s own scaling limits (Redshift vs Snowflake vs BigQuery vs ClickHouse), not the Snowplow collection/enrichment pipeline.

  • Is data-driven Shapley attribution provably more accurate than position-based heuristics, given typical conversion path lengths (median 1–3 clicks)? Recent academic work suggests the gap is small for short paths.
  • How should the dedup window scale with attribution window — 30 days of dedup state at a billion-clicks-per-day scale is petabyte-grade.
  • Can per-event differential privacy noise be added at the collector to satisfy strict privacy budgets without destroying reporting accuracy? Recent work on the SKAdNetwork / Privacy Sandbox approach gives one answer.

15. See Also

Graph View

Backlinks