vault

URL Shortener System Design

25 min read

URL Shortener System Design

A URL shortener maps a long Uniform Resource Locator (URL) — possibly hundreds of characters of query string and tracking parameters — to a short, sharable, opaque identifier such as https://bit.ly/3xQ9aZ. When the short URL is visited, the service issues an HTTP redirect (typically 301 Moved Permanently or 302 Found) to the original long URL. The canonical industrial deployments are Bitly (general-purpose, founded 2008), TinyURL (general-purpose, founded 2002), and Twitter t.co (Twitter-internal, mandatory since 2011 for safety/length-counting). The system appears in nearly every introductory system-design interview because it is small enough to scope in 45 minutes yet exercises every fundamental theme: identifier generation under collision risk, write-once read-many caching, hot keys, sharding, asynchronous analytics, abuse mitigation, and the deceptively-subtle choice between 301 and 302 redirects.

1. Why This System Appears in Interviews

Interviewers like URL shortener problems because the requirements look trivial — “map A → B and redirect” — but the design choices proliferate the moment one starts asking depth questions:

  • How do you generate the short code without collisions across a fleet of write servers?
  • How do you keep redirect latency below the user’s noticing threshold (typically < 200ms wall-clock, < 100ms server-side p99)?
  • How do you store 100 million new URLs per month for a decade without the database becoming the bottleneck?
  • How do you count clicks per short URL without serializing every redirect through a write?
  • How do you stop someone from registering bit.ly/login and pointing it at a phishing page?

Every one of these has a defensible textbook answer, and an interviewer can probe arbitrarily deep on any of them. The problem also has a clean read-to-write asymmetry (about 100:1 in real workloads, per Bitly’s published numbers — see §10) that motivates aggressive caching, which is the most-tested system-design lever after sharding.

2. Requirements

2.1 Functional Requirements

  1. Shorten. Given a long URL, return a short URL. The short code is a few characters of [A-Za-z0-9] (Base62 alphabet — 62 possible characters per position).
  2. Redirect. Given a short URL, redirect the browser to the long URL via an HTTP 3xx response.
  3. Custom alias (optional). Allow a user to request a specific short code (e.g., bit.ly/launch-2026), provided it is not taken.
  4. Expiration (optional). Allow a user to set a TTL (Time-To-Live) after which the short URL stops resolving. Default: never expire.
  5. Click analytics. Maintain per-short-URL click counts, with breakdowns by referrer, country, device, and time window (hourly / daily). Analytics are eventually consistent — they may lag the redirect by seconds to minutes.
  6. Authentication for management. Owners of an account can list and revoke their short URLs.

2.2 Non-Functional Requirements

  1. Low-latency redirect. End-to-end p99 < 100 ms server-side (the round-trip from the user is dominated by the public internet, which we cannot control).
  2. High availability. 99.99% (≈ 52 minutes downtime/year) for the redirect path. The shorten path can tolerate slightly less (99.9%) — users will retry.
  3. Scalability. Sustain 100 million new short URLs per month at peak times, and a 100:1 read:write ratio meaning 10 billion redirects per month.
  4. Durability. Once a short URL is created, it must keep resolving for the lifetime of the service (subject to expiration). Loss of a short → long mapping is data loss.
  5. Abuse resistance. Detect and block short URLs pointing to malware, phishing, or copyright-violating destinations.
  6. Geographically distributed. Redirects served from edge locations close to the user.

Uncertain uncertain

Verify: Bitly’s actual QPS, storage size, and availability SLA. Reason: not disclosed in any primary Bitly source; the figures above are interview-canonical design budgets (100M new URLs/month, 100:1 read:write, 99.99% redirect availability), not measured Bitly numbers. To resolve: a first-party Bitly capacity/SLA disclosure would settle it — none currently exists publicly. Treat these as a design budget any defensible system should hit, not as a citation of Bitly’s real numbers.

3. Capacity Estimation

The arithmetic below is the foundation of every later decision. Show your work in the interview — the numbers drive your sharding factor, cache size, and bandwidth provisioning.

3.1 Write QPS (Queries Per Second)

new short URLs per month       = 100,000,000  (100M)
seconds per month              ≈ 30 × 24 × 3600
                              = 2,592,000 s
average write QPS              = 100,000,000 / 2,592,000
                              ≈ 38.6 writes/s
peak write QPS (3× average)    ≈ 116 writes/s

A few hundred writes per second is trivial — a single PostgreSQL primary handles this without breaking a sweat. The write path is not the scaling pressure point; the read path is.

3.2 Read QPS

read:write ratio               = 100:1
average read QPS               = 38.6 × 100
                              ≈ 3,860 reads/s
peak read QPS (5× average,
   accounting for diurnal +
   viral spikes)               ≈ 19,300 reads/s

About 20K read QPS at peak. This is enough that a single relational database primary with naive lookups would not be reliable; we need a cache in front, and we need to plan for sharding once we exceed what a cache layer can hold.

3.3 Storage

Per record:

short_code        7 bytes  (Base62, 7 chars — see §8.1)
long_url        500 bytes  (typical max; URLs can legally be longer)
created_at        8 bytes
user_id           8 bytes
expires_at        8 bytes  (nullable)
flags             4 bytes  (custom alias, banned, ...)
overhead         50 bytes  (B-tree index, row overhead)
                ─────────
total          ≈ 585 bytes ≈ 600 bytes per record

Annual ingestion:

records/year    = 100M × 12 = 1.2 × 10^9 records
storage/year    = 1.2 × 10^9 × 600 bytes
                = 7.2 × 10^11 bytes
                ≈ 720 GB/year
ten-year storage ≈ 7.2 TB (raw, before replication)
with 3× replication ≈ 21.6 TB

A few terabytes per decade. A single modern server with NVMe SSD can hold this, but for redundancy and read throughput we will shard across many smaller machines (see §9.2).

3.4 Bandwidth

A redirect response body is essentially empty — only headers (Location: <long URL>, a few hundred bytes). Treating headers as ~600 bytes:

peak read bandwidth ≈ 19,300 reads/s × 600 bytes
                   ≈ 11.6 MB/s
                   ≈ 92 Mbps

About 100 Mbps egress for redirects. Negligible compared to the bandwidth of the content the user goes on to fetch from the long URL — which is not our problem.

3.5 Cache Sizing

If the access distribution is Pareto-like (top 20% of URLs see 80% of clicks — empirically reasonable for shareable content), then keeping the top 20% of recent records in cache covers ~80% of reads. Cache only the recent year (assume older URLs are essentially cold):

recent records  = 1.2 × 10^9 (one year worth)
top 20%         = 2.4 × 10^8 records
cache entry     ≈ 200 bytes (short_code + long_url + minimal metadata)
total cache     ≈ 4.8 × 10^10 bytes
                ≈ 48 GB

A few Redis instances of 32–64 GB cover this. With consistent hashing across cache nodes (see Consistent Hashing) we both shard the working set and survive single-node failures with minimal cache reshuffling.

4. API Design

The external Application Programming Interface (API) is REST-shaped. The two important endpoints are POST /api/v1/shorten and GET /:short_code.

4.1 Shorten

POST /api/v1/shorten
Authorization: Bearer <JWT>
Content-Type: application/json

{
  "long_url":     "https://example.com/path?utm=campaign&ref=abc&...",
  "custom_alias": "launch-2026",          // optional
  "expires_at":   "2027-05-08T00:00:00Z"  // optional, ISO-8601
}

Successful response:

HTTP/1.1 201 Created
Content-Type: application/json

{
  "short_url":  "https://bit.ly/launch-2026",
  "short_code": "launch-2026",
  "long_url":   "https://example.com/...",
  "expires_at": "2027-05-08T00:00:00Z",
  "created_at": "2026-05-08T14:23:11Z"
}

Error cases: 409 Conflict on alias collision, 400 Bad Request on malformed long URL, 429 Too Many Requests on rate limit, 403 Forbidden if the long URL is on a blocklist.

4.2 Redirect

GET /launch-2026 HTTP/1.1
Host: bit.ly

Response:

HTTP/1.1 302 Found
Location: https://example.com/path?utm=...
Cache-Control: private, max-age=90
Content-Length: 0

The interview-favorite question: 301 vs 302. 301 Moved Permanently lets browsers cache the redirect indefinitely — meaning subsequent visits skip our server entirely (great for latency, terrible for click analytics, since we never see the click). 302 Found (and 307 Temporary Redirect) tell browsers not to cache, so every click traverses our infrastructure. Most production shorteners use 302 precisely so that analytics work; the latency tradeoff (one extra round-trip per click) is accepted as the price of attribution. RFC 7231 §6.4.3 specifies that 302 does not authorize caching unless an explicit Cache-Control header opts in, which is what we want.

4.3 Analytics (separate read-only endpoint)

GET /api/v1/clicks/launch-2026?since=2026-05-01&granularity=day
→ 200 OK
{
  "short_code": "launch-2026",
  "total_clicks": 12483,
  "by_day": [...],
  "top_countries": [...],
  "top_referrers": [...]
}

5. Data Model

5.1 Primary Mapping Table

CREATE TABLE short_urls (
    short_code    VARCHAR(10)  PRIMARY KEY,        -- 7-char Base62 typical
    long_url      VARCHAR(2048) NOT NULL,
    user_id       BIGINT       NULL,
    created_at    TIMESTAMP    NOT NULL DEFAULT CURRENT_TIMESTAMP,
    expires_at    TIMESTAMP    NULL,
    is_custom     BOOLEAN      NOT NULL DEFAULT FALSE,
    is_banned     BOOLEAN      NOT NULL DEFAULT FALSE,
    INDEX idx_user_created (user_id, created_at DESC),
    INDEX idx_expires (expires_at) WHERE expires_at IS NOT NULL
);

The primary key is short_code because every read is a point-lookup by short code. Range-scans by user are secondary and supported by the composite index (user_id, created_at DESC).

The table can be modeled as a key-value store (DynamoDB, Cassandra, Bigtable) just as easily — and for scale beyond a single PostgreSQL primary, we will (§9.2). A KV layout in Cassandra:

Keyspace:    shortener
Table:       short_urls
Partition:   short_code         -- hashes to one of N nodes
Clustering:  (none)
Columns:     long_url, user_id, created_at, expires_at, is_custom, is_banned

The short_code is the partition key, ensuring uniform distribution across nodes via Consistent Hashing-based partitioning. Range scans by user_id require a secondary index or a separate denormalized table urls_by_user.

5.2 Click Counters (Analytics Path)

Click events are recorded in a separate write-optimized table or stream — we do not update the primary mapping table on each click. Updating a row 19,000 times per second across hot URLs would create write contention; instead, click events flow into a Kafka topic (or equivalent — see Distributed Log System Design) and are aggregated asynchronously:

CREATE TABLE click_events_hourly (
    short_code VARCHAR(10),
    hour_utc   TIMESTAMP,
    country    VARCHAR(2),
    referrer   VARCHAR(255),
    device     VARCHAR(20),
    count      BIGINT,
    PRIMARY KEY (short_code, hour_utc, country, referrer, device)
);

The aggregation job (a stream processor: Flink, Kafka Streams, or Spark Structured Streaming) reads the raw event stream and rolls up per-hour bucketed counts. The user-facing analytics endpoint queries this table.

For the grand-total click counter per short URL, a sharded distributed counter (see Distributed Counter System Design) is appropriate — naive UPDATE ... SET count = count + 1 does not survive 19K writes/s on hot keys.

6. High-Level Architecture

flowchart TB
    User[User Browser] -->|GET /code or POST /shorten| Edge[CDN / Edge POPs]
    Edge -->|cache miss / write| LB[L7 Load Balancer]
    LB -->|GET /code| ReadSvc[Redirect Service<br/>stateless, autoscaled]
    LB -->|POST /shorten| WriteSvc[Shorten Service<br/>stateless, autoscaled]
    ReadSvc -->|cache get| Redis[Redis Cluster<br/>consistent-hashed shards]
    ReadSvc -->|cache miss| KV[(Sharded KV Store<br/>Cassandra / DynamoDB)]
    WriteSvc -->|insert if not exists| KV
    WriteSvc -->|fetch ID| IDGen[ID Generator<br/>counter or Snowflake]
    ReadSvc -->|emit click event| Kafka[Kafka — clicks topic]
    Kafka --> Aggregator[Stream Processor<br/>Flink / Spark]
    Aggregator --> AnalyticsDB[(Analytics DB<br/>ClickHouse / Druid)]
    User2[Account Owner] -->|query analytics| AnalyticsAPI[Analytics API]
    AnalyticsAPI --> AnalyticsDB
    WriteSvc -->|safety check| Safety[Abuse Detection<br/>blocklist + ML scoring]

What this diagram shows. The system splits cleanly into a synchronous redirect path (top half: edge → load balancer → redirect service → cache → KV store) and an asynchronous analytics path (right side: Kafka → stream processor → analytics database). Each path scales independently. The edge tier (CDN POPs) terminates TLS close to users and serves cached redirects directly when possible. The redirect service is stateless, so we autoscale horizontally on read QPS without worrying about session affinity. The KV store is sharded by short_code via consistent hashing to spread load. The shorten service consults an ID generator (a separate concern — see §8.1) and writes to the KV store. Click events fork off the redirect path into Kafka; the redirect itself does not block on analytics. The abuse-detection module (left of the shorten service) inspects every long URL against a blocklist and ML-based scoring before allowing the write to proceed.

7. Request Flow / Sequence

7.1 Shorten (Create)

sequenceDiagram
    participant U as User
    participant W as Shorten Service
    participant ID as ID Generator
    participant S as Safety Checker
    participant KV as KV Store
    participant C as Cache (Redis)

    U->>W: POST /api/v1/shorten {long_url, custom_alias?}
    W->>S: check_safe(long_url)
    S-->>W: ok / blocked
    alt blocked
        W-->>U: 403 Forbidden
    else ok and custom_alias provided
        W->>KV: INSERT IF NOT EXISTS (custom_alias, long_url, ...)
        KV-->>W: ok / conflict
        alt conflict
            W-->>U: 409 Conflict
        else ok
            W->>C: SET short_code → long_url (TTL = 7d)
            W-->>U: 201 Created {short_url}
        end
    else ok and no custom alias
        W->>ID: next_id()
        ID-->>W: 64-bit numeric id (e.g., 18,251,902,341)
        W->>W: short_code = base62_encode(id)  → "4Bf7Qx9"
        W->>KV: INSERT (short_code, long_url, ...)
        KV-->>W: ok
        W->>C: SET short_code → long_url
        W-->>U: 201 Created
    end

7.2 Redirect (Read)

sequenceDiagram
    participant B as Browser
    participant CDN as CDN Edge
    participant R as Redirect Service
    participant C as Cache (Redis)
    participant KV as KV Store
    participant K as Kafka

    B->>CDN: GET /4Bf7Qx9
    alt edge cache hit (rare for 302; common for 301)
        CDN-->>B: 302 Location: <long_url>
    else edge miss
        CDN->>R: GET /4Bf7Qx9
        R->>C: GET 4Bf7Qx9
        alt cache hit
            C-->>R: <long_url>
        else cache miss
            R->>KV: SELECT long_url WHERE short_code = '4Bf7Qx9'
            KV-->>R: <long_url> | not_found
            alt not found
                R-->>CDN: 404
                CDN-->>B: 404
            else found
                R->>C: SET 4Bf7Qx9 → <long_url> (TTL = 1h)
                R-->>CDN: 302 Location: <long_url>
                CDN-->>B: 302
            end
        end
        R->>K: emit click event {short_code, ts, ip, ua, referrer}
    end

The click event is emitted after the response is sent (or in a background task, depending on the framework) to avoid stretching the redirect’s latency budget.

8. Deep Dive

8.1 Short Code Generation

This is the single most-probed corner of the design. There are three families of approaches; each has distinctive tradeoffs.

8.1.1 Hash-and-Truncate

Compute MD5(long_url) (or SHA-256(long_url)), take the first N characters of the Base62-encoded digest. Pros: deterministic — the same long URL produces the same short code, useful for deduplication. Cons: collisions. With 7-character Base62 codes, the space is 62⁷ ≈ 3.5 × 10¹² codes. After inserting ~√(3.5×10¹²) ≈ 1.9 million codes, the birthday paradox makes a collision likely. So every write must do a check-then-insert (SELECT WHERE short_code = ? followed by INSERT), which costs at least one extra round-trip and is racy unless the storage engine supports INSERT IF NOT EXISTS atomically. Worse: when a collision happens, you have to alter the input — append a salt and re-hash — undoing the determinism.

8.1.2 Counter + Base62 Encode

Maintain a single global counter; the next short code is base62_encode(counter). Pros: zero collisions by construction; the code is the shortest possible (a counter at 10¹² needs only 7 Base62 digits). Cons: a global counter is a single point of contention. Mitigations:

  • Range pre-allocation. Each shorten-service instance reserves a range of, say, 10,000 IDs at a time from the central counter. Within the range, the instance assigns IDs locally without crossing the network. The central counter is hit only on range exhaustion (hundreds of writes between RPCs). This is how URL shorteners typically run a SQL BIGINT counter.
  • Per-shard counters. Each KV-store shard maintains its own counter; the short code embeds the shard ID so reads can route correctly. Eliminates the central counter at the cost of slightly longer codes.

The Base62 encoding step:

ALPHABET = "0-9A-Za-z"  (62 chars)
def base62_encode(n):
    s = ""
    while n > 0:
        s = ALPHABET[n % 62] + s
        n //= 62
    return s
# example: base62_encode(18,251,902,341) = "JdW2Qb"  (~6 chars at 10^10)

A counter at 100 million writes/month × 12 months × 10 years ≈ 1.2 × 10¹⁰ stays within 7 Base62 chars (62⁷ = 3.5 × 10¹²). Comfortable for a decade.

8.1.3 Snowflake-style ID + Base62

Use Twitter’s Snowflake (64-bit ID composed of 41-bit timestamp + 10-bit machine ID + 12-bit sequence — see Twitter’s 2010 announcement). Each shorten-service instance generates IDs locally in microseconds, with no central counter at all. Encode the 64-bit ID in Base62 → ~11 chars (longer than the counter approach but still fine). Pros: fully decentralized; no coordination on the write path. Cons: longer codes than the pure counter; predictability of IDs leaks information (the timestamp is recoverable, which a paranoid security model dislikes).

8.1.4 The Choice

For a cleanly-designed greenfield system, counter + Base62 with range pre-allocation is the preferred answer in interviews because it produces the shortest possible codes, has zero collision logic, and the central-counter contention is mitigated by ranges. Bitly historically used a MySQL auto-increment counter, per their engineering blog. If the interviewer pushes on “what if the counter database fails?”, the answer is multi-master counter ranges (each region gets a disjoint range of the ID space) plus replication — a soft single-point-of-failure becomes a soft regional failover.

8.2 The Read Path and Cache Coherence

The redirect path is the hot path. Its latency budget at the application layer is roughly 10 ms (after subtracting network RTT and TLS termination). Within that budget:

  1. Cache lookup (Redis). ~0.5 ms median. A Redis cluster sharded by short_code via consistent hashing serves this.
  2. Cache miss → KV lookup. ~5–10 ms p99 for a single-row lookup in a properly indexed Cassandra/Dynamo. Fill the cache with a TTL of 1–24 hours so future hits skip the KV.
  3. Click event emission. Fire-and-forget into Kafka (~1 ms enqueue), or buffer locally and flush in batches.

Cache invalidation. Short URL records are immutable after creation in the simple design (the short code → long URL mapping never changes). This means we do not need cache invalidation on writes — write through, then the entry only needs to expire on TTL. For deletions or bans, we explicitly purge the cache entry. The “two hard things in computer science” cliché does not bite us here, which is why the system is interview-friendly.

Cache stampede. When a hot key expires from the cache and many concurrent requests miss simultaneously, all of them hit the KV store at once. Standard mitigations apply: probabilistic early refresh, or a request-coalescing layer (“singleflight”) that lets one request fill the cache while others wait.

8.3 Custom Alias Collisions

A user requests bit.ly/launch-2026. The system must atomically check that this code is unused and reserve it. In SQL: INSERT INTO short_urls (short_code, ...) VALUES ('launch-2026', ...) ON CONFLICT DO NOTHING. In Cassandra: a lightweight transaction (LWT) — INSERT ... IF NOT EXISTS. LWTs in Cassandra cost an order of magnitude more than a regular write because they require Paxos rounds (see Paxos High-Level); for the rare custom-alias case this is acceptable.

A subtler issue: custom aliases must not collide with the generated code space. Either reserve a separate prefix for custom aliases (e.g., they always contain a -, generated codes never do), or accept that the counter will occasionally produce a code that happens to match a previously-claimed custom alias and then skip on conflict.

9. Scaling Strategy

The progression below is the canonical “what breaks first” walk that interviewers expect.

9.1 Stage 1 — Single Server

A single server with PostgreSQL handles a few hundred QPS comfortably. Until ~1,000 read QPS with hot-key amplification, no special infrastructure is needed.

9.2 Stage 2 — Add Read Cache

Insert Redis in front of the database. With 90%+ cache hit rate (the typical Pareto-skewed access pattern delivers this easily), the database’s read load drops 10×. This buys you to ~10K–20K read QPS on a single primary database.

9.3 Stage 3 — Shard the KV Store

Beyond what one database primary can handle (write or storage), shard. The natural sharding key is short_code; partition via consistent hashing (see Consistent Hashing) across N nodes. Redis is similarly sharded (Redis Cluster does this natively, also via hash slots). A single shard now handles 1/N of the load, and adding shards moves only ~1/N of the data — minimal disruption.

9.4 Stage 4 — CDN for Redirects

Push redirects to the edge. Two flavors:

  • 301-cacheable redirects. If you do not need per-click analytics, a 301 Moved Permanently lets the user’s browser and the CDN cache the redirect for hours/days, removing your origin entirely from the hot path for repeat visits. Latency drops to whatever the CDN edge measures.
  • Edge KV (Cloudflare Workers KV, Fastly Compute@Edge). Replicate the short-code → long-URL map to edge locations. The edge serves the redirect with a few milliseconds of compute and still emits a click event back to origin (over an asynchronous queue). This is the modern best-of-both-worlds approach.

9.5 Stage 5 — Geo-Replication

Replicate the KV store across regions. Writes go to the user’s home region; reads are local. Cross-region replication is asynchronous (eventual consistency) — acceptable because URL records are immutable, so the only edge case is “create in region A, immediately read in region B” (rare; the redirect just looks like a momentary 404 that resolves within the replication lag of seconds).

9.6 What Breaks First

Empirically, the order is:

  1. Database write contention on the global counter. Solved by range pre-allocation.
  2. Database read QPS. Solved by Redis.
  3. Hot-key concentration. A single viral short URL can spike to 10K+ reads/s on one cache node. Solved by replicating hot keys to multiple cache nodes (key splitting) or using bounded-load consistent hashing.
  4. Click-event ingestion bandwidth. Kafka brokers per-topic-partition QPS limits matter. Solved by partitioning the Kafka topic with enough partitions to spread the load.
  5. Analytics aggregation lag. As click volume grows, the stream processor falls behind. Solved by adding more processor parallelism and pre-aggregating in Kafka via Kafka Streams.

10. Real-World Example

Bitly. Bitly disclosed in engineering blog posts (and at conference talks circa 2012–2018) that they use a combination of MySQL (sharded), Redis (caching), and a custom analytics pipeline. Their write path uses a database-backed counter with range pre-allocation per Bitly’s engineering blog. Reads are heavily cached in Redis with a multi-region setup. Bitly serves billions of clicks per month; their published numbers historically claim a 100:1 read-to-write ratio. (Bitly engineering blog, various; specific architecture details may have evolved.)

Twitter t.co. Twitter (now X) introduced the t.co link service in 2010 and made it mandatory for all links in 2011. It is not a public shortening service but an internal click-through-and-safety layer: every URL pasted into a tweet is wrapped in t.co at composition time, and the tweet stores both the original URL (for display) and the t.co URL (for click-through). The official t.co interstitial states the service exists “to protect users from harmful activity, to provide value for the developer ecosystem, and as a quality signal for surfacing relevant, interesting Tweets” — i.e. three co-equal stated purposes, of which safety (real-time scanning of destination URLs against malware/phishing/spam) is one (per the t.co interstitial text quoted by Mesh Security and the X URL-shortener help page). Note that t.co also de-couples the display length of a link from its true length, which mattered when tweets had a hard 140-character limit — but the published rationale leads with safety and ecosystem value, not character-counting.

TinyURL. Operating since 2002 — the original URL shortener at scale. Architecture is not publicly documented in any primary source I have verified; it is generally assumed to follow the same playbook (sharded KV, cache, CDN).

Uncertain uncertain

Verify: Bitly’s internal shard count and Redis cluster size, and the precise 100:1 read-to-write ratio. Reason: the architectural shape (sharded MySQL + Redis + custom analytics pipeline) is described across Bitly conference talks circa 2012–2018, but exact infrastructure sizing and the 100:1 figure are widely-repeated in interview literature without a citable Bitly publication behind them. To resolve: locate the original Bitly engineering post or talk that states the read:write ratio. Do not quote specific Bitly internals as fact in an interview.

11. Tradeoffs

Design ChoiceOption AOption BWhen A winsWhen B wins
Short code generationHash-and-truncate (MD5/SHA)Counter + Base62Need deterministic dedup (same URL → same code)Want shortest codes, zero collision logic
Redirect status code301 Permanent302 FoundCacheable, ultra-fast, no analytics neededNeed click analytics, accept extra round-trip
Storage backendRelational (PostgreSQL)KV store (Cassandra/Dynamo)< 100M records, one region, ACID needed for billingMulti-region, many billions of records, eventual consistency OK
Custom alias collisionHard fail (409)Suggest alternativeAdmin tool, expert usersConsumer product, new-user-friendly
Click counter strategySynchronous (UPDATE count + 1)Async event streamLow QPS, want immediate feedbackHigh QPS, OK with seconds-of-lag analytics
Multi-region writesSingle primary (failover)Multi-master (per-region writes)Simpler operationally, accept regional outage = read-onlyNeed write availability under regional failure, accept conflict resolution complexity
Cache evictionLRU (see LRU Cache)TTL onlyWorking set fits in cache, classic Pareto skewPredictable expiry policy, simpler reasoning

12. Pitfalls

  1. Forgetting to validate the long URL. Open-redirect bugs let attackers craft bit.ly/<your-domain-via-shortener> that bounces phishing through your service. Validate the long URL is well-formed and run it through a malware/phishing blocklist (Google Safe Browsing, PhishTank).

  2. 301 vs 302 confusion. Using 301 blocks analytics because browsers and CDNs cache permanently — first click hits you, subsequent clicks do not. Standardize on 302 for analytics-bearing shorteners; document explicitly why.

  3. The counter as a hot row. A single UPDATE counter SET value = value + 1 row gets lock contention at high write QPS. Range pre-allocation (each instance reserves a block of IDs) is the standard fix. Without it, your write path quietly degrades under load.

  4. Cache stampede on hot-key expiry. When a viral URL’s cache entry TTLs out, every concurrent reader re-fetches from the KV store at once — potentially thousands of duplicate queries. Use probabilistic early refresh (refresh near, but before, expiration with rising probability) or a singleflight pattern.

  5. Click counters that lie under contention. Naive UPDATE clicks = clicks + 1 on a hot row loses writes (or creates lock contention). Use a sharded distributed counter (Distributed Counter System Design) or async event aggregation.

  6. No abuse-rate-limiting on the shorten endpoint. Without a per-IP, per-user Token Bucket or Sliding Window Rate Limiter, a botnet can mint a billion short URLs in a day, exhausting the ID space and burying real users in spam.

  7. Banned-URL revocation latency. When a short URL turns out to point to malware, you need to revoke it everywhere — cache, edge KV, CDN. A revocation that propagates over hours is a compliance issue. Build a fast-purge path that targets all caching layers.

  8. Custom aliases that collide with reserved paths. /login, /admin, /api must be reserved against custom-alias claims. Maintain a blocklist of reserved short codes.

  9. Predictable IDs leaking creation order. If short codes are sequential Base62, an attacker can enumerate aaaaaaa, aaaaaab, ... and discover all shortened URLs ever created — including private ones meant to be secret. Either obfuscate the ID (e.g., XOR with a secret before encoding) or allow users to mark a short URL “private/no-listing” and serve 404 to any unauthenticated request.

  10. Forgetting that the long URL itself can be a shortener. Chains of bit.ly → t.co → short.io → ... are real. Detect cycles and limit chain depth at validation time to prevent loops.

13. Common Interview Variants and Follow-Ups

  • “Now add per-user analytics.” Add a user_id column on short_urls; aggregate clicks both per-short-URL and per-user. Touches data warehouse design.
  • “Now make it support 10× more traffic.” Push to edge KV; add per-region writes; possibly switch from generated short codes to per-region prefixes.
  • “How would you implement custom domain support (e.g., mycompany.link/foo)?” Add a domain column to the schema; the short code becomes (domain, code) instead of just code; route by domain at the edge. SSL certificate management becomes a new subsystem.
  • “What if the user wants the short URL to expire and recycle?” TTL index in the database; a background reaper deletes expired records. Decide whether to recycle the short code (risky — re-use confuses caches and old links suddenly resolve to new content) or leave it dead (410 Gone).
  • “How do you handle a viral short URL that goes to 100K QPS?” Hot-key replication: serve the same code from multiple cache shards. Bounded-load consistent hashing. Possibly serve directly from CDN with 301 once the URL is “verified safe”.
  • “Add A/B testing — different users see different long URLs from the same short.” A new dimension on the mapping table (variant_id); the redirect service consults a variant assignment service to choose which long URL.
  • “Build link-in-bio (a single short URL that resolves to a page with multiple links).” Now the short URL maps to a resource page not a redirect — substantially different system; the database stores HTML/JSON content keyed by code.

14. Open Questions / Uncertain

Uncertain uncertain

Verify: the exact short-code-generation strategy Bitly uses today. Reason: Bitly’s current engineering blog does not document its ID-generation internals; the “counter + Base62” pattern is confirmed as the industry-standard approach (multiple secondary write-ups describe a counter atomically incremented and Base62-encoded, sometimes with ZooKeeper-managed range allocation — e.g. Hello Interview’s Bitly breakdown), but I could not confirm whether Bitly specifically still uses a MySQL auto-increment counter versus a Snowflake-style decentralized scheme. To resolve: find a current first-party Bitly engineering post or conference talk describing their key-generation service.

Uncertain uncertain

Verify: that a real shortener sustains 90%+ cache hit rate. Reason: this figure is a modeling assumption for a Pareto-skewed (top-20%-of-URLs-get-80%-of-clicks) access pattern, not a measured number from a named system. Actual hit rate depends on the viral-hot vs long-tail traffic mix, which is workload-specific. To resolve: cite a published cache-hit-rate measurement from a production shortener — none is currently disclosed in a primary source.

15. See Also

Graph View

Backlinks