Sharded Architecture

A sharded architecture (also called horizontal partitioning or simply sharding) is the topology in which a dataset and its associated workload are split across N independent partitions (called shards), each owned by a distinct node or replica group, with a partition key determining which shard each piece of data belongs to. Reads and writes are routed to the shard owning the relevant key; cross-shard operations require either expensive coordination (Two-Phase Commit or higher-level distributed transactions) or are explicitly avoided in the data model. Sharding is the canonical answer to horizontal scaling beyond a single node’s capacity: a single database can store at most a few terabytes and serve at most a few hundred thousand operations per second; ten shards can store ten times that at proportional throughput. The partition strategy — hash partitioning via Consistent Hashing, range partitioning by ID range, geo partitioning by region, or directory-based via a lookup service — is the single most consequential design choice in any sharded system, because the partition key choice cannot easily be changed after data is loaded, and getting it wrong produces hot shards (one shard sees disproportionate traffic, throughput collapses) that are very expensive to fix. Sharded architectures appear in Cassandra rings, MongoDB sharded clusters, Vitess (the YouTube-derived MySQL sharding layer used by Slack, GitHub, Square), Twitter’s Manhattan datastore, Slack’s Vitess clusters (originally sharded per workspace, later resharded onto finer keys such as channel ID), DynamoDB’s partition-key model, and the storage engines of Spanner / CockroachDB / TiDB (where each “range” or “tablet” is a shard). The architecture is almost always composed with other architectures: a sharded system typically replicates each shard using Leader-Follower Replication Architecture or Leaderless Replication Architecture for durability, and then exposes the unified shard set through a routing layer that hides the partitioning from the application.

1. When to Use, When Not to Use

Use sharded architecture when:

• Your dataset exceeds a single node’s storage. A single relational database tops out around 10-100 TB before performance degrades; a single MongoDB or Cassandra node tops out around 4-10 TB of useful data. If your data is bigger, you must shard.
• Your write workload exceeds a single node’s IOPS. A leader-follower system bottlenecks on the leader. If your write rate exceeds ~50K-100K writes/sec sustained on the best available hardware, you must shard the write path across multiple leaders.
• You can identify a good partition key. Most systems have a natural partition key — user ID, tenant ID, geographic region, time bucket. If the partition key is obvious and the workload distributes evenly across keys, sharding is straightforward.
• Cross-partition queries are rare or avoidable. Most queries hit one shard (look up a user, fetch a tenant’s data). Cross-shard queries (analytics, reporting, joins) are infrequent and tolerated being slower.
• You need to scale linearly. Adding more shards adds more capacity. The architecture’s central scaling property.

Avoid sharded architecture when:

• Your dataset fits on a single node and your traffic doesn’t justify the complexity. Sharding adds enormous operational complexity. If a $100/month database server can handle the load, do not shard. The most common premature-sharding mistake.
• Your workload doesn’t have a natural partition key. If queries don’t have an obvious “key” they target — analytical workloads, complex joins, queries that aggregate across the whole dataset — sharding makes everything worse. Use a different architecture (data warehouse, denormalized analytics store).
• Cross-partition transactions are central to the workload. A bank that needs to atomically transfer between any two of N million accounts has cross-shard transactions on the foreground path. Two-Phase Commit makes them work, but at high latency and operational cost. Spanner / CockroachDB are designed for this; ad-hoc sharding is not.
• Your workload is dominated by a few hot keys. Sharding distributes load across many keys. A single hot key still gets all its traffic on its one shard. No amount of additional shards helps.
• You need to query the dataset by many different attributes. Sharding by user ID makes queries-by-user fast and queries-by-time slow (must visit all shards). If you need fast queries by both, you need either denormalization (dual-write into multiple sharded indexes) or a different architecture.

2. Structure

flowchart TB
    subgraph Clients
        C1[App Server 1]
        C2[App Server 2]
    end

    Router[Routing Layer<br/>partition_key -> shard_id<br/>e.g., consistent hashing,<br/>or directory lookup]

    subgraph Shard1[Shard 1: keys [0, 1000)]
        S1L[(Shard 1 Leader)]
        S1F1[(Shard 1 Follower)]
        S1F2[(Shard 1 Follower)]
        S1L --> S1F1
        S1L --> S1F2
    end

    subgraph Shard2[Shard 2: keys [1000, 2000)]
        S2L[(Shard 2 Leader)]
        S2F1[(Shard 2 Follower)]
        S2F2[(Shard 2 Follower)]
        S2L --> S2F1
        S2L --> S2F2
    end

    subgraph Shard3[Shard 3: keys [2000, 3000)]
        S3L[(Shard 3 Leader)]
        S3F1[(Shard 3 Follower)]
        S3F2[(Shard 3 Follower)]
        S3L --> S3F1
        S3L --> S3F2
    end

    C1 --> Router
    C2 --> Router
    Router -->|key=42 -> shard 1| S1L
    Router -->|key=1500 -> shard 2| S2L
    Router -->|key=2700 -> shard 3| S3L

    Coord[Coordination /<br/>Configuration:<br/>shard map,<br/>rebalance state] -.-> Router

    style S1L fill:#fab,stroke:#933
    style S2L fill:#fab,stroke:#933
    style S3L fill:#fab,stroke:#933

What this diagram shows. A sharded cluster of three shards, each independently replicated. The application clients (top) send requests through a routing layer that consults a shard map — a logical or physical directory mapping each partition key to its owning shard. For range-partitioned schemes, the shard map is a list of (start_key, end_key, shard_id) triples; for hash-partitioned schemes, it’s a consistent hash ring with shard-id annotations. The router computes the destination shard for each request and forwards it to that shard’s leader. Each shard internally is a Leader-Follower Replication Architecture group of three nodes: one leader handles writes (and serves linearizable reads); two followers serve eventually-consistent reads and provide hot-standby durability. The composition is sharding for horizontal scale, replication for durability per shard.

The coordination component (bottom) maintains the authoritative shard map and orchestrates rebalancing operations (when a shard splits, a new shard is added, or a shard is decommissioned). This is typically a small Raft or Paxos High-Level cluster — etcd, ZooKeeper, or an embedded consensus group. The router is a client-side library, a proxy (Vitess, ProxySQL), or a gateway service — but in all cases it consults the coordination layer for the current shard map.

The architectural property to internalize: the system has N independent leaders, each scaling its own shard. Total cluster throughput is the sum of per-shard throughputs. Cross-shard operations require coordinating multiple leaders — making them slower and harder to reason about.

3. Core Principles

Principle 1: Each piece of data has exactly one shard. Every key in the dataset belongs to one specific shard determined by the partition strategy. There is no ambiguity, no “this row could be on shard A or B.” The mapping is deterministic given the partition key.

Principle 2: Shards are operationally independent. A shard’s hardware failure, slow operation, or maintenance window affects only that shard’s data — not the rest of the dataset. This isolation is the architecture’s defining resilience property and the operational benefit over single-node systems.

Principle 3: The partition key choice is everything. The partition key (also called shard key or partitioning column) determines the entire distribution of data and load across shards. A good partition key produces uniform distribution; a bad one produces hot shards (some shards have far more data or far more traffic than others). The partition key must be chosen at design time and is essentially impossible to change later without an offline migration.

The weight of this principle is hard to overstate, so it deserves direct examples of how partition-key choice plays out across well-known production systems. Consider three contrasting cases. Case A — sharding by user_id (good). A consumer-facing application where most reads and writes are scoped to a single user — fetching a user’s profile, posting a status, listing the user’s recent activity. Hashing user_id distributes traffic uniformly (every user contributes roughly the same load over many users), every common query carries the user’s ID as a parameter so the router can localize to one shard, and adding capacity is straightforward via Consistent Hashing. This is the textbook case and the one most B2C systems should default to (Kleppmann DDIA 2017 Ch.6 walks through this in detail). Case B — sharding by created_at timestamp (bad). An event-tracking system that shards events by their creation time, with each shard owning a contiguous time range. The most-recent shard receives 100% of incoming writes; older shards become read-only and idle. The cluster’s effective write throughput collapses to a single node’s throughput regardless of how many shards exist. The DynamoDB documentation on partition keys explicitly warns against this anti-pattern; pre-2018 DynamoDB users hitting this received “hot partition” surcharges. Mitigation requires either changing the partition key (to a hash of something else) or using a composite key like (random_bucket, created_at) with random_bucket chosen from a small fixed range to spread writes across buckets — at the cost of making “events in time range” queries fan out across all buckets. Case C — sharding by tenant_id (nuanced). Multi-tenant SaaS where each customer (“tenant”) has their own data. Sharding by tenant gives transactional consistency within a tenant, simple authorization (tenant boundaries are shard boundaries), and natural rate-limit isolation. The pain emerges with tenant-size skew: if tenant 42 is 100× larger than the median tenant, shard-42 is overloaded while neighboring shards idle. Slack is the canonical cautionary tale here, and importantly its history runs the opposite direction from “tenant sharding is fine”: Slack originally sharded by workspace (team), with each shard holding thousands of workspaces and all their messages and channels — but as it onboarded ever-larger customers, a single big customer’s designated shard “reached the largest available hardware” and Slack “ended up with a few hot spots in [the] database tier” that could not be spread across the fleet (per Slack’s own Scaling Datastores at Slack with Vitess). Slack’s resolution was to abandon workspace-level sharding and reshard to far finer-grained keys — for example, sharding the messages table by channel ID rather than workspace, on the reasoning that a single channel is unlikely to outgrow a shard — using Vitess’s flexible resharding so application code did not have to change. The lesson generalizes, and it is sharper than “tenant sharding works until a tenant grows large”: sharding by tenant is attractive for isolation but carries a built-in failure mode — the largest tenant eventually outgrows the largest single machine, and at that point the only escape is to reshard onto a sub-tenant key (channel, user, object ID) or move the tenant to a dedicated cluster. Designs that assume tenant boundaries are permanent shard boundaries are setting up exactly the hotspot Slack hit.

Principle 4: Routing is a separate concern. The application does not (or should not) know which shard its query targets. The routing layer abstracts this away. The application sees a unified API; the router translates each request into the appropriate per-shard request. The router can be an application-side library, a network proxy, or a gateway service.

Principle 5: Cross-shard operations are expensive. A query that must visit multiple shards pays the fan-out cost — the latency of the slowest shard’s response. A transaction that must update multiple shards requires Two-Phase Commit or similar coordination. Designs minimize these.

Principle 6: Rebalancing is a continuous concern. Adding a shard, removing a shard, or splitting a hot shard requires moving data. The rebalance strategy must move minimum data, must work online (with traffic continuing), and must not corrupt during failure. Consistent Hashing minimizes data movement on add/remove; range-partitioning rebalances by splitting hot ranges.

The rebalancing problem deserves more depth because the naïve solutions are catastrophic and real systems use surprisingly different strategies. The textbook anti-pattern is hash-mod-N partitioning: assign each key to hash(key) mod N shards. When N changes (a shard is added or removed), the modulus changes, and nearly every key maps to a new shard. With N going from 10 → 11, approximately 91% of keys must move — the rebalance cost dwarfs the actual capacity addition. This is why hash-mod-N is essentially never used in production sharded systems despite being the obvious first design. Consistent Hashing (Karger et al. STOC 1997) replaces modular arithmetic with a hash ring: each shard occupies a position on a 2³²-point ring, each key maps to its position’s clockwise-next shard, and adding a shard relocates only the keys between the new shard’s position and the prior shard’s position — approximately K/N keys for an even distribution, two orders of magnitude better than hash-mod-N. The Cassandra (Lakshman + Malik 2010) and DynamoDB (DeCandia et al. SOSP 2007) implementations both refine this with virtual nodes (each physical shard owns hundreds of ring positions) so that load is statistically smoothed even with imperfect hash functions or heterogeneous shard capacities.

Range-partitioned systems take a different approach. Vitess (the YouTube-derived MySQL sharding layer used by Slack, GitHub, Square) implements resharding via cutover, built on its VReplication engine. Per the Vitess Reshard docs, the workflow has explicit phases: Create (start streaming data from source to target shards via VReplication, a Change Data Capture-style change stream), a copy/catch-up period monitored with show/status, a VDiff verification step that compares source and target data row-by-row, SwitchTraffic (move replica, read-only, and primary traffic to the new shards — and VtGate can buffer in-flight queries during the primary switch so application users see virtually no impact), and Complete (tear down the VReplication streams and old shards). It is online (no maintenance window) but operationally heavy — a large-scale resharding takes days to weeks of human attention — and reverse-replication streams are created by default so the cutover can be rolled back via ReverseTraffic. CockroachDB and Spanner (Corbett et al. OSDI 2012) take a more elegant approach with the splittable shard pattern: each “range” is a unit of data and replication, and the system automatically splits a range into two when it exceeds a size or load threshold, with the split being a metadata operation followed by background data movement. This means rebalancing happens continuously and incrementally rather than as a discrete migration project — the trade-off is that the storage layer must be designed from the ground up around split-by-range as a primitive, which legacy SQL engines cannot retrofit.

4. Request Flow

sequenceDiagram
    actor C as Client
    participant R as Router
    participant SM as Shard Map / Coord.
    participant S1L as Shard 1 Leader
    participant S2L as Shard 2 Leader

    Note over C,S2L: Single-shard read (the common case)
    C->>R: read(key=42)
    R->>SM: lookup(42)
    SM-->>R: shard_1
    R->>S1L: read(42)
    S1L-->>R: value
    R-->>C: value

    Note over C,S2L: Cross-shard query (slower; fan-out)
    C->>R: scan(predicate)
    R->>SM: lookup(all shards)
    SM-->>R: [shard_1, shard_2]
    par
      R->>S1L: scan(predicate)
      R->>S2L: scan(predicate)
    end
    S1L-->>R: partial results
    S2L-->>R: partial results
    R->>R: merge results
    R-->>C: combined results

What this diagram shows. Two request paths. Single-shard read: the client requests read(key=42); the router consults the shard map, learns key 42 lives on shard 1, and forwards to shard 1’s leader. The leader responds; the router forwards to the client. Total latency is one shard hop plus router overhead — fast, predictable. Cross-shard query: the client requests a scan predicate that does not specify a partition key. The router cannot localize this to a single shard, so it fans out the query to all shards in parallel, waits for partial results from each, merges them, and returns the combined result. Total latency is the slowest shard’s response time — much higher than single-shard, with high tail latency because any one slow shard delays the whole query.

The performance gap between the two is dramatic: single-shard reads can complete in 5-10 ms; cross-shard scans on a 100-shard cluster routinely take seconds. Sharded architectures push the data model to avoid cross-shard queries by denormalizing — storing the same data in multiple sharded ways, each optimized for one access pattern.

5. Variants

The variants are mostly partition strategies. Each has its own access patterns, hot-spot risks, and rebalancing properties.

5.1 Hash Partitioning (Consistent Hashing)

The default for most modern systems. The partition key’s hash is computed (MurmurHash3, MD5, etc.) and mapped to a shard via Consistent Hashing. Properties:

• Uniform distribution under random keys. Hash spreads any key set evenly across shards (subject to per-shard hash quality and virtual-node count).
• No range queries. Adjacent keys (user_42, user_43) hash to unrelated positions — they’re on different shards. Queries like “give me all users with ID 100-200” must fan out across all shards.
• Adding/removing shards moves ~K/N keys. Karger’s Theorem 2 from STOC 1997: adding the Nth shard relocates ~1/N of all keys.
• Used by: Cassandra, DynamoDB partition keys, Riak, Memcached client-side sharding, MongoDB hashed shard keys.

5.2 Range Partitioning

The partition key’s value determines the shard, with each shard owning a contiguous range of values. Properties:

• Range queries are fast. Adjacent keys live on the same (or adjacent) shards; “user IDs 100-200” lives on one or two shards.
• Hot spots are easy to create. Time-ordered keys (e.g., timestamp) all funnel to the most-recent shard, which becomes a write hot spot. Avoid sequential keys in range partitioning unless you intentionally bucket them (e.g., (user_id, timestamp) as the composite key, with hash-partitioning on user_id within range partitioning on timestamp).
• Rebalancing splits hot ranges. When a range gets too big or too hot, the system splits it into two sub-ranges, each on its own shard. Enables organic growth.
• Used by: HBase, Bigtable, Spanner, CockroachDB, MongoDB ranged shard keys.

5.3 Geo Partitioning

Shards correspond to geographic regions; partition key determines the user’s home region. Properties:

• Locality-aware reads and writes. A US user’s data lives on US shards; their requests route locally. Latency is low.
• Awkward for users with multi-region data. A user in San Francisco who travels to Tokyo wants their data accessible there; geo-partitioning forces either cross-region routing (slow) or replicating user data across regions (combining geo with replication strategies).
• Useful for compliance. GDPR / data-sovereignty requirements may require user data to remain in their region; geo-partitioning makes this enforceable at the architectural level.
• Used by: Spanner placements, CockroachDB locality, Discord per-region guild routing, Slack regional pods.

5.4 Directory-Based Partitioning

A lookup service maintains the per-key shard mapping. Each key’s shard is recorded explicitly (not computed from the key’s hash or range). Properties:

• Maximum flexibility. Keys can be moved between shards without their identifiers changing. Hot keys can be moved to dedicated shards. Tenant-aware placement.
• Lookup-service overhead. Every request must consult the directory, adding latency. Directory cache helps but cache invalidation is a familiar pain.
• Lookup-service is a single point of failure. Often replicated via Raft or Paxos High-Level for durability.
• Used by: Couchbase vBucket map; Vitess’s lookup Vindexes (a lookup table mapping key values to shards). Slack’s early “team → shard” assignment was a directory-style scheme (each workspace recorded against a shard, movable manually), but Slack has since moved to Vitess fine-grained sharding (see §6) — a reminder that directory schemes are often a stepping stone rather than a destination.

5.5 Sharded + Replicated Composition

The most common production pattern: shards each have their own Leader-Follower Replication Architecture (3-5 replicas per shard) or Leaderless Replication Architecture (Cassandra’s per-shard quorum). The shard provides scale-out; replication provides durability and read scaling within the shard. Cassandra’s RF=3 plus 256 vnodes per node implements this elegantly: tokens (vnode positions) define the shards, and each token’s primary plus next two clockwise neighbors form its replica set.

5.6 Co-Located Partitioning (Parent-Child Keys)

When related data must be on the same shard for join performance, the partition strategy uses a parent partition key. For example, in a multi-tenant database, every row’s partition key starts with the tenant ID; rows for the same tenant are co-located on the same shard. Cross-tenant joins are hard, but within-tenant joins (the common case) are fast. Vitess’s tablet-keyspace and Cassandra’s clustering keys both implement variations.

5.7 Bounded-Load Consistent Hashing (Mirrokni 2018)

A refinement of Consistent Hashing: cap each shard at (1+ε) × average_load. If a key would be placed on a shard already at the cap, walk forward to the next shard. Used by Vimeo for video routing and HAProxy for load balancing. Mitigates hot-key problems within the consistent-hashing framework.

The Mirrokni–Thorup–Zadimoghaddam 2018 paper proves that the resulting placement is both near-uniform in expectation (within an ε factor of perfect balance) and maintains the consistent-hashing locality property (most keys stay on the same shard when shards are added or removed). The trade-off is the lookup procedure: with strict consistent hashing, finding a key’s shard is O(log N) via a sorted ring; with bounded-load, the lookup may need to walk forward through several shards before finding one with capacity, making the worst case O(N) — though in practice, for ε around 0.1, the average walk length stays at 1-2 hops. The HAProxy implementation uses this for upstream-server selection: hashing client connections to backend servers with bounded overflow gives session affinity (consistent placement so a client returns to the same backend) plus avoids overloading any single backend. Production trade-off: bounded-load is most valuable when the underlying load distribution is moderately skewed but not pathologically so. For severe hot keys (a single key receiving 90% of traffic), bounded-load doesn’t help — the cap doesn’t apply at the key level, and any shard hosting that key still gets 90% of traffic.

5.8 Per-Shard Cells (Twitter Manhattan, AWS DynamoDB Global Tables)

A refinement that wraps each shard in a “cell” — a self-contained unit with its own routing, replication, and capacity allocation. Cells are operated as relatively independent units (separate deployments, separate failure domains), and the cluster is a fleet of cells. AWS Cell-Based Architecture describes this pattern: small enough cells to be operationally manageable, large enough to amortize fixed costs, with explicit isolation between cells. Manhattan’s cells (Twitter), DynamoDB’s partitions, and Aurora’s database shards are all variants. The architectural advantage over flat sharding is failure-domain isolation: a bad deployment can only break one cell, not the entire cluster, so blast radius is bounded. The cost is more sophisticated routing (the router must know not just which shard but which cell, and cell-level versions may differ).

5.9 Composite / Multi-Level Sharding

Some workloads benefit from sharding at two levels — for example, sharding by (tenant_id, user_id) where the outer level (tenant) determines a coarse cell and the inner level (user_id) determines a sub-shard within the cell. This is the natural extension of the parent-partition-key pattern when individual tenants grow large enough to need their own internal sharding. CockroachDB’s “geo-partitioned with internal hash sharding” feature (declared via PARTITION BY plus INTERLEAVE) is the relational form. Cassandra’s compound primary keys ((tenant_id, user_id, message_id) where the partition key is (tenant_id) and the clustering keys are (user_id, message_id)) is a closely-related form. The architectural property: related data is co-located within a cell while unrelated data is distributed across cells, satisfying both within-tenant transactional needs and cross-tenant load balancing.

6. Real-World Examples

Cassandra rings. Each cluster is a logical ring; each node owns a set of token ranges (vnodes). Partition keys are hashed to ring positions; each key lives on the node owning its position plus the next N-1 nodes (replicas). Rebalancing is semi-automatic; adding a node steals tokens from existing nodes proportionally.

MongoDB sharded clusters. A mongos router fronts the cluster; metadata is in config servers (a small replica set). Each shard is a replica set. Partition keys can be hashed (uniform distribution, no range queries) or ranged (range queries possible, hot-spot risk).

Vitess (YouTube-derived, used by Slack, GitHub, Square, Etsy). MySQL sharding layer. The vtgate proxy fronts the cluster; each shard is a separate MySQL replica set. Vindexes (Vitess’s name for partition strategies) support hash, range, lookup-table-based, and “consistent lookup” mappings. Resharding (splitting a shard into two) is online with bounded data movement.

Twitter’s Manhattan. Multi-tenant key-value store with sharding by tenant + key hash. Different tenants get different replica counts and consistency levels. Twitter’s 2014 blog post. Manhattan partitions the keyspace into “cells” — coarse-grained shards that own a hash range — and within each cell uses sub-shards to spread load further. The architecture explicitly handles celebrity hot keys via per-cell read replication (a hot user’s data lives in a cell with extra replicas) and via materialized hot-key caches fronting the cells. Manhattan also distinguishes between latency-sensitive and throughput-sensitive workloads at the cell level, allowing different cells to use different storage engines (for example, an SSD-backed cell for tweets vs an HDD-backed cell for archived DMs). The architectural pattern of “cells + within-cell sub-sharding + materialized hot-key caches” has informed many subsequent sharded designs at scale.

Slack’s sharding evolution (team-sharding → Vitess fine-grained sharding). This example is often mis-told, including in earlier versions of this note, so it is worth getting right. Slack originally sharded by workspace (“team”): each shard held thousands of workspaces and all their data — messages, channels — which gave clean failure isolation (one workspace’s problems stayed on its shard). But because workspaces have wildly different sizes, large customers’ shards grew until, per Slack’s Scaling Datastores at Slack with Vitess, “their designated shard reached the largest available hardware and we were regularly hitting the limits of what that single host could sustain,” producing un-spreadable database hotspots. Slack’s fix was to migrate to Vitess and reshard onto finer-grained keys — most notably resharding the messages table to shard by channel ID instead of workspace, so load spreads evenly and no single channel is likely to overwhelm a shard. In Vitess, applications connect to the VtGate routing tier, which knows each table’s sharding column; this let Slack change sharding schemes without rewriting application logic. The takeaway for interviews: workspace/tenant sharding is a starting point that buys isolation, but it has a built-in scaling ceiling at the largest tenant’s size, and the mature design moves to a finer key (channel, user) — the opposite of “give the big tenant a dedicated shard via a directory,” which is one mitigation but not what Slack ultimately did. Slack reports serving ~2.3M QPS at peak (≈2M reads, ≈300K writes) with median query latency ~2 ms and p99 ~11 ms on this architecture.

DynamoDB’s partition-key model. Each item has a partition_key (hashed to a physical partition) and an optional sort_key (clustering within the partition). Historically, throughput was divided evenly across partitions, so a skewed workload would throttle one partition with ProvisionedThroughputExceededException even when the table as a whole had spare capacity — the infamous “hot partition” pain. DynamoDB adaptive capacity (described in AWS’s database blog) reallocates capacity toward hot partitions; it is on by default for every table, and as of the 2019-05-24 update it is instant rather than the earlier 5–30-minute reaction time. DynamoDB also does “split for heat” — automatically splitting a sustained-hot partition into two, doubling the capacity available to those items. Two nuances matter for design discussions: (1) adaptive capacity does not abolish the per-partition hard ceiling — a single physical partition still caps at 3,000 read capacity units (RCU) and 1,000 write capacity units (WCU) per second, so a single hot key (which always lands on one partition) cannot exceed those limits no matter how much adaptive capacity is available; and (2) adaptive capacity is not a license to ignore data-model skew — it smooths moderate imbalance, not pathological single-key concentration.

Spanner / CockroachDB / TiDB. Each “tablet” or “range” is a shard with its own Raft group. Range-partitioned by primary key; rebalancing splits hot ranges. Cross-range transactions use Two-Phase Commit with the per-range Raft commits as the local commit primitive.

ProxySQL for MySQL. Routing-layer proxy that fronts multiple MySQL backends; supports hash-based and rule-based routing. Lightweight; commonly used for read scaling and lightweight sharding.

YouTube Manhattan / Knoll / Photon Storage. Internal Google storage systems for YouTube metadata; shard by video ID hash, partitioned across many backends.

Discord guild → server assignment (pre-Manhattan). A Kademlia-derived consistent-hashing scheme assigning ~150M Discord guilds to backend processes.

Redis Cluster. A built-in sharding mechanism for Redis: per the Redis Cluster specification, there are exactly 16384 hash slots assigned across master nodes, and each key’s slot is CRC16(key) mod 16384 — the CRC16 is specifically the XMODEM variant (polynomial 0x1021, init 0x0000), and only 14 of its 16 output bits are used (hence the modulo by 2¹⁴ = 16384). Multi-key operations require co-location, achieved via hash tags: if a key contains {...}, only the substring between the first { and the first } is hashed, so {user1000}.following and {user1000}.followers land on the same slot. Clients cache the slot map and route directly to the owning node; on a stale route the node replies MOVED <slot> <host:port> (permanent reassignment — update the map and retry) or ASK <slot> <host:port> (a slot mid-migration — send just this one request to the target, prefixed with ASKING, without updating the map). Per-slot replication is leader-follower. Lightweight compared to Vitess/MongoDB sharding; well-suited to caching workloads.

Memcached client-side sharding (libketama). Pure client-side: clients independently maintain a Consistent Hashing ring of memcached servers and route gets/sets by key hash. No server-side awareness; adding a server invalidates only ~1/N of cached keys, not the whole fleet. The de-facto standard since ~2007 for memcached deployments.

Apache HBase / Bigtable. Range-partitioned by row key; each region (Bigtable’s term: tablet) is a contiguous key range owned by one region server. The region master coordinates region assignment to region servers; ZooKeeper / Chubby handles failover. Hot ranges are split automatically when they exceed a size threshold.

TiDB / TiKV. Range-partitioned with per-range Raft groups. The PD (Placement Driver) is the cluster’s metadata service; it maintains the range-to-replica map and orchestrates rebalancing. Effectively a CockroachDB-style architecture optimized for MySQL compatibility.

Notion’s per-workspace sharding. Notion shards by workspace — each workspace’s data lives on a specific shard, and cross-workspace queries are rare. This is the tenant-sharding pattern Slack started with; Notion’s public documentation is thinner than Slack’s, so whether (and how) it handles giant-workspace hotspots is less clear. The pattern is common across multi-tenant SaaS as a first design, with the caveat (see Case C in §3 and Pitfall 10) that it carries a single-largest-tenant scaling ceiling.

MongoDB chunk migration. MongoDB’s sharded clusters internally divide each shard into “chunks” — contiguous ranges of the shard key. Per the MongoDB sharding docs, the default range (chunk) size is 128 MB (not the older 64 MB figure, which was the default in MongoDB versions before 6.0 — getting this wrong in an interview dates your knowledge). Current MongoDB also reframes the model from “chunk” toward “range,” and crucially changed what the balancer balances: rather than equalizing chunk counts per shard, the balancer (since 6.0) balances by data size, migrating ranges only when the data-size difference between the most- and least-loaded shard for a collection exceeds a threshold of three times the configured range size (i.e. 384 MB at the 128 MB default) — this avoids churning on tiny imbalances. The migration itself is online: writes to the range continue during the move, with a brief final cutover where the source rejects writes while the destination takes ownership. Range/chunk migration is the operational bedrock of MongoDB sharding — it makes initial uneven distribution self-healing and lets shards be added or removed without manual rebalancing scripts. The cost is that the balancer itself becomes a workload component during heavy migrations, contending with foreground traffic for IO; production deployments often configure balancer windows (e.g., “only run 02:00–06:00 UTC”) to keep migrations off the critical path.

7. Tradeoffs

Scale vs operational complexity. Sharded systems scale beyond any single node’s limits. They also require shard management, partition-key design, rebalance machinery, and cross-shard query awareness — orders of magnitude more operational complexity than single-node databases.

Single-shard latency vs cross-shard latency. Single-shard requests are as fast as the underlying database. Cross-shard requests are bottlenecked by the slowest shard’s response time and pay router fan-out overhead. The latency gap is often 10× to 100×.

Partition-key flexibility vs partition-key reusability. A clever partition key (e.g., (tenant_id, sub_key)) optimizes for the most common access pattern but locks in that pattern. Adding a new common access pattern on a different attribute (e.g., “find all items where status=X across all tenants”) becomes expensive — must fan out across all shards.

Consistent hashing’s uniformity vs range queries. Hash partitioning gives uniform distribution but precludes range queries. Range partitioning enables range queries but introduces hot-spot risk for sequential keys.

Resharding vs initial design. A correctly-chosen partition key pays off forever; a wrong one becomes increasingly painful as data grows. Migrations to a new partition key are major projects (offline, dual-write, gradual cutover).

Strong-consistency-within-shard vs weak-consistency-across-shards. Sharded systems can offer strong consistency within a shard (per-shard linearizability via leader-follower) for free, but cross-shard operations either require expensive coordination (2PC, Spanner-style HLC) or accept weaker semantics (eventual consistency, read-your-writes). This is in tension with how applications often want to think about their data: as one logically-coherent dataset where any operation can see any state. The “the database is sharded” detail leaks into the application’s mental model whenever true cross-shard consistency is needed. Even systems that paper over this with cross-shard transaction support (Spanner, CockroachDB) pay measurable latency cost that is visible in p99 monitoring.

Coordination layer cost. The shard map / directory is on every request’s critical path. It must be fast (in-memory, replicated, possibly client-cached) and correct (rebalances must be coordinated). The coordination layer is often the most-replicated component in the system.

The coordination layer is also a recurring source of operational incidents. Common failure modes: (1) the metadata store goes down (etcd / ZooKeeper outage), and the routing layer either freezes or operates on stale data; (2) a partial network partition between routers and metadata store leads to split-brain routing decisions; (3) cache invalidation lag during shard migrations causes router-side stale data; (4) the metadata store’s write throughput becomes the cluster’s bottleneck for shard-map updates during heavy rebalancing. Production designs mitigate these by replicating the metadata store via Raft (etcd, ZooKeeper) for availability, embedding multiple routers per region with local caches, and treating shard-map updates as low-frequency, high-priority operations distinct from data-plane writes.

8. Migration Path

From single-instance to sharded. Identify the partition key (this is the project’s hardest decision); split the data; build a routing layer; cut traffic over gradually. Some teams do this with a “dual-write phase” where writes go to both the old single-instance and the new sharded system, with reads cutover incrementally. The full migration of a TB-scale database typically takes months and is a defining project for the engineering team.

From sharded to differently-sharded (resharding). When the original partition key is wrong (hot shards, cross-shard queries dominating), migrate to a new key. Strategies: (a) parallel build of the new layout with dual-writes; (b) incremental tablet splits in range-partitioned systems (Vitess does this); (c) full offline migration. None are fast.

From shard-N to shard-(N+1) (adding capacity). With Consistent Hashing, adding a shard moves ~K/N keys. With range partitioning, adding a shard requires splitting hot ranges. Modern systems do this online; older ad-hoc sharding may require maintenance windows.

From sharded leader-follower to sharded leaderless. Both per-shard. The composition of Sharded Architecture with the per-shard replication scheme is independent — you can change the per-shard scheme without changing the partitioning. Cassandra is sharded leaderless; CockroachDB is sharded leader-follower per range.

9. Pitfalls

Pitfall 1: The hot-key problem. A single partition-key value gets a disproportionate share of traffic. The classic example is celebrity fanout in social networks (Twitter Newsfeed System Design’s celebrity problem) — a celebrity’s tweets are distributed to millions of followers, and the celebrity’s user record sees vastly more traffic than typical users. Sharding doesn’t help: the celebrity’s record lives on one shard, and that shard sees disproportionate load. Mitigations are application-level: client-side caching, sub-key splitting (celebrity:42:slot:0, celebrity:42:slot:1, etc.), pre-computed materialized views, hot-key replication-factor increase.

Worked example — @TaylorSwift’s hot-shard problem. Concrete numbers make the pain visible. Suppose @TaylorSwift has 100 million followers. Under the naïve “shard the user table by user_id” model, @TaylorSwift’s row lives on one shard. Each time she posts a tweet, the fanout job reads her user record (to enumerate followers, fetch the tweet text, build a fanout payload), and pushes the tweet into 100M follower timelines. If posting frequency is 10 tweets/day during a press cycle, that’s 10 × 100M = 1 billion follower-timeline writes per day attributable to one row, plus the read amplification on her source row. The shard hosting her row sees orders of magnitude more traffic than shards hosting median users with 100 followers. Twitter’s documented response (see Manhattan and the historical blog posts on fanout architecture) combines several techniques: (1) Read-through cache for the celebrity’s record so the underlying shard isn’t queried per-fanout-task; the celebrity’s record is fronted by a global Memcached / Redis tier with high hit rates. (2) “Celebrity treatment” — abandon push-fanout for the largest accounts, instead doing pull-on-read. When a regular user opens their timeline, the timeline service merges the user’s pre-computed home timeline (push-fanout from their non-celebrity friends) with a synchronous fetch of recent tweets from each celebrity the user follows (merge-on-read). The cost is shifted from celebrity-write-time to follower-read-time, but read-time merging is bounded by the small number of celebrities a user follows whereas write-time fanout scales with follower count. (3) Per-shard replicas with read load balancing — the celebrity’s shard gets extra replicas, and read traffic is round-robined; this trades storage for read throughput. (4) Pre-computed materialized views — celebrity tweets are written once to a global “celebrity tweet log” indexed by user, sidestepping per-shard read amplification. The full architectural response in Twitter’s case became substantial enough to be a recurring topic in their engineering talks (see the Manhattan paper and various QCon presentations); the takeaway for design interviews is that sharding solves load distribution across keys, not load on any individual key, and the hot-key problem requires application-layer solutions on top of whatever sharding scheme is chosen.

Pitfall 2: Hot-shard from sequential keys. Range partitioning by a monotonically-increasing key (timestamp, autoincrement) sends all writes to the most-recent shard. The top shard becomes the bottleneck while older shards idle. Mitigations: hash-partition on a different prefix; reverse-key indexing (write the key in reverse); randomization ((random_bucket, timestamp) composite key).

Pitfall 3: Partition-key choice cannot be easily changed. The most consequential decision in sharded design is also the hardest to undo. Many teams discover months later that their partition key was wrong (e.g., sharded by user when most queries are by-product). Migration is expensive. Spend disproportionate time on this decision.

Pitfall 4: Cross-shard transactions are slow and complex. A transaction touching N shards must coordinate via Two-Phase Commit or a higher-level protocol; latency increases with N; partial failures must be handled. Design data models to keep transactions within a shard wherever possible.

The depth of this pain merits a worked treatment. Suppose a banking application is sharded by account_id. Every account-local operation (balance check, deposit, single-account withdrawal) is a single-shard transaction and is fast — the per-shard storage engine commits in milliseconds. But the canonical banking operation, transferring between two accounts, hits two shards if the accounts have different account_id hashes. The classical solution is Two-Phase Commit: a transaction coordinator sends PREPARE to both shards (each shard locks the relevant rows, writes a prepare record to its log, and replies OK), then sends COMMIT to both (each shard finalizes the change and releases locks). The protocol guarantees atomicity but has two structural weaknesses. First, latency: 2PC requires four network round-trips in the happy path (prepare-out, prepare-ack, commit-out, commit-ack), and each round-trip crosses the network. A single-shard transaction commits in 5-10 ms; a 2PC transaction across two shards costs 30-50 ms minimum, often more if the shards are in different regions. Second, blocking on coordinator failure: if the coordinator crashes after sending PREPARE but before sending COMMIT, the shards are left holding prepared-but-uncommitted state with locked rows — they cannot proceed without the coordinator’s recovery. The Spanner paper (Corbett et al. OSDI 2012) addresses both issues with TrueTime + HLC-style hybrid logical clocks: each transaction has a globally-ordered commit timestamp from TrueTime, ranges agree on commit order via Paxos within each range, and 2PC across ranges uses one of the participating Paxos groups as the coordinator (so coordinator failure is just Paxos failover). CockroachDB uses a similar HLC-based 2PC. Both systems offer cross-shard ACID at substantial latency cost (cross-region transactions cost 100+ ms because TrueTime/HLC must wait out clock uncertainty windows). The typical advice in design interviews is therefore: don’t. Whenever possible, design the data model so that transactional consistency is required only within a shard. For the banking example, this means putting both source and destination accounts on the same shard via a parent partition key (e.g., shard by customer_id rather than account_id, on the assumption that most transfers are intra-customer). When that’s impossible — true cross-customer transfers — the application accepts the latency cost or designs around it with eventual consistency (write a transfer-intent record, settle async, reverse on conflict). The general pattern across well-engineered sharded systems (Slack, Notion, Stripe pre-Spanner) is to engineer transaction boundaries to coincide with shard boundaries and treat true cross-shard transactions as exceptional.

Pitfall 5: Rebalancing during traffic spikes can cascade. Adding a shard (or splitting a hot range) requires moving data — which consumes IO bandwidth on both source and destination shards. If done during a traffic spike, the rebalance contends with the regular workload, slowing both. Operational guidance: rebalance during off-peak windows; throttle aggressively.

Pitfall 6: Aggregate queries across all shards are fundamentally slow. “Total user count,” “all tweets in the last hour,” “global statistics” — these queries must visit every shard. With 100 shards, the query takes the slowest shard’s response time. Mitigations: pre-aggregate (separate analytics store), use approximate algorithms (HyperLogLog cardinality estimates), accept slower queries.

Pitfall 7: Routing-layer staleness. The shard map changes when shards are added, removed, or split. Clients with stale shard maps route to the wrong shard, which must redirect or refuse. The redirect protocol (e.g., MongoDB’s cluster reroute response, Redis Cluster’s MOVED response) is essential but adds complexity.

The redirect mechanism is worth examining in detail because it’s a common source of subtle bugs. Redis Cluster’s protocol: when a client sends a command for key K to node N, but N no longer owns K’s hash slot (because the slot has been migrated), N replies with MOVED <slot> <new_node>:<port>. The client is expected to update its slot map and retry against the new node. If the slot is currently being migrated (some keys moved, some not), N may instead reply with ASK <slot> <new_node>:<port>, indicating the client should send this specific request to the new node but not update its slot map. The distinction between MOVED (permanent reassignment) and ASK (transient migration) avoids stale-map oscillation during migrations. Stale-map bugs typically manifest as latency spikes during rebalance: the client repeatedly hits the wrong node, gets MOVED, retries, and the retry chain compounds across many slots being migrated simultaneously. Production deployments mitigate by client-side caching of the slot map (with TTL invalidation), aggressive map refreshes on MOVED-storm detection, and operations-team coordination of migrations to off-peak windows.

Pitfall 8: Shard fan-out increases tail latency. Even when each shard is fast, a query touching all 100 shards completes only when the slowest of 100 responses arrives. The “tail-at-scale” problem (Dean & Barroso 2013). Mitigations: hedged requests (re-issue to a replica after percentile timeout), tied requests (race two shards’ replicas).

Pitfall 9: Operational tooling is per-shard. Backup, restore, schema migrations, index builds — each must be performed on every shard. Errors happen; some shards get the operation, others do not. Tools for orchestrating shard-wide operations (Vitess’s online schema change, Cassandra’s nodetool repair --pr) are essential and themselves complex.

Pitfall 10: Multi-tenancy interacts poorly with sharding. A small tenant on a shard with a large tenant suffers when the large tenant’s traffic crowds out the small. “Noisy neighbor” problem. Mitigations: per-tenant rate limits, dedicated shards for largest tenants, tenant-isolation strategies.

The big-tenant problem deserves more concrete framing because it’s a recurring SaaS architecture failure. A typical SaaS customer-distribution follows a power law: 80-90% of revenue comes from 5-10% of tenants. Sharding by tenant_id will, by sheer luck, place several large tenants on the same shard occasionally. That shard then sees disproportionate load — it’s the largest customer’s shard plus a handful of medium customers. Symptoms: this shard’s p99 latency degrades while the rest of the cluster is fine; on-call investigates, finds nothing wrong with the shard’s hardware, and eventually realizes load skew. There are two broad mitigation families, and real systems pick based on whether their data model permits sub-tenant sharding. Family one — tier the shard population: the largest 1% of tenants get dedicated shards (one tenant per shard, hardware sized for it); the next ~10% get small-fleet shards (a few tenants each, with rate-limit isolation); the remaining ~89% share bulk shards (hundreds of small tenants). This works when tenants are opaque units that cannot be split. Family two — reshard onto a sub-tenant key: if the data model has a natural finer key (channel, user, object), abandon tenant-level sharding entirely and shard on that key, spreading even a giant tenant’s load across the whole fleet. Slack chose family two, resharding messages by channel ID via Vitess (see §6) precisely because dedicated big-tenant shards still bottlenecked on single-host limits. Notion’s per-workspace model is closer to family one, though publicly documented detail is thinner. The cost of either is more sophisticated routing and capacity planning, plus explicit operator awareness of the “large enough to need special treatment” threshold (storage bytes, request rate, or user count). The general lesson: dedicated shards postpone the ceiling; sub-tenant resharding removes it, but only if the data model exposes a usable finer key.

Pitfall 11: Index maintenance per-shard amplifies write cost. Each index on a sharded table is itself sharded — a write to a row updates the row’s shard and the corresponding entries in any local indexes on that shard. Cross-shard secondary indexes are even more expensive (require dual-write or fan-out queries). Schema design must be conscious of which indexes exist; the “free index” assumption from single-node databases breaks at scale.

Pitfall 12: Shard-local transactions can hide cross-shard requirements. Application code may assume BEGIN; UPDATE foo; UPDATE bar; COMMIT; is atomic — and it is, if foo and bar are on the same shard. If they’re on different shards, the transaction silently splits into two unrelated transactions, and a partial failure leaves inconsistent state. Schema design must ensure related rows that need transactional consistency live on the same shard (the parent-partition-key / co-located-data pattern).

Pitfall 13: Failover within a shard versus failover of a shard. A shard’s leader-follower group can fail over normally — that’s an internal-to-shard concern. But losing an entire shard (say all replicas in one region) requires either the cluster to operate without that shard’s data or a parallel shard recovery from off-site backup. Sharded architectures typically have higher aggregate availability for read but lower per-shard durability than non-sharded systems unless explicitly designed with cross-region per-shard replication.

Pitfall 14: Sharding amplifies “long-running query” pain. A query that runs slowly on one node runs slowly on each shard, and the overall result is bounded by the slowest. Long-running cross-shard queries — particularly those without query timeouts — can pin resources across the entire cluster. Robust shard implementations include per-query timeouts that fire on every shard.

Pitfall 15: Read-replica routing for hot shards must be explicit. A read-mostly hot shard can be partially mitigated by routing read traffic to additional follower replicas of that shard. But this requires the routing layer to be hot-shard-aware: ordinary round-robin replica selection underutilizes the extra replicas because requests for other shards bypass them. Production systems handle this by giving hot shards extra replicas and configuring the router to weight reads toward those replicas — Vitess’s tablet types (PRIMARY, REPLICA, RDONLY) and Cassandra’s LOCAL_ONE consistency level with rack-aware routing both expose hooks for this. Without explicit configuration, the extra replicas are wasted capacity.

Pitfall 16: Request-level sharding is sometimes the right hot-key escape hatch. When a single key is genuinely hot and other mitigations are insufficient, some systems shift from key-level to request-level sharding — for example, hashing (key, request_id) to spread writes for a hot key across many slots and then merging on read. This works for counter-style writes (incrementing a counter), is recoverable for accumulating values, and is awkward for arbitrary updates (the read side must merge slots correctly). Discord’s documented use of this for its message-counter and presence-tracker workloads (see their engineering blog on Cassandra usage) illustrates the pattern: the celebrity-message-thread receives incoming messages distributed across thousands of slots; reads merge slots within a time bucket; the hot-shard problem is converted into a “many warm shards” problem at the cost of read complexity.

10. Comparison with Sibling Architectures

	Sharded	Peer-to-Peer Architecture	Leader-Follower Replication Architecture
Placement	Operator-specified	Discovered (DHT routing)	Shared (all replicas have all data)
Scaling	Horizontal (add shards)	With participation	Bounded by leader
Cross-partition operations	Expensive (2PC)	Trivially fan-out	N/A
Operator control	High	Low	High
Latency predictability	High (single-shard); low (cross-shard)	Low	High
Operational complexity	High	Externalized	Medium
Data placement authority	Shard map service	Emergent via DHT	Single
Canonical example	Cassandra, MongoDB sharded, Vitess	BitTorrent, IPFS	PostgreSQL, MongoDB single replica set

The Peer-to-Peer Architecture / sharded contrast is foundational: in P2P, peers discover placement; in sharded, an operator specifies placement. Sharded systems know where their data is (via the shard map); P2P systems route to find their data (via DHT lookups). Sharded systems have higher per-request determinism; P2P has higher resilience to operator-takedown. Each is the right answer to a different problem.

The composition with Leader-Follower Replication Architecture is the modern default: shards for scale, leader-follower per shard for durability. Cassandra is the leaderless variant; CockroachDB is the Raft-based variant; classical RDBMS sharding (Vitess, ProxySQL) is the leader-follower variant.

11. Common Interview Discussion Points

• “What is sharding?” Horizontal partitioning of data and load across N nodes by a partition key; each shard owns a subset of the data.

• “How do you choose a partition key?” Look at the read/write patterns; pick the attribute that distributes load uniformly and that most queries can specify. Common answers: user ID for per-user systems; tenant ID for SaaS; geo for geo-distributed; hash of a natural identifier for purely uniform load.

• “What’s the difference between hash and range partitioning?” Hash gives uniform distribution but no range queries; range gives range queries but introduces hot-spot risk for sequential keys.

• “What is the hot-key problem and how do you solve it?” A single partition-key value gets disproportionate traffic. Sharding doesn’t help. Mitigations: client-side caching, sub-key splitting, pre-computed views, hot-key replication.

• “How do you reshard?” Online with Consistent Hashing (only K/N keys move); split-based for range partitioning; offline migration for fundamental partition-key changes.

• “How do cross-shard queries work?” Fan-out: router sends to all (or a subset of) shards in parallel, merges results. Latency is the slowest shard’s response. Try to avoid by data-model choice.

• “How do cross-shard transactions work?” Two-Phase Commit or higher-level distributed transactions. Latency is multiple cross-shard round trips. Spanner / CockroachDB design for this.

• “What’s the celebrity problem and how does Twitter handle it?” Celebrity users have far more followers than typical users, so their fanout dominates. Twitter pre-computes home timelines per user via async fanout — but for the largest celebrities, this is too expensive, and they fall back to merge-on-read at follower request time. See Twitter Newsfeed System Design.

• “Why doesn’t Consistent Hashing solve hot keys?” It distributes load across many keys. A single hot key still goes to its one shard.

• “How does Slack shard its data?” The accurate answer is a story, not a snapshot: Slack originally sharded by workspace (team) for failure isolation, but big customers’ shards hit single-host limits and created un-spreadable hotspots, so Slack migrated to Vitess and resharded onto finer-grained keys (e.g., messages by channel ID), with the VtGate routing tier hiding the sharding column from the application. Citing “Slack shards by team” as the present tense is the common mistake.

• “Compare Cassandra’s sharding to MongoDB’s.” Cassandra: token-ring sharding via Consistent Hashing with vnodes; per-shard leaderless quorum replication. MongoDB: hash or range sharding; per-shard leader-follower replication. Cassandra is more uniformly horizontal; MongoDB is more familiar to RDBMS operators.

• “How would you migrate from a single Postgres instance to a sharded Postgres cluster?” (1) Identify the partition key based on access patterns. (2) Build the sharded layer (Citus, Vitess-for-Postgres, or app-side sharding). (3) Dual-write phase: writes go to both old and new systems. (4) Backfill historical data into the new sharded layer. (5) Read cutover: shift reads to the sharded layer. (6) Stop writes to the old system. (7) Decommission. The whole project is months long; the dual-write phase is the longest because reconciling discrepancies between the two systems requires careful logging and verification.

• “What’s the difference between sharding and replication?” Sharding splits the dataset across nodes (each shard has different data); replication copies the same data across nodes (each replica has the same data). They’re orthogonal — most production systems do both: sharded across many shards, with each shard internally replicated for durability.

• “What is the ‘fan-out write’ problem?” When a write must be propagated to many places (multiple shards, multiple indexes, multiple caches, multiple analytics destinations), each propagation point can fail or lag independently. The classic example is social-media fanout (a tweet goes to many follower timelines). Solutions: async fanout queues, idempotent writes, eventual reconciliation. See Twitter Newsfeed System Design.

• “How do you handle a shard becoming a hot spot?” In order: detect via per-shard metrics; cache hot data at the application layer; add follower replicas to the shard; split hot keys into sub-keys; give the hot tenant a dedicated shard; if pervasive, re-architect with a different partition key. Most production systems live at the first three layers.

11.1. Time-Travel Queries and Cross-Shard Reads

A common requirement in modern sharded systems is “give me a consistent read across all shards as of timestamp T.” This is the basis for snapshot reads, backup-consistency, and cross-shard analytics. In a single-node database, snapshot reads are a trivial MVCC primitive; in a sharded system, they require coordination.

The Spanner solution (Corbett et al. OSDI 2012) uses TrueTime — a globally-synchronized clock with bounded uncertainty. Every transaction commits at a TrueTime timestamp; reads at timestamp T see all transactions with commit timestamps ≤ T. The cross-shard guarantee comes from each shard agreeing on TrueTime independently — the global ordering is “free” because the clocks themselves are coordinated. The cost: TrueTime is hardware (atomic clocks, GPS receivers) plus a service layer; replicating it across regions requires Google’s infrastructure.

CockroachDB approximates this with Hybrid Logical Clocks (HLC) — a logical clock that tracks both physical time and a logical counter, with explicit handling for clock skew. Cross-shard reads at timestamp T require coordinating with each shard’s local HLC; the bounded skew window introduces 100+ ms latency for global reads. Spanner without TrueTime is what CockroachDB is, with the latency penalty TrueTime avoids.

The practical advice: cross-shard time-travel queries are expensive; reserve them for backup, analytics, and audit operations. For foreground reads, accept “consistent within a shard, eventually consistent across shards” semantics. The systems that build true cross-shard time-travel pay a real latency cost for it.

11.2. The Partition-Key Decision in Practice

Because partition-key choice cannot be easily changed and is the single most consequential decision in a sharded system, it deserves a dedicated decision framework. The questions to ask, in order:

1. What is the dominant access pattern? List the queries by frequency-weight. The most common query’s filter predicate is a strong candidate for the partition key — putting that attribute as the partition key means the most common query is single-shard. For a B2C consumer app, this is almost always user_id; for SaaS B2B, it’s almost always tenant_id; for an event-tracking system, it’s the event source (device_id, source_id), not the timestamp.

2. Does the partition key distribute traffic uniformly? Compute or estimate the load distribution across partition-key values. A user-id-keyed system has roughly uniform distribution because most users have similar activity levels; a tenant-id-keyed system has highly skewed distribution because tenant sizes follow a power law. If the distribution is severely skewed, plan for the directory pattern from day one (don’t try to fix this later).

3. Are within-key transactions the unit of consistency? If the application’s transactional boundaries align with the partition key (within-user transactions, within-tenant transactions), single-shard transactions cover the cases. If transactions cross the partition key (transfer between two users), expect to use 2PC or accept eventual consistency.

4. How will the system handle hot keys / hot tenants? Plan the mitigation layer (caching, replicas, key splitting, dedicated shards) before they’re needed. Production systems that didn’t plan for this and discovered hot keys at scale usually had outages while building the mitigation.

5. What’s the rebalancing strategy? Consistent hashing minimizes data movement; range partitioning enables splitting; directory partitioning enables explicit relocation. Pick the strategy that matches the system’s growth pattern.

6. What does cross-shard query performance look like? Some cross-shard queries will exist; estimate their frequency and tolerable latency. If they’re frequent, the partition key may be wrong; if they’re rare, accept the fanout cost. Don’t pretend they won’t happen.

7. What’s the migration path? If the partition key turns out to be wrong, what’s the cost to migrate? Vitess’s online resharding is months of effort; Cassandra requires rebuilding the cluster; CockroachDB’s automatic resharding is the gentlest. Choose the system whose migration story matches your appetite for operational pain.

These questions, asked early, prevent the most painful sharded-system mistakes. Asked late (after data is loaded and traffic is live), they often have no good answers — the only options are expensive migrations or accepting permanent operational pain.

11.3. Production Sharding Patterns Compared

A side-by-side examination of how three well-known production systems handle sharding makes the architectural variations concrete.

Cassandra (Lakshman + Malik 2010). Each row’s partition key is hashed (Murmur3 by default) to a 64-bit token; the cluster’s nodes each own ranges of token space (with virtual-node refinement, typically 256 vnodes per physical node). Replication is leaderless quorum within a replication factor (RF=3 typical). Adding a node steals tokens from the existing nodes proportional to the new node’s vnode count, redistributing data. The design’s strength is uniform horizontal scaling — adding nodes is operationally simple. The weakness is the lack of strong consistency primitives across rows: cross-partition transactions are fundamentally not supported, and even within-partition lightweight transactions (CAS via Paxos) have substantial latency cost. Cassandra’s sweet spot: high-write-throughput workloads that don’t need cross-partition transactions (time-series data, event logs, IoT telemetry, large-scale messaging).

Vitess (YouTube origin, deployed at Slack, GitHub, Square, Etsy). Each “vindex” is a partition strategy applied to a table; tables can have multiple vindexes for different access patterns. Per-shard storage is MySQL replica sets (leader-follower). Resharding is online via the workflow controller plus VReplication for data movement. The design’s strength is operational compatibility with existing MySQL — applications continue using MySQL clients, transactions within a keyspace work as expected, and the schema model is fully relational. The weakness is the lack of native cross-shard transactions (Vitess has 2PC support but it’s not the recommended pattern), and the cluster’s complexity (vtgate proxies, vttablets, topology service, VReplication workflows) is substantial. Vitess’s sweet spot: organizations with deep MySQL operational expertise who need to scale beyond a single replica set without rewriting applications.

Spanner / CockroachDB. Each “range” is a contiguous key range (typically 64-512 MB) with its own Raft consensus group. The cluster’s metadata maps key ranges to the Raft groups owning them; ranges split automatically when they exceed size or load thresholds. Cross-range transactions use 2PC with TrueTime / HLC for global ordering. The design’s strength is genuine cross-shard ACID with relational SQL; the weakness is the operational and performance cost of TrueTime / HLC (cross-region transactions are 100+ ms because of clock-uncertainty waits). Spanner’s sweet spot: workloads that genuinely need cross-shard transactional consistency at global scale (Google’s Ads system, financial systems requiring strict consistency). For workloads that don’t need this, the cost is unjustified — Cassandra or sharded MySQL would be cheaper and simpler.

The pattern: each system optimizes for a specific point in the consistency/availability/operability trade-off space, and the architectural details follow from that choice. Choose the system whose trade-off matches your workload, not the system with the best buzzwords.

11.4. Cross-Shard Joins and the Denormalization Pattern

Cross-shard joins are pervasive enough to warrant their own sub-treatment. A canonical example: in a sharded e-commerce system, Orders are sharded by customer_id and Products are sharded by product_id. A query like “show this customer’s orders with the product names” must join across the two shards. The naïve approach — fetch the orders, then for each order’s product fetch the product from its shard — is N round-trips per customer.

Three common solutions, each with trade-offs:

Solution 1 — Denormalize. Embed the product name in the order row at write time. The “show orders with product names” query is now a single-shard read of the orders. Cost: product-name updates require updating every historical order’s denormalized copy (effectively impossible if there are millions of orders). Acceptable if product names are stable; this is the dominant pattern in NoSQL systems where joins are expensive.

Solution 2 — Co-locate via the parent partition key. Shard both Orders and Products by customer_id, with each customer owning their own product catalog. Effective only if products are per-customer (B2B SaaS where each customer has their own products); useless if products are global.

Solution 3 — Application-side join. Fetch orders from one shard, build the list of needed product IDs, fetch products from their shards in parallel, join in application code. Adds latency but keeps the data model normalized. The approach Vitess and many MySQL-sharding systems take; sometimes optimized with caches or materialized views to avoid hitting the product shards on every request.

The general advice: in sharded systems, “joins are expensive” is more than a tuning note — joins across shards are a fundamentally different operation than joins within a single database. Schema design must explicitly account for this. Denormalization is not a hack but a first-class design pattern in sharded systems.

11.5. Worked Example: Designing the Shard Strategy for a SaaS Project Management Tool

To make the partition-key decision concrete, walk through a worked example: a project-management SaaS (think Asana, Linear, Jira-style) with multi-tenant teams managing projects, tasks, and comments. The data model has Tenant → Project → Task → Comment. Most queries are scoped to a tenant (“show all my projects,” “all tasks in this project,” “comments on this task”). Rare queries are cross-tenant (“how many tasks completed this quarter across all tenants” — internal analytics). Let’s evaluate three candidate partition strategies.

Candidate 1 — Shard by task_id (hash). Each task lands on a shard determined by hash(task_id). Read-task-by-ID is one shard. But: “show all tasks in project X” requires fanning out across all shards because adjacent tasks in the same project hash to unrelated shards. Even worse, cross-task transactions (“move 50 tasks from Project A to Project B as a batch”) become cross-shard 2PC. This is the wrong key — it optimizes a relatively rare access pattern (single-task-by-ID) at the cost of every common access pattern.

Candidate 2 — Shard by tenant_id (hash). All of a tenant’s data lives on one shard. Per-tenant queries are fast (single shard). Tenant-bounded transactions are atomic (single shard). Cross-tenant queries (analytics) require fanout, but they’re rare and tolerated being slow. Hot-tenant risk: a customer with 100K users on a shard hosting 10 customers with 1K users each will skew that shard’s load 10× over its peers. Mitigation: the directory pattern (Slack-style) — explicit tenant → shard mapping, with the operations team able to move large tenants to dedicated shards. This is the right answer for the common case.

Candidate 3 — Composite shard by (tenant_id, project_id) with tenant_id as the partition key. Same as Candidate 2 at the shard level (per-tenant locality), but with sub-locality within a tenant: project data clusters together. Cassandra-style clustering. Useful when individual tenants are large enough that within-tenant queries benefit from data layout (project-locality means scanning a project’s tasks reads contiguous storage). Marginal benefit over Candidate 2 for most workloads; clearly worth it for workloads where tenants exceed ~1 GB and project-locality reads are common.

The decision: Candidate 2 with the directory layer for large-tenant relocation, plus an analytics replica (Aurora read replica or Snowflake export) for the rare cross-tenant queries. The shard strategy is now bounded by the tenant size growth, with the escape valve of moving large tenants to dedicated shards. This pattern recurs in many SaaS products and is documented (with variations) by Slack, Notion, Linear, and others.

11.6. The Hot-Shard Problem in Detail

The hot-shard problem is the single most common operational pain in sharded systems, and the mitigations form a layered defense rather than a single fix. A clear breakdown of the layers:

Layer 1 — Detection. Per-shard metrics (request rate, latency, queue depth, IO utilization) are essential. A hot shard typically shows up as p99 latency degradation on that shard while neighboring shards are healthy. Modern observability stacks (Prometheus + Grafana, Datadog, Honeycomb) make per-shard dashboards a standard pattern. Without per-shard metrics, hot-shard problems manifest as “the database is slow sometimes” with no clear root cause.

Layer 2 — Caching at the application layer. A hot shard usually corresponds to a hot key, and the simplest mitigation is to cache that key’s reads at the application or CDN layer. Memcached, Redis, or CDN caches all work. The cache absorbs read traffic that would otherwise reach the shard. The cost is cache-coherency complexity (writes must invalidate the cache; cache misses still hit the shard; some workloads are write-heavy and benefit less). Most B2C systems have aggressive caching layers fronting their sharded databases for exactly this reason.

Layer 3 — Read replicas for the hot shard. Adding more follower replicas to the hot shard increases the read throughput available for that shard’s data. Requires the routing layer to be hot-shard-aware (route reads to the replicas, not just the leader). Effective for read-heavy hot shards; doesn’t help write-heavy hot shards.

Layer 4 — Key-level splitting. Replace the single hot key with multiple sub-keys (hot_key:slot_0, hot_key:slot_1, etc.) and shard the sub-keys independently. Effective for counter-style or accumulator workloads where merging slots on read is feasible. Awkward for arbitrary key-value workloads where the read path expects one key.

Layer 5 — Dedicated shard for the hot tenant/key. If one specific tenant or key is genuinely so hot that nothing else matches it, give it a dedicated shard with hardware sized for the load. Slack does this for IBM-scale customers; AWS DynamoDB’s adaptive capacity does this implicitly via partition splitting.

Layer 6 — Re-architect. If hot-shard problems are pervasive (not isolated incidents), the partition strategy itself is wrong. Reshard to a different key, or change the data model so the hot key isn’t a single key (e.g., split a global counter into per-region counters with periodic aggregation).

The pattern is to apply layers in order: detect, cache, replicate, split, dedicate, re-architect. Most production systems live at layers 1-3; only mature systems with extensive hot-shard experience reach layers 4-6.

11.7. Operational Lessons from Production Sharding Disasters

A few documented (or widely-known but not always cited) incidents illustrate how sharded architectures fail in practice:

Foursquare’s MongoDB outage (2010). A single shard’s data exceeded available memory; queries on that shard fell off the indexed-page cache; latency spiked across the cluster because routers were waiting on the slow shard. Compounding factor: the migration to add capacity required schema changes that were not online in MongoDB at the time. The outage lasted approximately 11 hours. The lesson: per-shard resource sizing must be planned for the largest shard, not the average; growth must trigger preemptive resharding before any shard exceeds its capacity.

The “GitHub October 2018 incident” (different from the replication outage; cluster-level). A network partition between regions caused split-brain in some shards’ replication groups, leading to inconsistent state. Recovery required manual reconciliation across shards. The lesson: per-shard replication groups must have explicit fencing protocols for region partitions; assuming connectivity within the cluster is reliable is a recipe for split-brain.

Twitter’s celebrity hot-shard incidents (various dates, recurring). Major celebrities posting during peak hours have repeatedly caused per-shard latency spikes. The “Justin Bieber” pattern (one user with hundreds of millions of followers) was the canonical case study that drove Twitter’s celebrity-treatment architecture. The lesson: in user-keyed systems, hot users will appear, and the architecture must have explicit handling for them.

AWS DynamoDB hot-partition surcharges (pre-2018). Customers hitting hot partitions saw their costs multiply because DynamoDB billed by per-partition capacity, not per-table. Customers with workloads that didn’t distribute evenly across partition keys received surprise bills. AWS’s response was the 2018 “adaptive capacity” feature that auto-balances capacity across partitions. The lesson: the billing model must align with the actual cost structure; surprising customers with hot-partition costs created such operational pain that AWS rebuilt the underlying capacity allocation.

These incidents share a common shape: a sharded architecture works fine in normal operation but degrades catastrophically when load skew exceeds expectations. Resilient sharded systems plan for skew explicitly — capacity headroom, hot-key detection, manual relocation tools, escape hatches.

12. Open Questions

• At what cluster scale does the operational overhead of explicit sharding (Vitess, Cassandra) become more or less expensive than building on a sharded-by-default system (Spanner, CockroachDB)?
• How well does bounded-load consistent hashing (Mirrokni 2018) work in practice compared to ad-hoc hot-key handling? Few public benchmarks compare them on real workloads.
• When does adopting per-key dedicated shards (the “hot tenant gets its own shard” pattern) outperform other hot-key mitigations?
• How do serverless / auto-scaling sharded systems (Aurora Serverless, DynamoDB on-demand, Spanner Autoscaler) change the operational profile? Do they remove the partition-key-decision pain or just hide it?
• What’s the future of “sharding by default” relational engines (Yugabyte, TiDB, CockroachDB) replacing manually sharded MySQL/Postgres deployments? Will the operational simplicity outweigh the per-transaction latency cost for typical workloads?

A note on one figure that should not be read as a hard fact: the “5–10% of revenue from 5–10% of tenants” power-law framing in Pitfall 10 is an illustrative heuristic, not a measured constant. The exact threshold at which a SaaS tenant warrants special treatment (dedicated shard, small-fleet shard, or sub-tenant resharding) is set by each system’s own operational economics — storage bytes, request rate, user count — and varies widely. Treat the numbers as shape, not specification.

13. See Also

• Consistent Hashing — the canonical hash-partitioning primitive
• Leader-Follower Replication Architecture — typically composed for per-shard replication
• Leaderless Replication Architecture — Cassandra’s per-shard composition
• Peer-to-Peer Architecture — discovered placement vs sharded specified placement
• Two-Phase Commit — used for cross-shard transactions
• Raft — used for per-shard consensus in CockroachDB / Spanner
• Multi-Leader Replication Architecture — multi-region sharded composition
• Master-Worker Architecture — different scale-out pattern; coordinator-driven rather than partitioned
• Bloom Filter — used to summarize per-shard contents for cheaper cross-shard queries
• LSM Tree — typical storage engine within each shard
• Distributed Key Value Store System Design — the canonical sharded + leaderless system
• Distributed SQL Database System Design — sharded + leader-follower per range
• Distributed Log System Design — Kafka per-partition leadership is a sharded composition
• Twitter Newsfeed System Design — celebrity fanout illustrates the hot-key problem
• Google File System Design — chunkserver placement is range-sharded
• Amazon S3 Object Storage System Design — metadata is sharded by key
• System Architectures MOC
• SWE Interview Preparation MOC