Netflix Streaming System Design

Netflix is the world’s largest subscription video-on-demand (SVOD) service, delivering a curated catalog of films and television to ~270 million subscribers across ~190 countries. Architecturally it is the inverse of YouTube: where YouTube ingests user-generated content at firehose rate, Netflix licenses or produces a (relatively) small catalog (~17 000 titles publicly estimated as of 2023) but distributes those titles at unprecedented scale — at peak hours Netflix has accounted for 30%+ of all downstream Internet traffic in North America (Sandvine “Global Internet Phenomena Report” 2019). The system is famous for three distinctive design decisions: a purpose-built CDN (Open Connect) deployed inside ISP networks, a microservices-on-AWS control plane (~700–1000 services depending on the year reported), and a chaos-engineering culture in which production failures are deliberately induced (Chaos Monkey, Simian Army). Together these encode a thesis: streaming is a content-delivery problem, not a compute problem; treat AWS as the brain and your own CDN as the muscle.

1. Functional and Non-Functional Requirements

Functional

• Stream movies and TV shows to phones, tablets, computers, smart TVs, set-top boxes, and consoles. Resolutions from 480p to 4K HDR with Dolby Vision / HDR10+ and Dolby Atmos audio.
• Profiles per account (typically up to 5), each with separate recommendations, watch history, parental controls, and language preferences.
• Offline downloads to mobile devices, with DRM-protected expiration.
• Recommendations — personalized homepage rows, “Continue Watching”, post-credit “Up Next”, thumbnails personalized per user.
• Search by title, actor, genre, or fuzzy phrase.
• A/B testing on virtually every UI surface (artwork, copy, row order, autoplay behavior).
• Multi-device sync — stop on phone, resume on TV at the exact frame.
• Subtitles and dubbed audio in dozens of languages, switchable per session.

Non-Functional

• Subscriber base ~270 M (2024 public earnings); peak concurrent streams in the tens of millions globally, with massive regional spikes (e.g., a popular release in Korea or India).
• Startup latency ≤ 2 s globally — the player must show video within 2 seconds of clicking Play.
• Rebuffer ratio < 1% (Netflix has internally reported this as their north-star quality-of-experience metric for years).
• Availability ≥ 99.99% for the play path (52 minutes / year). The browse / catalog path tolerates more downtime because users can still watch what they’re already watching.
• Durability of source masters: Netflix Originals are irreplaceable, requiring 11+ nines durability via cross-region replication.
• Encoding cost budgeted in the millions of CPU-hours per year — relevant because per-title and per-shot encoding multiplies that bill. The encoder farm is reportedly one of Netflix’s larger AWS Batch line items; per-title encoding for a marquee release can consume thousands of EC2 instance-hours.
• Subscriber data privacy. Watch history is sensitive — what a user watches reveals personal preferences and emotional states. Netflix encrypts watch history at rest and access-controls reads via service-level RBAC; subscriber-data subject-access requests (GDPR, CCPA) are operationally routine.
• Catalog rights and licensing. Each title has region-specific availability, expiration, and exclusivity windows. The catalog service enforces these atomically at request time; a missed enforcement check has direct contractual / legal consequences.
• Geographic reach — sub-100 ms first-byte from any populated country with an active Netflix presence.

2. Capacity Estimation — Back-of-the-Envelope

These are illustrative numbers; Netflix’s actual scale is opaque in current form.

Catalog size. ~17 000 titles × ~10 average rendition variants × ~90 minutes average runtime × ~5 Mbps ABR-weighted bitrate = roughly several PB of master encoded media, before per-title and per-language audio variants. With dubs and subtitles in 30+ languages, plus VP9 / H.264 / AV1 codec triplication for device coverage, the total Open Connect storage footprint is on the order of single-digit PB per region — small enough that a fully replicated copy can sit on a single OCA (Open Connect Appliance) cabinet at a large ISP.

Egress. During peak hours, public reports estimate 200+ Tbps of Netflix egress globally. With ~5 Mbps per stream weighted average, that supports tens of millions of concurrent streams. By far the dominant network cost: hence the existence of Open Connect.

Recommendation features. Per-user feature vectors (watch history, completion fractions, ratings) for 270 M users at ~kB each → ~hundreds of TB just for personalization features.

Catalog metadata. ~17 K titles × ~kB-of-metadata × hundreds of localized rows = small (~GBs); fits comfortably in any modern relational store.

Watch history events. ~270 M users × ~1 hour/day × event-per-segment cadence ≈ tens of billions of telemetry events / day. This is what drives the choice of Cassandra (high write throughput) for viewing history and Kafka for the event pipeline.

Concurrent stream peaks. A new global release (e.g., a marquee Netflix Original) can drive ~50 M concurrent streams within hours of launch. The OCA fleet must absorb this without origin fall-through; pre-positioning is the make-or-break operational capability. Streams are uneven across regions — APAC may peak when EMEA is asleep — so the global peak is lower than the regional peak summed across regions, which gives the fleet some breathing room.

Live event sizing (live sports on Netflix, 2024+). Live events lift the concurrency by another order of magnitude over typical SVOD peaks because everyone watches at the same time. Two 2024 events bracket the picture. The Jake Paul vs. Mike Tyson fight (16 November 2024) — which Netflix said peaked at ~65 million concurrent streams — was marred by widely-reported buffering and blurriness; Down Detector logged ~85,000 problem reports and a $50M class-action followed, a vivid demonstration of the gap between Netflix’s VOD-tuned, pre-positioned infrastructure and live’s bursty, unpredictable real-time demand (Variety). The Christmas Day NFL doubleheader (25 December 2024) went much better and set streaming records: Netflix/Nielsen reported a roughly 65 million unduplicated U.S. viewer day, with each game averaging ~24 million average-minute-audience and U.S. viewership peaking at over 27 million during Beyoncé’s halftime show (About Netflix). The contrast between the two — same scale, very different experience six weeks apart — is itself the story: Netflix said it applied lessons from the Tyson-Paul fight (added capacity, deeper ISP interconnection) before Christmas.

3. API Sketch

The viewer-facing API is roughly:

# Discovery
GET    /api/profile                                        → profile metadata
GET    /api/homepage?profileId=...                         → personalized rows
GET    /api/title/{titleId}?profileId=...                  → title detail + artwork
GET    /api/search?q=...&profileId=...                     → ranked results

# Playback
GET    /api/playback/license?titleId=...&deviceId=...      → DRM license + manifest URL
GET    /manifest/{titleId}?deviceId=...                    → DASH MPD or HLS m3u8
GET    /segment/{segmentKey}                               → media segment from OCA

# State sync
POST   /api/playback/event                                 → start/stop/heartbeat events
GET    /api/playback/bookmark?profileId=...&titleId=...    → resume position

# Engagement
POST   /api/rate                                           → thumbs-up/down
POST   /api/list/add                                       → add to "My List"

The interesting endpoint is /api/playback/license, which returns both a DRM license (Widevine / FairPlay / PlayReady, depending on platform) and a signed manifest URL. Without a valid license token, the OCA refuses to serve segments — DRM is enforced by the CDN itself, not just the player.

4. Data Model

Netflix’s storage decomposes by access pattern:

(A) Cassandra clusters for high-write-throughput per-user data: viewing history, bookmarks, recommendation features, “My List”. Cassandra (Lakshman & Malik 2010) is a Dynamo-derived wide-column store using consistent hashing for partitioning (see Consistent Hashing and Distributed Key Value Store System Design) and an LSM-tree-based storage engine (see LSM Tree). Netflix has been Cassandra’s largest public deployment for over a decade — 2018 talks reported 2500+ nodes, 100 PB+ total data, 1+ trillion ops/day.

(B) EVCache (Netflix’s Memcached fork) for sub-millisecond reads of frequently accessed data — homepage rows, ranking features, session tokens. EVCache is distributed (consistent-hashed across hundreds of nodes), multi-region replicated (writes propagate cross-region asynchronously), and layered with L1 in-process caches. The Netflix tech blog post “EVCache: Distributed in-memory datastore for cloud” (2013) describes the architecture in detail. See LRU Cache for the per-node eviction story.

(C) MySQL / Aurora for catalog metadata that requires relational integrity and is read-heavy but write-light — the canonical title list, royalty records, regional licensing windows.

(D) Elasticsearch for search and operational log indexing.

(E) S3 for the master encoded files. Open Connect pulls from S3 to populate OCAs.

(F) Kafka as the central event-bus; every meaningful action (play start, segment fetched, error encountered) emits a Kafka event consumed by analytics, recommendation training, billing, and ops dashboards. Netflix’s “Keystone” pipeline reports trillions of events per day.

A simplified data shape:

profiles:           profile_id (PK) | account_id | name | locale | maturity_level
viewing_history:    (profile_id, started_at) PK | title_id | duration_s | completion_pct
                    (Cassandra: partition by profile_id, clustering by started_at)
bookmarks:          (profile_id, title_id) PK | last_position_s | updated_at
catalog:            title_id (PK) | type | year | runtime | genre[] | artwork[] | rating
licensing_windows:  (title_id, region) PK | start_ts | end_ts
recs_features:      profile_id (PK) | feature_blob (binary)

5. High-Level Architecture

flowchart TB
    subgraph AWS["AWS Control Plane (Multi-Region Active-Active)"]
        Edge[Edge API<br/>Zuul Gateway]
        APIs[1000+<br/>Microservices]
        Edge --> APIs
        APIs --> Cas[(Cassandra)]
        APIs --> Cache[(EVCache)]
        APIs --> Search[(Elasticsearch)]
        APIs --> Kafka((Kafka))
        Kafka --> Stream[Stream Processing<br/>Mantis / Flink]
        Stream --> RecsTrain[Offline Recs<br/>Training]
        RecsTrain --> Cas
    end

    Encoder[Per-Title Encoder<br/>Reloaded on AWS Batch] -->|HLS/DASH segments| S3[(S3 Master Storage)]

    subgraph OC["Open Connect (Netflix's CDN)"]
        OCAEmbed[OCA Embedded<br/>in ISP Networks]
        OCAIX[OCA at Internet<br/>Exchange Points]
        OCAEmbed -.fill from S3.-> S3
        OCAIX -.fill from S3.-> S3
    end

    Client[Netflix Client<br/>TV / Phone / Web] -->|browse| Edge
    Edge -->|playback init| Client
    Client -->|segment fetch| OCAEmbed
    Client -.fallback.-> OCAIX
    Client -.last resort.-> S3

Walk-through. The architecture is sharply split between AWS (the brain) — handling all browsing, authentication, recommendations, billing, and metadata — and Open Connect (the muscle) — handling video segment delivery. A typical session involves perhaps 100 KB of HTTP traffic to AWS to bootstrap the player and render the homepage, then hundreds of MB to gigabytes of segment traffic to OCAs that AWS never sees. The dotted “fill from S3” arrows are infrequent — they happen overnight when the OCA pre-fetches new releases, not during user sessions. The two redundant fallback paths (OCA-IX, then S3) are why Netflix can claim 99.99% playback availability: even if an embedded OCA goes down, IX-located OCAs absorb the load; even if the entire OC layer fails, S3 serves directly (at higher latency / cost).

6. Request Flow — Starting a Stream

sequenceDiagram
    participant C as Client
    participant Z as Zuul (Edge API)
    participant P as Playback Service
    participant L as License Service
    participant CT as Catalog Service
    participant DS as DRM Server
    participant O as OCA (Open Connect)

    C->>Z: GET /playback?titleId=X
    Z->>P: route(req, authToken)
    P->>CT: getTitle(X, region, locale)
    CT-->>P: metadata + rendition list
    P->>L: requestLicense(titleId, deviceId, profile)
    L->>DS: issueLicense(deviceId, keyId)
    DS-->>L: signed license blob
    L-->>P: license + signed manifest URL
    P-->>Z: response
    Z-->>C: license + manifest URL + steering URL
    C->>C: parse manifest, choose initial rendition
    loop ABR loop, every ~4-10 s
        C->>O: GET /seg/{key} (signed URL)
        O-->>C: segment bytes
        C->>C: decrypt with license, decode, render
        C->>C: measure throughput, adjust rendition
    end
    C->>Z: POST /event (heartbeat)
    Z->>Kafka: publish event

The choreography exposes the steering mechanism: the client does not get a single CDN domain. The Steering Service (CODA) hands back signed URLs that point at specific OCAs chosen for that session — typically one inside the user’s own ISP (the embedded OCA, fastest path), with an IX-located OCA as fallback. CODA’s choice is grounded in the real-time reports the Cache Control Service collects from every OCA: health, file availability, and the BGP routes each appliance has learned, so steering reflects actual network topology rather than coarse geography.

7. Deep Dive — Three Critical Components

7.1 Open Connect — A Purpose-Built CDN

Open Connect is Netflix’s most distinctive infrastructure choice. Rather than buy CDN bandwidth from Akamai or Limelight (as it did briefly in the late 2000s, per Adhikari et al. INFOCOM 2012), Netflix designed its own CDN appliance — the Open Connect Appliance (OCA) — and convinced ISPs to host them inside their own networks for free. The deal: the ISP saves on transit and peering costs (Netflix traffic stays inside the ISP’s edge), and Netflix gets sub-millisecond network distance to the subscriber.

OCAs come in two functional flavors, both documented in the FreeBSD Foundation’s 2021 “Netflix Open Connect” case study (Greg Wallace, FreeBSD Journal Jan/Feb 2021). A storage OCA is a 2RU chassis holding ~248 TB of disk and a 40 Gb/s NIC — it holds a large slice of the catalog and is bandwidth-modest. A flash OCA is a 1RU chassis with ~16 TB of NVMe-attached flash, 96 GB of RAM, an Intel 6122-class CPU, and a 100 Gb/s NIC — it holds the hottest content and serves it at very high throughput. A large ISP receives a cabinet that mixes both; a small one may get a single appliance. (Note the correction to a common misconception: OCAs are 1RU/2RU rack units, not 4U, and the per-appliance flash capacity is tens of TB, not ~100 TB — the larger storage figures apply to the disk-based storage OCAs.)

Pre-positioning is the operational secret. Open Connect catalogs are not pulled lazily on user demand — that would defeat the embedded model. Instead, every night during configurable off-peak fill windows (when viewing in the region is very low), OCAs fill themselves with the day’s catalog updates, prioritized by Netflix’s popularity-prediction algorithms: today’s hot releases get fresh capacity; long-tail titles get cached only at IX-located OCAs. Crucially, OCAs can also fill from one another rather than always from a central origin, which minimizes use of expensive internet “backbone” capacity during the update cycle (per the Open Connect Overview). By the time peak hours begin, the most-requested content is already on hardware inside the user’s ISP.

Steering and the control plane. The Open Connect Overview names the actual AWS-side components: a Cache Control Service (CCS) receives periodic reports from every OCA (health, the BGP routes the appliance has learned from its peer router/switch, and which files it holds on disk), and a Steering Service (CODA) uses that data to select, per request, the OCAs that should serve the required files. When a user hits Play, the Playback Apps in AWS check authorization and licensing and decide which files are needed for that client and network condition; CODA then picks the OCAs that hold those files and are closest and healthy, generates signed URLs for them, and hands them back to the client. If those conditions cannot be met (cold title, embedded OCA full), CODA falls back to IX-located OCAs. Because the OCAs speak BGP to their host networks and report learned routes upstream, steering decisions are grounded in real network topology, not just geographic guesswork.

OCA software stack — NGINX on FreeBSD, not a bespoke server. A frequent misconception is that OCAs run a “custom HTTP stack.” In fact they run NGINX (developed in a long-running partnership between NGINX and Netflix), the BIRD internet routing daemon for BGP, and a lightly customized build of FreeBSD — and notably they track FreeBSD’s head (development) branch, not a stable release, because (per Netflix engineering manager Jonathan Looney at FOSDEM 2019) tracking head keeps feature velocity high and avoids the “vicious cycle of infrequent merges, many conflicts, slower velocity” that waiting for the stable branch would impose. The heavy customization is in the kernel data path, not a from-scratch web server: Netflix uses sendfile(2) with kernel TLS (kTLS) so that bulk encryption happens in the kernel, in the same sendfile pipeline that already moves bytes from the file system to the socket — TLS session negotiation and key management stay in NGINX’s userspace, but once keys are negotiated they are pushed into the kernel and associated with the socket, so no byte ever has to be copied to userspace just to be encrypted. The TCP stack is tuned for high-throughput WAN links, and NVMe queue depth/IOPS are continuously monitored because exceeding the SSD’s sustained-write capacity during cache fills can starve the read-serving path.

Why FreeBSD over Linux. It is tempting to assume the choice was a historical accident of team expertise — it was not. Per the same FreeBSD case study, Jonathan Looney’s own summary is that Netflix “came for the license and stayed for the efficiency.” The license came first: because OCAs are owned by Netflix but operated inside ISP networks, FreeBSD’s permissive BSD license raised fewer redistribution complications than the GPL would have. The efficiency is what kept them: FreeBSD’s network stack proved an excellent base for the kTLS / async-sendfile / NUMA work, and Netflix upstreams those contributions (the team employs several FreeBSD committers). Netflix measures the payoff in three dimensions — throughput efficiency, development efficiency, and operational efficiency.

Throughput evolution. Single-server throughput has climbed steeply as Netflix pushed the FreeBSD data path. The “Serving 100 Gbps from an Open Connect Appliance” generation (~2016, Intel-based, 16 TB NVMe, 100 G NIC) gave way to 200 Gbps servers around 2020, then ~400 Gbps in 2021 on AMD EPYC “Rome” (EPYC 7502P) with NIC-offloaded TLS on Mellanox ConnectX-6 Dx adapters, and a ~800 Gbps single-server milestone demonstrated in 2022 (Drew Gallatin, “Serving Netflix Video Traffic at 800Gb/s and Beyond,” EuroBSDCon 2022). The point of interest is not the raw number but the method: each doubling came from removing a software bottleneck (async sendfile, kTLS, NUMA-aware buffer placement, then offloading encryption to the NIC) rather than from buying faster silicon alone.

7.2 Per-Title and Per-Shot Encoding

Netflix’s Per-Title Encode Optimization (Aaron et al., 2015) was a watershed moment in adaptive bitrate streaming. The pre-2015 industry default was a fixed bitrate ladder — every title was encoded at the same set of (resolution, bitrate) points (e.g., 720p @ 3 Mbps, 1080p @ 5 Mbps, etc.). This wasted bandwidth on simple content (a talking-head documentary doesn’t benefit from 5 Mbps) and starved complex content (a fast-action sci-fi sequence might need 8 Mbps).

The per-title insight: run the encoder at many (resolution, bitrate) candidates, measure objective quality (PSNR, then later VMAF — Netflix’s own perceptual metric), and pick the convex hull of (bitrate, quality) points as the rendition ladder for that title. Some titles have a 360p rendition only at 200 kbps; others jump straight from 480p to 1080p because the in-between renditions don’t add quality.

The follow-up per-shot encoding (Netflix tech blog 2018) goes further: split the title into shots (camera-cuts) using shot detection, encode each shot independently with its own optimal parameters, then re-stitch. A film’s quiet exposition shots get less budget; its action set-pieces get more. Reported savings: 20–40% bandwidth at equivalent VMAF, sustained across the catalog.

VMAF — Video Multi-method Assessment Fusion. Traditional video quality metrics (PSNR, SSIM) correlate poorly with human-perceived quality. Netflix open-sourced VMAF (2016) — a learned model trained on thousands of human-rated quality scores that combines several elementary metrics. VMAF outputs a 0–100 score where 100 is “perceptually identical to source.” The encoding ladder is selected to maintain a target VMAF (e.g., 95 for premium tier, 80 for mobile) at the lowest possible bitrate. VMAF is now widely used across the industry beyond Netflix, including by competitor encoder vendors.

The cost: each release requires thousands of encoder runs to find the convex hull. Netflix runs this on AWS Batch, sometimes burning >1000 EC2 instances on a single new release. The bandwidth savings at delivery dwarf the encoding cost.

The encoding farm’s economics. Encoding is paid in CPU-hours; bandwidth is paid in bytes. The break-even point: when does an extra encoding-hour save more bytes than it costs? At Netflix’s scale, the answer is “almost always” for the popular tail — millions of viewers consuming each title, even a 1% bitrate reduction recovers the encoding cost in bandwidth saved within hours of release. For long-tail titles (watched a few hundred times total), the answer flips: aggressive per-shot encoding may not be worth its CPU cost. Netflix’s pipeline therefore tiers encoding aggressiveness by predicted viewership.

7.3 Cassandra and EVCache for Per-User State

Per-user state (viewing history, bookmarks, recs features) is the data-write workhorse. Strong consistency is unnecessary — if your bookmark is briefly stale, the world doesn’t end. Throughput, availability, and write latency dominate.

Cassandra delivers this via:

• Consistent-hashed partitioning (see Consistent Hashing) — partition_key (typically profile_id) determines which node owns the data; the next R nodes clockwise on the ring are replicas.
• LSM-tree storage engine (see LSM Tree) — writes are appended to an in-memory memtable and a write-ahead log, flushed to immutable SSTables, periodically compacted. This makes writes essentially sequential disk I/O, ideal for the high-write workload of per-user telemetry. Reads, however, may need to consult multiple SSTables and the memtable, plus filter by Bloom Filter to avoid disk I/O on missing keys. Cassandra’s read amplification is the central tradeoff against its write performance — operationally, cluster sizing is dominated by read latency and compaction overhead, not write throughput.
• Tunable consistency — ONE (any one replica responds) for low-latency writes, QUORUM for stronger reads. Netflix typically operates at the eventually-consistent end for telemetry.
• Multi-region replication with active-active writes; conflict resolution is “last-write-wins” via Cassandra’s hybrid logical clock.

EVCache sits in front: a Memcached-protocol cache distributed across hundreds of nodes per region, multi-region replicated. Hot reads (today’s homepage rows for a popular profile) hit EVCache in < 1 ms. Cold reads or stale-on-cache-miss reads fall through to Cassandra.

Why a fork of Memcached, not Redis? When EVCache was started (~2011), Redis’s persistence and replication features were less mature than they are today; Memcached was a simpler, faster cache primitive. Netflix forked Memcached to add: (1) cross-region replication, (2) per-key tagging for selective invalidation, (3) detailed metrics integration with Atlas, and (4) cluster topology managed by their internal service-discovery system Eureka. The fork is now publicly available as github.com/Netflix/EVCache.

The dual-tier pattern is what gives Netflix sub-second homepage latency at the cost of two synchronized data systems plus a custom client library that handles fan-out, retries, and consistency-vs-staleness tradeoffs.

EVCache’s three-tier write replication. Writes go to a primary EVCache cluster, then asynchronously to a secondary cluster in the same region (so cache survives the loss of the primary), then to clusters in other regions (so a region failover preserves cache state). The replication is best-effort and bounded-staleness — Netflix accepts that a region failover can briefly serve stale cache contents while the secondary catches up. This trade is justified by latency: if EVCache writes had to wait for cross-region acknowledgement, write latency would balloon from sub-millisecond to tens of milliseconds, and the homepage rendering would block on it.

Cassandra anti-entropy. Cassandra’s read-repair and Merkle-tree-based anti-entropy (Anti-Entropy when written) periodically compares replica copies and reconciles divergence caused by transient failures. Netflix’s experience has been that anti-entropy at scale is operationally heavy — repair jobs over multi-PB clusters can take days. The team has documented multiple Cassandra contributions improving repair behavior; the tombstone-behavior post by Spotify discusses similar patterns.

8. Scaling Strategies

Microservices on AWS. Netflix runs 700–1000 microservices (the exact count fluctuates and the precise modern figure isn’t published; pre-2017 talks consistently cited “around 700”). Each owns a slice of data and an API. The architectural style is Conway’s-Law-friendly: small teams own small services. The downside — cross-service consistency, request fan-out, and observability complexity — is paid for with pervasive standardized tooling (Spinnaker for deploys, Atlas for metrics, Mantis for stream processing, Hystrix historically for circuit breakers, replaced by resilience4j-style libraries).

Zuul as the edge gateway. Netflix’s Zuul is an open-source dynamic edge gateway — every browser, mobile, and TV request first lands at Zuul before reaching internal microservices. Zuul handles authentication, request routing, request enrichment (geolocation, user metadata), retries, and circuit-breaking. It’s also where A/B test variant assignment can be made (the Zuul filter chain mutates the request based on the user’s experiment buckets). Zuul has been rewritten multiple times across generations as the team learned more about asynchronous request handling — Zuul 2 moved from blocking (Tomcat / Servlet) to non-blocking (Netty) to handle higher concurrency per node.

Active-active multi-region. Netflix runs in three AWS regions (US-East, US-West, EU-Ireland traditionally). Each region serves traffic from its corresponding geography under normal conditions and can absorb the load of any other region in failover. The “Active-Active for Multi-Regional Resiliency” Netflix blog post (2013) documented the migration; it required Cassandra cross-region replication, EVCache cross-region replication, and substantial DNS / steering work.

Chaos engineering. Pioneered by Netflix; the original Chaos Monkey randomly terminates EC2 instances during business hours to ensure services tolerate node failure. The Simian Army extends this with Latency Monkey (introduces network slowness), Conformity Monkey (kills non-compliant instances), and the famous Chaos Kong which fails an entire AWS region. The thesis (Basiri et al. 2016): the only way to know your system tolerates a class of failure is to cause that failure regularly, ideally during normal business hours when your team can react.

Why business hours, not nights or weekends. The cultural insight in the Chaos Monkey design is that failures injected during off-hours are paged on, dealt with painfully, and mostly forgotten. Failures injected during business hours are dealt with by the team that wrote the code, in normal working time, with full team support. The fix doesn’t get lost in the on-call handoff. This subtle scheduling choice is what made chaos engineering culturally adoptable rather than a low-grade ops nightmare.

Recommendation pipeline scaling. Netflix’s recommender (Gomez-Uribe & Hunt 2015) uses a multi-stage candidate-generation + ranking pipeline. Candidates are precomputed offline (collaborative filtering, content-based, deep models) and cached in EVCache; ranking is online, reading user features and applying a model in tens of milliseconds. See forward reference Recommendation Engine System Design.

Per-row personalization at scale. The Netflix homepage is composed of dozens of rows (“Continue Watching”, “Trending Now”, “Because you watched X”); each row is independently ranked per profile. Computing this naively for 270 M profiles would be infeasible; instead, the system computes row-level rankings offline for the long tail of rows and only the top few rows online per session. Even artwork is personalized — the same title may show different thumbnails to different users based on which artwork they’re predicted to click on, an experiment-driven optimization documented in the Netflix tech blog (“Artwork Personalization at Netflix”, 2017).

Edge encoding for ad-supported tier. Netflix’s ad-supported tier (launched 2022) requires Server-Side Ad Insertion (SSAI) that splices ads into the manifest at request time, with personalized ad selection based on user features. This added a small but critical real-time component to a previously batch-dominated content pipeline.

9. Real-World Deployment Notes and Citations

• Open Connect. Whitepaper: openconnect.netflix.com — the most thorough public description of CCS, CODA steering, fill windows, and the two OCA deployment modes.
• OCA hardware & FreeBSD. FreeBSD Foundation “Netflix Open Connect” case study (Greg Wallace, FreeBSD Journal Jan/Feb 2021) — gives the 248 TB/2RU storage-OCA and 16 TB-NVMe/1RU/96 GB flash-OCA specs, the NGINX + BIRD + FreeBSD-head stack, kTLS/async-sendfile, and the “came for the license, stayed for the efficiency” rationale. Throughput evolution to ~800 Gbps: Gallatin, “Serving Netflix Video Traffic at 800Gb/s and Beyond,” EuroBSDCon 2022.
• Cloud migration. Izrailevsky, “Completing the Netflix Cloud Migration,” About Netflix (Feb 2016) — last streaming data center shut January 2016 after a seven-year migration.
• Cassandra at Netflix. Multiple Cassandra Summit talks; “Cassandra at Netflix” 2018 reported 2500+ nodes / 100 PB. Specific figures evolve; treat as historical anchor.
• Per-title encoding. Aaron et al. 2015 Netflix tech blog. Subsequent VMAF and per-shot work cited from the same blog.
• Chaos engineering. Izrailevsky & Tseitlin 2011 Netflix tech blog; Basiri et al. 2016 IEEE Software.
• EVCache. Netflix tech blog: “EVCache: Distributed in-memory datastore for cloud”.
• Recommendation system. Gomez-Uribe & Hunt, ACM TMIS 2015.

Uncertain

Verify: the microservice count (~700–1000) and the Cassandra node count (2500+) / total data (100 PB+). Reason: these are drawn from conference talks dated 2017–2018 and Netflix does not publish current figures; both have almost certainly grown. (The OCA hardware specs and form factor previously flagged here are now verified against the FreeBSD Foundation case study and corrected in §7.1.) To resolve: a recent Netflix engineering talk or a Cassandra/ScyllaDB summit update would settle the live numbers. uncertain

10. Tradeoffs

Dimension	Netflix’s Choice	Alternative	Why Netflix’s Choice
CDN	First-party (Open Connect)	Third-party (Akamai / Limelight)	Cost — at Netflix’s egress, third-party would be financially infeasible; also enables ISP partnerships
Compute platform	AWS	Own data centers	Operational simplicity; Netflix completed its seven-year cloud migration and shut its last streaming data center in January 2016 (Izrailevsky, About Netflix)
Per-user data	Cassandra (eventually consistent)	RDBMS (strongly consistent)	Throughput and availability dominate; staleness budget is generous
Encoding strategy	Per-title and per-shot	Fixed ladder	Bandwidth savings worth the encoding cost at scale
Failure response	Active-active multi-region + chaos engineering	Pilot-light DR	Test failover continuously rather than hope it works once
Microservice granularity	Many small services	Fewer larger services	Team ownership scales with service count; observability tooling absorbs the complexity
Manifest format	DASH primarily, HLS for iOS	Custom protocol	Standards adoption was complete by Netflix’s growth phase
Recommendation freshness	Mostly offline + online re-rank	Fully online	Cost / latency tradeoff favors offline batch with online personalization

11. Pitfalls and Gotchas

1. OCA pre-positioning gone wrong. A surprise viral release that wasn’t pre-positioned forces fall-through to S3, which is slow and expensive. Netflix mitigates with rapid-promote heuristics and a manual “fast-fill” tool ops can trigger.
2. Cassandra hot partition. A celebrity profile receiving a flood of writes (e.g., a Times Square promotional account) creates a single-partition hot spot. Mitigation: synthetic partitioning where the partition key includes a salt or shard suffix.
3. EVCache invalidation lag. Cross-region replication is asynchronous; users moving between regions (or, more commonly, services reading from a non-primary region) see stale data briefly. Acceptable for personalization, dangerous for billing.
4. DRM key rotation. When Netflix rotates DRM keys, existing pre-fetched offline downloads can become unplayable until renewed. Renewal logic in the client must handle this gracefully.
5. Catalog licensing windows. A title is available in some regions and not others, with hard start / end timestamps. The catalog service must enforce these atomically; an ad-supported tier or a regional rights expiration that leaks even one minute can trigger contractual penalties.
6. Region failover. When a region is failed over (deliberately, via Chaos Kong, or accidentally), in-flight sessions must seamlessly migrate. The client retries on the steering URL, which now points to the surviving regions; sessions resume from bookmarks. Bugs in this path are caught only by exercising the failure regularly.
7. A/B test contamination. With hundreds of A/B tests running simultaneously, interaction effects between experiments become significant. Netflix has invested heavily in causal-inference tooling to disentangle.
8. Encoder bug retroactivity. A bug in the encoder or VMAF metric that ships into production may degrade the entire catalog. Re-encoding 17 K titles is a multi-month project; Netflix maintains aggressive pre-release validation to avoid this.
9. OCA software updates. Pushing a new OCA build to ~thousands of appliances spread across ISP networks worldwide is non-trivial. A bad build can take down ISP-embedded boxes that Netflix doesn’t have direct physical access to, requiring ISP cooperation to recover. Updates are staged (canary appliances first) and the Open Connect team maintains rollback procedures.
10. Subtitle / dub asset drift. A title may have 30 subtitle tracks and 15 dubbed audio tracks. Each is an independent asset that can be updated independently (e.g., correcting a translation error). The manifest service must reconcile multiple asset versions atomically — a viewer mid-playback shouldn’t see subtitles for the new audio with the old timing.
11. Profile-vs-account confusion in analytics. Netflix’s “5 profiles per account” model means usage data is per-profile, but billing and entitlement is per-account. Recommendation models trained on profile-level data, mixed with account-level signals (subscription tier, region), require careful joining logic. Bugs here mis-attribute behavior across family members.
12. Live streaming on a VOD-first stack. Netflix’s recent live events exposed mismatches between the OCA model (which assumes pre-positioned content) and live’s lack of foreknowledge — most visibly the Jake Paul vs. Mike Tyson fight (Nov 2024), which buffered badly at ~65 M concurrent streams before the smoother NFL Christmas doubleheader six weeks later. Live ingest, real-time transcode, and rapid edge-fill are bolted onto an architecture that wasn’t built for them; pre-positioning, the OCA model’s core advantage, simply does not apply when the bytes don’t exist until the moment they are watched.
13. Cross-region cache coherence. EVCache replication is asynchronous; a write in US-East takes seconds to propagate to EU-Ireland. Services that read in EU-Ireland after writing in US-East may see stale data. Most code paths tolerate this; some (e.g., entitlement checks for new subscribers) do not, requiring explicit cross-region read-after-write checks that bypass cache.
14. Global account state vs regional data residency. Some jurisdictions (EU GDPR, China, Russia) require user data to remain within national borders. Netflix’s global active-active model collides with these requirements, requiring per-region data partitioning that violates the otherwise-uniform replication model. Implementations vary from “store in the home region only” to “encrypt at rest with region-specific keys.”
15. Service mesh sprawl. With ~1000 microservices, dependency graphs become deep and brittle. A single page render can fan out to 50+ services; a slow service buried 10 hops deep can cause cascading slowness. Hystrix-style circuit breakers limit blast radius but they’re a band-aid; the deeper fix is latency budget allocation — every hop in the call graph has a published p99 latency budget and slow services are detected and bypassed via fallbacks.

Adaptive Bitrate algorithms in production. The ABR algorithm selecting the next rendition to fetch is a research area in its own right — model-predictive controllers, reinforcement-learning-tuned controllers, hybrid heuristic + learning approaches. Netflix has published multiple papers on its production ABR algorithm; the canonical research line traces through Huang et al. SIGCOMM 2014 (buffer-based) → Yin et al. SIGCOMM 2015 (model-predictive) → Mao et al. SIGCOMM 2017 (Pensieve, RL-based). Netflix’s production deployment is a hybrid that combines a buffer-based safety floor with a learned policy for quality climbing decisions.

Graphic / artwork personalization pipeline. Every title has multiple “artwork” candidates — different thumbnails, key art, character-focused images. The platform A/B tests which artwork resonates with each user, and over time, a learned model predicts the best artwork per profile. The “Artwork Personalization at Netflix” 2017 tech blog walks through the contextual-bandit formulation. This is a meaningful UX lever: changing a title’s artwork can shift its view rate by double-digit percentages for some user segments.

12. Common Interview Variants

• Design Netflix — the canonical SVOD prompt.
• Design YouTube — see YouTube Video Streaming System Design; contrast UGC ingest and ad-supported revenue.
• Design Hulu / Disney+ / HBO Max — same primitives; differentiation is in content licensing, live channels (Hulu), or franchise integration (Disney+).
• Design a CDN — see Content Delivery Network System Design; Open Connect is one specialized instantiation.
• Design a chaos engineering platform — fault injection harness, blast-radius limits, automated experiment hypotheses.
• Design Cassandra-style KV store — see Distributed Key Value Store System Design.

13. Open Questions and Areas of Uncertainty

• How does Netflix’s ad-supported tier (launched 2022) change the architecture? Ad insertion / SCTE-35 markers / personalized ad selection are non-trivial additions.
• At what point does the per-shot encoding cost exceed the bandwidth savings? Has Netflix found the equilibrium or is the encoder still getting smarter?
• How does live sports streaming (NFL deal, WWE Raw 2025) change the OCA model? Live can’t be pre-positioned in the same way.
• Has the chaos-engineering practice scaled to the same intensity post-2020, or has it been moderated as the system matured?

Uncertain

Verify: the internal architecture of Netflix live streaming (ingest topology, real-time transcode pipeline, how live content reaches OCAs without pre-positioning). Reason: the outcomes are now well-sourced (Tyson-Paul ~65 M concurrent streams with buffering, Nov 2024; NFL Christmas ~65 M unduplicated U.S. viewers, Dec 2024 — both cited in §2), but Netflix has not published a thorough engineering deep-dive on the live mechanism the way it has for Open Connect VOD. To resolve: a Netflix Tech Blog post or conference talk on the live pipeline. Treat the live extension as evolving. uncertain

14. Why Netflix’s Architecture Matters Beyond Netflix

Netflix’s published engineering practices have influenced an entire generation of distributed-systems design. The cultural exports include:

• Microservices on AWS as a default. Netflix’s success on AWS in 2009–2015 was a major proof point that elastic public cloud could host streaming-class workloads at scale, displacing the expectation that media companies must run their own datacenters.
• Chaos engineering as a discipline. What started as Chaos Monkey is now an entire field with conferences, books, and certifications. Companies far smaller than Netflix run chaos experiments because the practice’s value scales down.
• Open-source contributions. Hystrix, Eureka, Zuul, Spinnaker, EVCache, VMAF, Atlas, Mantis — all open-sourced from Netflix — have been adopted broadly across the industry, often in deployments that don’t run any Netflix-style streaming.
• Per-title encoding. Now standard across video platforms (YouTube, Twitch, AWS Elemental MediaConvert).
• The “Backend for Frontend” pattern. Zuul is the canonical example of a server tier specifically tuned to consolidate / mediate calls for a particular client family (mobile, TV, web).

When the user asks “design Netflix” in an interview, they often want to hear evidence that the candidate can articulate why these patterns are good, not just enumerate them. Each of the items above is a tradeoff with costs as well as benefits — the discriminating answer mentions the cost.

15. See Also

• Major System Designs MOC
• SWE Interview Preparation MOC
• YouTube Video Streaming System Design — UGC counterpart
• Twitch Live Streaming System Design — live-first counterpart
• Spotify Audio Streaming System Design — audio-domain counterpart
• Content Delivery Network System Design — Open Connect generalized
• Consistent Hashing — Cassandra and EVCache partitioning
• LSM Tree — Cassandra’s storage engine
• LRU Cache — EVCache’s per-node eviction
• Distributed Key Value Store System Design — Cassandra-class systems
• Recommendation Engine System Design — homepage and “Up Next” pipeline
• Distributed Counter System Design — view counts and engagement
• Distributed Tracing System Design — Netflix’s observability stack