Adaptive Bitrate Streaming

Adaptive Bitrate Streaming (ABR) is the dominant technique for delivering video over the public Internet. The sender pre-encodes the same source asset at several quality tiers — a bitrate ladder — chops each tier into short segments (typically 2–10 seconds each), and publishes a manifest that lists every segment of every tier. The receiving player downloads segments over plain HTTP, picking the tier each time based on its measured network throughput and its current buffer occupancy. Over the last fifteen years three formats have converged around this design: HLS (HTTP Live Streaming, Apple 2009, RFC 8216) using .m3u8 text manifests; DASH (Dynamic Adaptive Streaming over HTTP, MPEG/ISO 2012, ISO/IEC 23009-1) using .mpd XML manifests; and CMAF (Common Media Application Format, MPEG/ISO 2018, ISO/IEC 23000-19) — a unified segment format that lets a single physical container be addressed by both HLS and DASH manifests, eliminating the cost of dual-encoding the same content twice. Together they underpin essentially every video delivery service on the open Internet: YouTube Video Streaming System Design (DASH), Netflix Streaming System Design (DASH then CMAF), Twitch Live Streaming System Design (HLS, including Low-Latency HLS), Apple TV+ (HLS), Disney+ (CMAF), and most live news streams. ABR replaced earlier proprietary stacks — Adobe’s RTMP-based dynamic streaming, Microsoft Smooth Streaming — because HTTP works through every firewall, every CDN already speaks it, and every cache understands it for free, which collapses the operational cost of large-scale video.

1. Intuition — Why ABR Replaced Streaming-Over-RTSP

Throughout the 2000s, video on the open Internet meant either Adobe Flash with Real-Time Messaging Protocol (Real-Time Messaging Protocol, RTMP) — a persistent TCP session with the encoder pushing frames — or one of several Real Time Streaming Protocol (RTSP) variants. Both shared a fatal weakness: the server held a stateful connection per viewer, and intermediate boxes (corporate firewalls, NAT routers, content delivery networks designed for static files) often blocked or mangled the non-HTTP traffic. Scaling that to YouTube’s tens of millions of concurrent viewers required custom edge servers everywhere.

The ABR insight (developed independently at Move Networks ~2006 and Apple ~2009) was to treat video like static files. Pre-chunk the asset into small segments. Store each segment as a regular HTTP object. Let the player do the streaming logic in the client. Suddenly every existing CDN, every existing reverse proxy, every existing browser cache, every existing corporate web proxy “just works” — the bytes look like any other HTTP response. The server side is stateless: edge servers cache segments like they cache JPEGs, the origin handles only cache misses, and load scaling is reduced to “how many GET requests per second can our CDN handle,” which is a problem CDNs already solved.

The price paid for this simplicity is latency: a client cannot start playing segment N until enough of segment N is on disk to decode (and historically until segment N is fully written). With ten-second segments, live streams trail real-time by 20–40 seconds. The Low-Latency HLS and Low-Latency DASH variants (since 2019/2020) shrink this by exposing partial segments before they are complete, but the architectural cost is real and ongoing.

2. The Core ABR Mechanics

2.1 Bitrate ladders

The encoder produces N renditions of the same source at different (resolution, bitrate, codec) triples. A typical 2024 ladder for a 4K live stream looks like:

Tier	Resolution	Frame rate	Video bitrate	Audio bitrate	Codec
1	256×144	30 fps	200 kbps	64 kbps	H.264 / AAC
2	426×240	30 fps	400 kbps	64 kbps	H.264 / AAC
3	640×360	30 fps	800 kbps	96 kbps	H.264 / AAC
4	854×480	30 fps	1.5 Mbps	96 kbps	H.264 / AAC
5	1280×720	30 fps	3.0 Mbps	128 kbps	H.264 / AAC
6	1920×1080	30/60 fps	5.0 Mbps	128 kbps	H.264 / AAC
7	3840×2160	60 fps	15.0 Mbps	192 kbps	HEVC or AV1

Bitrates roughly double tier-to-tier — the standard logarithmic spacing. Each tier is a fully-decodable stream; the player can switch between them at segment boundaries, and the result is gapless playback at varying quality.

2.2 Segments and manifests

Each rendition is sliced into segments. Segment duration is the first major design knob:

• Short segments (2 s). Lower start-up latency; lower live-stream end-to-end latency; players can react to bandwidth changes faster (next decision is 2 s away, not 10 s). Costs: more HTTP requests per minute → more overhead, more cache lookups, more keyframe density (every segment must start with a keyframe, which encodes ~30% larger than a P-frame), and worse compression efficiency.
• Long segments (10 s). Better compression (rare keyframes), fewer HTTP requests, simpler caching. Costs: 10 s of latency added before any switch decision, slow start-up, sluggish reaction to congestion.

The historical default was 10 s (Apple’s original HLS recommendation, 2009). It has trended down: most production deployments now use 4–6 s, and Low-Latency HLS uses partial segments of 0.2–0.5 s (more in §6 below).

The manifest enumerates every segment URL of every rendition, plus the metadata the player needs to choose between them.

HLS manifest structure

HLS uses two-level .m3u8 text files. The master playlist lists the renditions:

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-STREAM-INF:BANDWIDTH=400000,RESOLUTION=426x240,CODECS="avc1.4d401e,mp4a.40.2"
240p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360,CODECS="avc1.4d401e,mp4a.40.2"
360p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1500000,RESOLUTION=854x480,CODECS="avc1.4d401e,mp4a.40.2"
480p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3000000,RESOLUTION=1280x720,CODECS="avc1.4d401e,mp4a.40.2"
720p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080,CODECS="avc1.4d401e,mp4a.40.2"
1080p/index.m3u8

Walking the lines: #EXTM3U is the file-format magic. EXT-X-STREAM-INF advertises one rendition, with BANDWIDTH (peak bitrate including container overhead, in bits/sec — the player’s main throughput target) and CODECS strings (the RFC 6381 codec identifiers — avc1.4d401e is H.264 Main Profile @ Level 3.0; mp4a.40.2 is AAC-LC). The next line is a relative URL to the media playlist for that rendition.

The media playlist for a single rendition lists segments:

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:0
#EXT-X-MAP:URI="init.mp4"
#EXTINF:6.0,
seg-00001.m4s
#EXTINF:6.0,
seg-00002.m4s
#EXTINF:6.0,
seg-00003.m4s
#EXT-X-ENDLIST

EXT-X-TARGETDURATION is the upper bound on segment length. EXT-X-MAP points to the initialization segment — for fragmented MP4 (CMAF) renditions, this is the ftyp + moov boxes that the player needs once before the first segment. Each EXTINF entry is one segment, with its actual duration (which can be slightly under TARGETDURATION at scene boundaries). EXT-X-ENDLIST signals VOD (Video On Demand); for live streams that line is absent, and the player re-fetches the media playlist periodically to learn about newly published segments.

DASH manifest structure

DASH uses one Media Presentation Description (.mpd) XML file:

<MPD xmlns="urn:mpeg:dash:schema:mpd:2011" type="static" mediaPresentationDuration="PT1800S"
     minBufferTime="PT4S" profiles="urn:mpeg:dash:profile:isoff-on-demand:2011">
  <Period id="0" start="PT0S">
    <AdaptationSet contentType="video" mimeType="video/mp4" segmentAlignment="true">
      <Representation id="240p" bandwidth="400000" width="426" height="240" codecs="avc1.4d401e">
        <BaseURL>240p/</BaseURL>
        <SegmentTemplate timescale="90000" duration="540000"
                         media="seg-$Number$.m4s" initialization="init.mp4" startNumber="1"/>
      </Representation>
      <Representation id="720p" bandwidth="3000000" width="1280" height="720" codecs="avc1.4d401e">
        <BaseURL>720p/</BaseURL>
        <SegmentTemplate timescale="90000" duration="540000"
                         media="seg-$Number$.m4s" initialization="init.mp4" startNumber="1"/>
      </Representation>
    </AdaptationSet>
    <AdaptationSet contentType="audio" lang="en" mimeType="audio/mp4">
      <Representation id="audio-en" bandwidth="128000" codecs="mp4a.40.2">
        <BaseURL>audio-en/</BaseURL>
        <SegmentTemplate timescale="48000" duration="288000"
                         media="seg-$Number$.m4s" initialization="init.mp4" startNumber="1"/>
      </Representation>
    </AdaptationSet>
  </Period>
</MPD>

DASH’s AdaptationSet groups renditions of the same content type (one for video, one per audio language), and Representation is one rendition. SegmentTemplate with the $Number$ placeholder lets the manifest stay short — segment URLs are derived by template instead of enumerated. timescale is the units of duration (90 000 Hz is the conventional video timestamp clock). DASH’s MPD is more flexible than HLS’s M3U8 (it supports multiple periods, ad insertion via MPD chaining, multiple BaseURL entries for redundant CDNs) but the conceptual structure is the same: enumerate renditions, slice into segments, the player picks.

2.3 The player’s adaptation loop

At the heart of every ABR player sits a control loop. Once per segment download, the player decides which rendition to request next.

loop forever:
    measured_bw = exponential_moving_avg(segment_size / segment_download_time)
    buffer_level = total_seconds_of_decoded_video_currently_held
    chosen_tier = select_tier(measured_bw, buffer_level, currently_playing_tier)
    fetch(chosen_tier.next_segment_url)

The body of select_tier is where the published research clusters. The three families:

Rate-based ABR. Pick the highest tier whose BANDWIDTH is below measured_bw × safety_factor (typically 0.7–0.9). Simple, intuitive — the player chases the network’s headroom. Pathological behavior: when the network’s measured throughput swings (a single delayed TCP retransmit can halve the apparent throughput for one segment), the player oscillates, bouncing between tiers every few seconds. This is unwatchable. The early HLS reference player used pure rate-based; users complained about “quality flapping.”

Buffer-based ABR. Pick a tier as a function of buffer occupancy: low buffer → low tier (defensively rebuild the buffer); high buffer → high tier (we have headroom, spend it on quality). This is the BBA family — Buffer-Based Adaptation, introduced in Huang et al.’s 2014 SIGCOMM paper “A buffer-based approach to rate adaptation.” The insight is that the buffer itself is the integral of (download speed − playback speed); you don’t need to estimate bandwidth separately.

Hybrid ABR (BOLA). Spiteri, Urgaonkar & Sitaraman’s BOLA algorithm (INFOCOM 2016, “BOLA: Near-optimal bitrate adaptation for online videos”) frames the problem as Lyapunov optimization, mathematically combining buffer occupancy with utility-of-quality. In practice it picks tier as argmax_tier (V·utility(tier) + buffer_seconds − bitrate(tier)·segment_duration / measured_bw). The V parameter trades off rebuffering risk against quality. BOLA shipped in dash.js (the reference DASH client) in 2016 and is the de-facto modern open-source baseline.

Model-predictive (MPC). Yin et al.’s 2015 SIGCOMM paper (“A control-theoretic approach for dynamic adaptive video streaming over HTTP”) frames bitrate selection as a finite-horizon optimization: at each step, look ahead k segments using a forecast of the bandwidth, and pick the sequence of tiers that maximizes a quality-vs-rebuffer objective. More compute on the client, marginally better QoE in steady state.

Production players (Apple’s iOS HLS, ExoPlayer for Android, Shaka Player for the web, dash.js, hls.js) typically combine elements: a buffer-based core for stability, with a rate-based safety net to react quickly to large drops. Netflix’s “Throughput Predictor” paper (2014) is the classic published industry example.

2.4 Switching mechanics

Switching between renditions happens at segment boundaries. To make that work, every segment must start with an Instantaneous Decoder Refresh (IDR) frame — a self-contained keyframe that doesn’t reference any earlier frame. The player feeds the new rendition’s segment to the decoder, the decoder resets at the IDR, and playback continues seamlessly.

The segment alignment requirement is why all renditions must use the same segment boundaries (segmentAlignment="true" in DASH; CMAF guarantees this with cmfd profile constraints). If the 720p segment 5 covers 30.0–36.0 s and the 480p segment 5 covers 30.5–36.5 s, the player can’t switch mid-segment without a half-second of skip or freeze.

3. Per-Title and Per-Shot Encoding

A naïve ABR ladder uses the same bitrates for every asset. Netflix’s 2015 research (Aaron et al., “Per-Title Encode Optimization”) showed this is wasteful. A talky drama at 720p compresses cleanly at 1 Mbps; a soccer match — full of high-frequency motion that defeats inter-frame prediction — needs 3 Mbps to look the same. Conversely, an animated cartoon may look perfect at 600 kbps. Netflix re-encoded their entire catalog with a per-title bitrate ladder tuned by running the source through several candidate encodes, scoring each with a perceptual metric (initially PSNR, later VMAF — Video Multimethod Assessment Fusion, Netflix’s own perceptual model), and picking the convex hull of (bitrate, quality) points.

Per-title savings are large: Netflix reported 20% bitrate reduction at iso-quality across the catalog. Per-shot encoding (Netflix 2018 follow-up paper) goes further: every shot in the asset is independently optimized, since “shot” boundaries align with scene cuts and within a shot the encoder’s optimal rate-distortion curve is locally consistent.

The price is encoding cost: encoding the same asset at five renditions with VMAF scoring at every step costs 10–50× a baseline encode. For a service amortizing it across millions of streams, the bandwidth savings dominate.

4. CMAF — Eliminating Dual Encoding

Through 2017, every video service had to encode each asset twice: once as MPEG-TS segments for HLS (Apple’s mandate until iOS 10), once as fragmented MP4 segments for DASH. Two storage costs, two encode passes, two manifest pipelines, two CDN URL spaces. CMAF (ISO/IEC 23000-19, finalized 2018) was MPEG and Apple’s joint resolution: a single segment format — fragmented MP4 (.m4s) using ISO-BMFF boxes — that both HLS and DASH manifests can reference.

The mechanics: produce one set of CMAF segments per rendition. Generate one DASH MPD pointing at them, and one HLS m3u8 pointing at the same segments. Apple’s iOS 10+ HLS supports fragmented MP4 segments (it had previously required MPEG-TS). DASH always supported fragmented MP4. The two manifests differ in syntax (XML vs M3U8) and in some metadata (DASH advertises ad insertion via <EventStream>; HLS via EXT-X-DATERANGE), but the byte-stream the player downloads is identical.

Real-world savings. A 4K source encoded at six tiers with audio in three languages has roughly 9 renditions. Pre-CMAF: 18 stored variants (each rendition twice, MPEG-TS and fMP4), 18 cache pools. With CMAF: 9. Storage halves. Origin bandwidth halves. CDN cache fill halves. For Netflix-scale catalogs, this is hundreds of petabytes of storage saved.

CMAF also standardized chunked-transfer-encoded segments, the foundation for low-latency variants discussed below.

5. Live ABR — the Sliding-Window Manifest

For Video On Demand (VOD), the manifest enumerates all segments and is static. For live streams, the manifest is a sliding window of recently-published segments. The encoder is producing segments in real time; the manifest holds, say, the last 30 segments (3 minutes at 6-second segments); each refresh advances by one or more.

The HLS live media playlist:

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:6
#EXT-X-MEDIA-SEQUENCE:1042
#EXTINF:6.0,
seg-01042.m4s
#EXTINF:6.0,
seg-01043.m4s
#EXTINF:6.0,
seg-01044.m4s
#EXTINF:6.0,
seg-01045.m4s

(No EXT-X-ENDLIST — that’s the live signal.) The player polls the media playlist roughly every TARGETDURATION seconds, observes new segments at the bottom and old ones falling off the top, and downloads the new ones.

This polling-and-segments architecture has an irreducible latency floor. A live event happens at wallclock time T. The encoder produces a segment covering [T, T+6]. It can only publish that segment after T+6. The player needs 1–2 segments buffered before it starts playback (to absorb network jitter). End-to-end delay: 12–24 seconds typical for a 6 s segment ladder; it grows worse with longer segments.

6. Low-Latency Variants — LL-HLS and LL-DASH

Latency was tolerable for non-interactive use cases (concerts, news) but unacceptable for live sports betting, auctions, gaming, and especially for Twitch Live Streaming System Design where chat reacts to the stream in real time. Three approaches emerged:

WebRTC for the last mile. Web Real-Time Communication (WebRTC) has sub-second latency by design. But WebRTC’s per-viewer cost is high (each viewer is a peer, not a cache hit), so it only scales by adding Selective Forwarding Units, which look more like conferencing servers than CDN edges. Not a CDN replacement.

Low-Latency DASH (LL-DASH). Specified in DASH-IF Live Profile 2017+. Uses HTTP/1.1 chunked transfer encoding: as the encoder produces frames within a 4 s segment, it streams CMAF chunks (each ~200–500 ms of media) over an open chunked HTTP response. The player fetches the segment with chunked transfer and starts decoding the first chunks before the segment is complete. End-to-end latency drops to 2–4 seconds.

Low-Latency HLS (LL-HLS). Apple introduced this at WWDC 2019, draft RFC draft-pantos-hls-rfc8216bis. Two key additions:

• Partial segments (EXT-X-PART) — sub-second slices of an in-progress segment, advertised in the playlist as soon as they exist.
• Blocking playlist reload (EXT-X-SERVER-CONTROL with CAN-BLOCK-RELOAD=YES) — when the player polls the playlist, the server holds the request until a new partial segment is available, then responds. Eliminates polling overhead and gets the player notified within milliseconds of new content.

Both achieve ~2 s end-to-end latency for live, comparable to broadcast TV. Twitch shipped LL-HLS in 2020; Apple TV+ uses it for live events.

7. Server-Side Ad Insertion (SSAI)

A complication that ABR addressed beautifully — and that earlier protocols did not — is Server-Side Ad Insertion (SSAI), the practice of stitching advertisements into the stream on the server side rather than relying on the client player to fetch them separately. Client-Side Ad Insertion (CSAI), the older approach using the IAB’s VAST/VMAP standards, has the player pause the main content, fetch a different ad video from a different URL (often a different domain), play it, and resume. CSAI is fragile: ad blockers easily detect and skip the second URL, the ad provider’s CDN may have different latency than the content CDN, and the user experiences a visible cutover.

SSAI, by contrast, manipulates the manifest. The ABR origin or a dedicated SSAI service produces a per-user manifest where ad segments and content segments are interleaved into a single playlist with the same encoding parameters, so the player just plays a continuous stream. The HLS manifest looks like:

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-TARGETDURATION:6
#EXTINF:6.0,
content/seg-001.m4s
#EXTINF:6.0,
content/seg-002.m4s
#EXT-X-DISCONTINUITY
#EXTINF:5.0,
ads/userid-12345/seg-001.m4s
#EXTINF:5.0,
ads/userid-12345/seg-002.m4s
#EXT-X-DISCONTINUITY
#EXTINF:6.0,
content/seg-003.m4s

The EXT-X-DISCONTINUITY tag tells the player that the codec parameters or timestamps may discontinue at this segment boundary; the player resets its internal decoder state but keeps the playback session alive. The ad segments must be transcoded to exactly the same codec, resolution, and bitrate ladder as the content for seamless integration — a non-trivial transcode requirement that sustains an entire industry of SSAI vendors (Yospace, AWS MediaTailor, Google Ad Manager DAI, Brightcove SSAI). The DASH equivalent uses <Period> boundaries: each ad break is a separate Period in the MPD, with its own AdaptationSets and Representations matching the content’s.

SSAI’s advantages over CSAI are profound: ad blockers can’t trivially detect the ads (they appear as ordinary segments in the same domain), the per-user manifest enables fine-grained personalization without the player needing to do anything special, and the ad and content share CDN paths so latency is consistent. The disadvantages: ad targeting must happen at manifest-generation time (less dynamic than CSAI’s ability to evaluate ad inventory at the moment of playback), and SSAI infrastructure must scale to the rate of new manifest generations (each user’s join produces a fresh manifest). Hulu, Disney+, Peacock, and Sling TV all use SSAI for their ad-supported tiers.

8. DRM Integration

Premium video must be encrypted. ABR formats support encryption at the segment level via Common Encryption (CENC, ISO/IEC 23001-7): the segments are AES-CTR encrypted with a content key, and the manifest carries pssh boxes (Protection System Specific Header) describing how to obtain the key from a license server.

Three DRM systems dominate, partitioned by browser/OS:

DRM	Vendor	Used by	Required by
Widevine	Google	Chrome, Firefox, Android, smart TVs	Most platforms
FairPlay Streaming	Apple	Safari, iOS, tvOS, macOS	Apple platforms (mandatory)
PlayReady	Microsoft	Edge, Xbox, Windows, smart TVs	Microsoft platforms

A premium service must integrate all three (often via a multi-DRM service like ExpressPlay, Axinom, or Verimatrix). The same encrypted segment can be decrypted by any of them — they share the AES key — but each has its own license-server protocol. CMAF specified a common encryption scheme (cbcs mode) that all three can decrypt, eliminating the previous need to encrypt the same segment three different ways.

9. Real-World Deployments

YouTube uses MPEG-DASH end-to-end. The DASH MPD is generated dynamically by the YouTube serving stack, and the player is YouTube’s own (the standard “html5 video” player wrapping the Shaka Player codebase Google open-sourced in 2015). Per-rendition codecs include H.264, VP9, and AV1; AV1 has been rolled out to high-traffic videos since 2018 for bandwidth savings.

Netflix uses DASH delivery to all platforms except iOS/tvOS (which receive HLS over CMAF). The Open Connect CDN (Content Delivery Network System Design) caches the segments at thousands of ISP-embedded edge boxes. Per-title encoding is the standard since 2015; per-shot since 2018. Codecs: H.264 baseline for legacy devices, HEVC for premium tiers (4K), AV1 progressively rolling out. Netflix’s player ABR is custom-built; the throughput predictor and buffer-based selection are tuned via massive A/B testing.

Twitch uses HLS for both live and VOD. Live streams: ingest is RTMP → transcoded to an HLS ladder → distributed via CDN. Low-Latency HLS is enabled for popular streams; “normal-latency” HLS is the fallback. Twitch’s ABR is constrained by the live nature: latency is non-negotiable for chat-driven engagement, so the segment duration is small (2 s) and the buffer target is small (~5 s).

Apple TV+ uses HLS exclusively (Apple’s vertical-integration play). FairPlay DRM. CMAF-fMP4 segments since 2017. Live events use LL-HLS.

Disney+ uses CMAF with both HLS and DASH manifests pointing to the same segments. Multi-DRM (Widevine, FairPlay, PlayReady).

10. Pitfalls

1. Segment boundaries that don’t align across renditions. Switching tier mid-segment requires aligned boundaries; a rendition with a slightly different keyframe interval produces unwatchable hitches on switch. Encoders enforce closed-GOP encoding with fixed GOP length and keyframe intervals matching segment duration.
2. ABR oscillation. The player flips tiers every segment because the bandwidth estimator over-reacts to single-segment timing noise. Fix: longer averaging windows, hysteresis (require multiple sustained samples before a switch up), buffer-based stabilization. Pure rate-based ABR with EMA window of 1 segment is the textbook way to make this happen.
3. Cold-cache CDN problem. When ABR adaptation shifts a popular live stream’s traffic from tier 720p to tier 1080p (e.g., when most viewers’ networks improve in the early evening), the 1080p segments have cold caches across the CDN. Origin servers see a sudden 10× spike. Mitigation: pre-warm the CDN by issuing internal fetches when a new segment is published; use “tiered caching” within the CDN (Content Delivery Network System Design) so the spike hits regional caches first, not the origin.
4. Audio-video desync after switch. If the audio rendition has a different segment duration than video, switches can drift audio relative to video. CMAF’s segmentAlignment requirement and DASH’s subsegmentAlignment constraint exist to prevent this, but operators routinely violate them and ship busted streams.
5. Live edge instability. The player is at the manifest’s tail (live edge) where segments may be incomplete. Aggressive players that download the latest segment as soon as it appears can race with the encoder and download a partial segment. LL-HLS makes this an explicit feature; non-LL HLS players need a “live edge offset” of 1–2 segments to leave headroom.
6. DRM key rotation breaks players. Some services rotate the content key periodically (every 24 hours); buggy players cache the key past the rotation and silently fail. Test the rotation paths; emit clear “key rotation needed” signals to the player.
7. Manifest size for long-form live. A 24-hour live stream with 2 s segments has 43 200 segments; an HLS playlist enumerating them is 2 MB and grows. Use sliding-window manifests (only recent N segments listed); use DASH SegmentTemplate to avoid enumerating every segment.
8. Many concurrent ABR clients hammering the manifest. Each player polls the live manifest every TARGETDURATION; a million concurrent viewers polling a 6-second manifest = 167K req/sec. Manifest endpoint must be cache-friendly (short TTL but cacheable). LL-HLS’s blocking-reload variant requires uncacheable responses, putting more load on origin — operators run dedicated manifest-distribution boxes.
9. HTTP/1.1 head-of-line blocking on chunked transfer. LL-DASH’s chunked-transfer trick requires HTTP/1.1 (or HTTP/2 with care). On HTTP/2, multiplexing means a slow chunked stream can block other streams; some implementations use HTTP/3 (over QUIC) to sidestep this.
10. Codec licensing pain. H.264 royalties are managed by MPEG-LA; HEVC royalties by three different patent pools (MPEG-LA, HEVC Advance, Velos Media) with overlapping but inconsistent terms. AV1 (Alliance for Open Media, 2018) is royalty-free, the explicit motivation for its existence. Codec choice is as much a legal decision as a technical one.

11. Quality Metrics and the QoE Question

ABR optimization assumes some metric of “quality” the player is trying to maximize. The naïve choice — bitrate — is misleading: 5 Mbps of a flat scene looks identical to 1 Mbps of the same flat scene. The serious choice has been perceptual quality metrics, of which the relevant ones are:

• PSNR (Peak Signal-to-Noise Ratio). The classical metric: 10·log₁₀(MAX² / MSE) where MSE is mean squared error between the decoded frame and the original. Easy to compute, weakly correlated with human perception. The metric every codec paper uses but nobody seriously believes for production decisions.
• SSIM (Structural Similarity Index). Wang et al. 2004. Compares local statistics (mean, variance, covariance) in small windows of the image; better correlated with perception than PSNR but still not great. Range [-1, 1]; ≥ 0.95 is “indistinguishable” by SSIM’s own standards.
• VMAF (Video Multimethod Assessment Fusion). Netflix’s open-source perceptual model, published 2016. Trains an SVM to combine several primitive features (anti-noise SNR, detail-loss measures, motion features) into a 0–100 score where ~93+ is “indistinguishable from source” on a viewer test. Now industry-standard for codec comparison. Per-title encoding (Section 3 above) was driven by VMAF: encoding ladders are tuned to maximize VMAF at each bitrate.

Beyond per-frame quality, the player also optimizes a Quality of Experience (QoE) function that combines:

• Average rendition quality (high is good)
• Rebuffer events (zero is good)
• Quality switches per minute (low is good — switches are annoying even if the average quality is fine)
• Startup latency (low is good)

The published BOLA paper formalizes a specific QoE objective; production players use proprietary ones. Netflix’s QoE function has been described in various tech-blog posts as roughly score = avg_quality − k_rebuffer · rebuffer_seconds − k_switch · num_switches, with the constants tuned via massive A/B testing.

The interaction between rendition quality and QoE is non-trivial. A player that always picks the highest quality the buffer can sustain may cause one rebuffer per hour; a more conservative player may cause zero rebuffers but average 20% lower quality. Which is preferred depends on the user — A/B testing reveals that “no rebuffers, slightly lower quality” is generally preferred for narrative content (drama, documentaries) while “highest quality, occasional rebuffer” wins for high-motion content (sports, action). Production systems split-test these tradeoffs continuously.

12. Worked Example — A 30-Minute Episode

Take a 30-minute streaming TV episode with the 6-tier ladder above. Segment duration: 6 seconds. Total segments per rendition: 300. Total renditions: 6 video + 3 audio = 9. Total CMAF segments stored: 2 700.

Manifest (HLS master):

#EXTM3U
#EXT-X-VERSION:7
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-en",NAME="English",LANGUAGE="en",DEFAULT=YES,AUTOSELECT=YES,URI="audio-en/index.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio-es",NAME="Spanish",LANGUAGE="es",AUTOSELECT=YES,URI="audio-es/index.m3u8"
#EXT-X-STREAM-INF:BANDWIDTH=400000,RESOLUTION=426x240,CODECS="avc1.4d401e,mp4a.40.2",AUDIO="audio-en"
240p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=800000,RESOLUTION=640x360,CODECS="avc1.4d401e,mp4a.40.2",AUDIO="audio-en"
360p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1500000,RESOLUTION=854x480,CODECS="avc1.4d401e,mp4a.40.2",AUDIO="audio-en"
480p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3000000,RESOLUTION=1280x720,CODECS="avc1.4d401e,mp4a.40.2",AUDIO="audio-en"
720p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=5000000,RESOLUTION=1920x1080,CODECS="avc1.4d401e,mp4a.40.2",AUDIO="audio-en"
1080p/index.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=15000000,RESOLUTION=3840x2160,CODECS="hev1.2.4.L150.B0,mp4a.40.2",AUDIO="audio-en"
2160p/index.m3u8

Player decision flow during a bandwidth drop. Suppose the player is happily streaming 1080p (5 Mbps) on a 6 Mbps WiFi connection. At t=120 s, another household member starts a video call and WiFi throughput drops to 2 Mbps. Walking through what happens:

1. t=120 s: player requests segment 21 of 1080p (size ≈ 3.75 MB at 5 Mbps × 6 s).
2. t=120 s to t=125 s: segment 21 download takes 5 s instead of 1 s. Bandwidth EMA drops from 6 Mbps to ~2.5 Mbps after the EMA’s smoothing.
3. t=125 s: player decides next segment. With safety factor 0.8, target bitrate = 2.5 × 0.8 = 2 Mbps. Highest tier ≤ 2 Mbps is 720p (3 Mbps — too high) → fall to 480p (1.5 Mbps).
4. t=125 s to t=126 s: segment 22 of 480p downloads in ~1 s (0.9 MB at 2 Mbps actual). Buffer rebuilds from ~5 s back toward 10 s.
5. t=131 s: bandwidth EMA stabilizes at 2 Mbps. Player picks 480p again. Steady state.
6. t=300 s: video call ends, WiFi recovers to 6 Mbps. Player measures 6 Mbps over a few segments. With hysteresis (require 3 segments above the target before switching up), the player switches back to 1080p at t=320 s.

The visible artifact: a one-segment quality drop at t=126 s, immediately stable at 480p, recovering after the bandwidth has been good for 18 s. No rebuffer; no oscillation. This is the ABR happy path.

What goes wrong without buffer-based stabilization. Pure rate-based without hysteresis: the player toggles 480p / 720p / 480p every other segment as the EMA wobbles around 2.5 Mbps. The viewer sees visible quality flicker every 6 s. BOLA or BBA-style buffer-based selection prevents this by anchoring on buffer health.

13. See Also

• YouTube Video Streaming System Design — DASH at YouTube scale; CDN integration
• Netflix Streaming System Design — Open Connect CDN; per-title encoding details
• Twitch Live Streaming System Design — LL-HLS for live; RTMP ingest → HLS egress
• Real-Time Messaging Protocol — the ingest counterpart that feeds ABR transcoders
• Web Real-Time Communication — sub-second latency alternative to ABR
• NAT Traversal — relevant for WebRTC alternatives; not directly for ABR
• Content Delivery Network System Design — caching, edge selection, tiered caching for ABR segments
• WebSocket Protocol — used as the LL-HLS playlist update transport in some clients
• Video Conferencing System Design — uses simulcast (SFU-side ABR) instead of HTTP ABR; cross-link the two adaptation concepts
• Major System Designs MOC
• SWE Interview Preparation MOC