vault

Bufferbloat and Active Queue Management

12 min read

Bufferbloat and Active Queue Management

Bufferbloat is the pathology in which oversized, unmanaged network buffers fill up and stay full, converting a link’s spare capacity into crippling, persistent latency. It arises from a toxic pairing: memory got cheap, so router and modem vendors fitted huge buffers “so packets are never dropped,” while the dominant loss-based congestion controllers (CUBIC, Reno) are designed to keep sending faster until a packet drops — so they inflate the queue until it overflows, and a big buffer means a big, standing queue that adds hundreds of milliseconds of delay to every packet behind it (Gettys & Nichols, “Bufferbloat: Dark Buffers in the Internet,” CACM 2011). The cure is Active Queue Management (AQM): instead of dropping only when the buffer is completely full (tail-drop), an AQM algorithm signals congestion early — by dropping or ECN-marking a few packets — to keep the queue short. Crucially, modern AQM controls delay, not queue length (RFC 8289, CoDel). This note owns the concept and the algorithms (tail-drop’s failures, RED, CoDel, FQ-CoDel, PIE, CAKE, ECN); the Linux implementation — how BQL and the qdisc layer wire these into the kernel — lives in Byte Queue Limits and Buffer Bloat.

Mental Model: Good Queue vs Bad Queue

A queue is not inherently bad. Networks are bursty, and a buffer that absorbs a transient burst and drains it within a round-trip time is doing its job — it keeps the bottleneck link busy that it would otherwise leave idle. RFC 8289 names this a “good queue.” The disease is the “bad queue”: a standing queue that never drains, present at all times, contributing only delay and no throughput.

RFC 8289’s worked example makes the distinction crisp. Take a TCP flow whose window is 25 packets over a path that can hold 20 packets in flight (its bandwidth-delay product — see Latency Bandwidth and the Bandwidth-Delay Product). At all times 25 packets are “in flight,” but only 20 fit in the pipe, so 5 packets are always sitting in the bottleneck buffer. Those 5 packets add pure delay: the link is already 100% utilized by the 20 in-transit packets, so the standing 5 buy nothing. That 5-packet standing queue is bufferbloat in miniature — and with a modem buffer sized for thousands of packets, the standing queue can reach seconds.

flowchart TB
  subgraph GOOD["Good queue - transient"]
    G1["burst arrives"] --> G2["buffer briefly fills"] --> G3["drains within 1 RTT<br/>keeps link busy, low delay"]
  end
  subgraph BAD["Bad queue - standing / bufferbloat"]
    B1["loss-based sender<br/>keeps growing cwnd"] --> B2["huge buffer never drops"] --> B3["queue fills and STAYS full<br/>seconds of delay for everyone"]
  end
  GOOD -.->|"AQM keeps it here"| BAD

Good queue versus bad queue. What it shows: the same buffer is healthy when it absorbs a burst and empties, and diseased when a loss-based controller inflates it into a permanent backlog. The insight: the useful signal is not how many packets are queued (that varies with link rate and sampling instant) but how long a packet waits in the queue — the sojourn time. AQM’s whole job is to hold sojourn time near a small target so queues stay in the “good” regime.

Why Tail-Drop Fails

The default, do-nothing policy is tail-drop: accept packets until the buffer is full, then drop new arrivals. RFC 7567 (“IETF Recommendations Regarding Active Queue Management,” July 2015, obsoleting RFC 2309) enumerates why this is actively harmful:

  • Full queues. Tail-drop only signals congestion when the buffer is already full, which means it maintains maximum queue depth — and maximum delay — as its steady state. It optimizes for exactly the wrong thing.
  • Lock-out. Through timing quirks, a few flows can monopolize the buffer and lock others out entirely, starving them.
  • Bursts. A burst arriving at a near-full buffer gets mass-dropped, wrecking the TCP ACK clock of several flows at once.
  • Global synchronization. When the buffer overflows, many flows lose packets simultaneously, all back off together, all ramp up together, and the link oscillates between congested and idle — sawtooth utilization with high jitter.

AQM attacks all four by keeping the queue short in the common case, dropping/marking a few packets early and spread out rather than many late and synchronized.

RED: The First Attempt, and Why It Was Abandoned as a Default

Random Early Detection (RED) (Floyd & Jacobson, 1993) was the first widely-specified AQM. It maintains an exponentially-weighted moving average queue length and, once that average crosses a minimum threshold, drops (or marks) arriving packets with a probability that rises linearly toward a maximum threshold. By dropping early and randomly, RED avoids the full-queue and global-synchronization problems in principle.

In practice RED was hard to deploy because its behavior depends on queue length in packets, a quantity whose meaning shifts with link rate and buffer size. Getting it to work required hand-tuning four coupled knobs — min_th, max_th, max_p, and the averaging weight — per link, and a mistuned RED often performed no better than tail-drop. RFC 2309 (1998) had recommended deploying RED by default; RFC 7567 explicitly retracts that recommendation, calling instead for self-tuning algorithms that need no operator configuration (RFC 7567). That retraction is the pivot from length-based to delay-based AQM.

CoDel: Controlling Delay, Not Length

CoDel (“Controlled Delay,” pronounced “coddle”) by Kathleen Nichols and Van Jacobson (ACM Queue, May 2012; standardized as RFC 8289, Jan 2018) is the algorithm that made AQM finally deployable, and its central move is conceptual: measure the time each packet spends in the queue (its sojourn time), not the queue’s length. Sojourn time is the right variable because it is what the user actually experiences and because it is invariant to link rate — a 5 ms delay is 5 ms whether the link is 1 Mbps or 1 Gbps, whereas “50 packets queued” means wildly different delays at different rates.

CoDel is governed by two constants, both with physical justification and neither needing tuning:

  • TARGET = 5 ms — the maximum acceptable standing queue delay. It is derived from Kleinrock’s “power” metric (throughput divided by delay), which is optimized when the standing queue sits at roughly 5–10% of the connection’s RTT; 5 ms is ~5% of the ~100 ms terrestrial-Internet RTT ceiling, high enough to keep the link busy, low enough to feel instant.
  • INTERVAL = 100 ms — the observation window, chosen to be “at least a round-trip time, and not much more,” so CoDel distinguishes a persistent standing queue from a transient burst without over-reacting to normal RTT-scale variation.

The algorithm, executed only at dequeue (which keeps the enqueue and dequeue paths lock-free), tracks the minimum sojourn time observed over each INTERVAL. Using the minimum is the trick that ignores bursts: a burst raises the maximum and average sojourn, but as long as the queue fully drains at least once in the interval, the minimum touches zero and CoDel does nothing. Only when the minimum stays above TARGET for a whole INTERVAL does CoDel conclude the queue is standing and enter the dropping state, dropping one packet and scheduling the next drop at:

next_drop = now + INTERVAL / sqrt(count)

where count is the number of drops so far in this dropping episode. The inverse-square-root schedule is not arbitrary: TCP throughput is proportional to 1/sqrt(p) for drop probability p, so spacing drops by 1/sqrt(count) makes the delay respond linearly to the control effort, gently ramping the drop rate until the queue relents rather than over-dropping and starving the link. CoDel also refuses to drop when the buffer holds less than one MTU, guaranteeing it never starves the output. The result is no knobs: the same constants work from 64 Kbps DSL to 100 Mbps cable (RFC 8289).

FQ-CoDel: Adding Fairness

CoDel controls the depth of a single queue but does nothing about fairness between flows — one bulk download can still crowd out a latency-sensitive DNS lookup sharing the queue. FQ-CoDel (“Flow Queue CoDel,” RFC 8290, Jan 2018) fixes this by combining CoDel with fair queueing:

  • Packets are hashed by their 5-tuple (source/destination IP, source/destination port, protocol) into 1024 sub-queues by default, with a random salt to defeat deliberate hash-collision attacks. CoDel runs independently on each sub-queue.
  • A Deficit Round Robin (DRR) scheduler visits the sub-queues, granting each a quantum (default 1514 bytes) of transmission credit per round, so flows share bandwidth fairly regardless of packet size.
  • FQ-CoDel splits sub-queues into “new” and “old” lists and gives new queues scheduling priority. A flow that sends only occasionally — a DNS query, a TCP ACK, an interactive SSH keystroke, a VoIP packet — never builds a backlog, stays “new,” and is dispatched almost immediately, ahead of any bulk flow. This gives sparse, latency-sensitive traffic near-zero queuing delay without any classification or configuration — arguably FQ-CoDel’s most important practical property.

FQ-CoDel shipped in Linux 3.5 (2012) and is the default queueing discipline on most modern Linux distributions (systemd sets net.core.default_qdisc = fq_codel) and on OpenWrt home routers (RFC 8290).

PIE, CAKE, and the Rest of the Family

  • PIE (“Proportional Integral controller Enhanced,” RFC 8033, Feb 2017) targets a delay setpoint like CoDel but computes a drop probability with a proportional-integral controller driven by the current delay estimate and its rate of change, dropping at enqueue. PIE was adopted by DOCSIS 3.1, so it is the AQM running in a large fraction of the world’s cable modems.
  • CAKE (“Common Applications Kept Enhanced”) is the bufferbloat project’s all-in-one successor to FQ-CoDel, upstreamed in Linux 4.19 (2018) and documented in the “Piece of CAKE” paper rather than a standards-track RFC (bufferbloat.net). CAKE folds several jobs into one qdisc: an integral-controller traffic shaper (so it can move the queue off an uncontrollable downstream device — see failure modes), 8-way set-associative flow hashing (fewer collisions than FQ-CoDel’s plain hash), combined per-host and per-flow fairness, DiffServ-aware tin prioritization, ACK filtering, and framing compensation for DSL/ATM/PPPoE overhead. It is the recommended qdisc for a home gateway’s uplink.

Uncertain

Verify: that CAKE has no standards-track RFC as of 2026 (only the “Piece of CAKE” paper and an expired Internet-Draft). Reason: the bufferbloat.net wiki confirms it is upstream since Linux 4.19 and documented in a paper, but does not assert RFC status either way; standards status can change. To resolve: check the IETF datatracker for a draft-*-cake that reached RFC. #uncertain

ECN: Mark Instead of Drop

Dropping a packet is a blunt congestion signal — it costs a retransmission and a round trip. Explicit Congestion Notification (ECN, RFC 3168, 2001) lets an AQM instead mark a bit in the IP header (setting the CE, “Congestion Experienced,” codepoint) on an ECN-capable packet; the receiver echoes the mark back to the sender, which slows down exactly as if it had seen a loss — but without losing the packet. RFC 7567 recommends that deployed AQMs mark ECN-capable traffic rather than drop it whenever they can. ECN is the substrate for L4S (Low Latency, Low Loss, Scalable throughput), a newer effort to drive queuing delay down to sub-millisecond levels using more aggressive, high-fidelity ECN marking. Marking is strictly better than dropping when both ends support it; its slow historical uptake was due to a few broken middleboxes that mangled the ECN bits.

How AQM and BBR Relate

There are two independent places to fight bufferbloat: in the network (AQM at the bottleneck queue) and at the sender (a congestion controller that refuses to fill the queue in the first place). BBR Congestion Control takes the sender-side route: rather than probing for bandwidth by inducing loss (which requires filling the buffer), BBR continuously estimates the bottleneck bandwidth (BtlBw) and the round-trip propagation time (RTprop, the RTT with no queuing), and paces itself to keep the pipe exactly full — one BDP in flight, zero standing queue — so it sidesteps bufferbloat even across a dumb tail-drop buffer. AQM and BBR are complementary, not rivals: AQM protects a link from the many loss-based flows still in the wild, while BBR protects a single flow from bloating a link that lacks AQM.

Failure Modes and Diagnosis

  • The bottleneck buffer is one you don’t control. AQM only works on the queue you manage. Home bufferbloat usually lives in the ISP’s modem or the DSLAM, downstream of your router, where your qdisc never runs. The workaround is shaping: rate-limit your uplink (with CAKE or HTB+fq_codel) to just below the real link rate, so the queue forms in your device — where AQM can manage it — instead of in the uncontrollable modem. Setting the shaped rate too high leaves the queue in the modem; too low wastes bandwidth.
  • WiFi aggregation defeats per-packet AQM. 802.11n/ac aggregate many frames into one airtime transmission, so treating packets individually mismanages the real bottleneck (airtime). This drove the separate “make-wifi-fast” work and airtime-fair queueing in the Linux WiFi stack.
  • Diagnosing bufferbloat. The signature is latency that spikes only under load: idle ping is 20 ms, but start an upload and ping jumps to 500+ ms. The flent RRUL test and the “waveform bufferbloat test” / dslreports speed-plus-latency tests quantify it as a letter grade. If idle latency is fine but loaded latency is terrible, you have bufferbloat, and the fix is AQM plus shaping — not more bandwidth.
  • Mistuned RED making things worse. A RED instance with thresholds set for the wrong link rate can drop too early (throughput loss) or too late (still bloated). This is precisely why the self-tuning CoDel family displaced it.

See Also

Graph View

Backlinks