vault

TCP Flow Control and the Sliding Window

19 min read

TCP Flow Control and the Sliding Window

Flow control is the mechanism by which a Transmission Control Protocol (TCP) receiver stops a fast sender from overrunning its buffer: in every acknowledgement (ACK) it sends, the receiver advertises a receive window — a 16-bit count of “how many more bytes I have room for right now” — and the sender is forbidden from having more than that many bytes unacknowledged and in flight. This advertised window slides along the byte stream as data is acknowledged and drained, which is why the whole scheme is the sliding-window protocol. Crucially, this is a control loop about the receiver’s buffer only — it is not congestion control, which is a separate, sender-side loop about the network. The most recent consolidated specification is RFC 9293 (August 2022, Internet Standard STD 7), which obsoletes the original RFC 793 of 1981 and folds in the host requirements of RFC 1122 (RFC 9293 §3.4, §3.8.6).

Uncertain

Verify: the exact fractional threshold Fs and the precise multi-branch form of the sender-side Silly Window Syndrome (SWS) avoidance algorithm (§ Silly Window Syndrome and Its Avoidance below). Reason: RFC 9293 §3.8.6.2.1 was consulted via a summarizing fetch rather than read glyph-by-glyph, and the classic value Fs = 1/2 comes from RFC 1122 §4.2.3.4, whose full §4.2.3 text could not be retrieved verbatim in this session (the page truncated before §4.2). To resolve: read RFC 9293 §3.8.6.2.1 and RFC 1122 §4.2.3.4 directly. The receiver-side rule (advertise no growth until min(MSS, ½·buffer) — SHLD-15) and the shrink prohibition (MUST-31) and zero-window probe rule (MUST-32) were retrieved verbatim and are solid. #uncertain

This note is the protocol / RFC view of flow control: the sliding-window state variables, the on-the-wire 16-bit window field, the window-scale option that rescues it, zero-window probing and the persist timer as specified, and Silly Window Syndrome avoidance as specified. It is the deliberate counterpart to TCP Send and Receive Windows, which owns the Linux kernel mechanism — __tcp_select_window(), receive-buffer autotuning, SO_RCVBUF, the ss diagnostics, and the pinned 6.12 source paths. Where this note says “the receiver computes a window,” the kernel note shows the C that computes it; read them together, but the RFC-level algorithm and the design rationale live here. The other window — the congestion window (cwnd) — is the subject of Congestion Control Fundamentals and AIMD; the single most important idea in this whole area is that rwnd and cwnd are different windows solving different problems, developed at length below.

Mental Model — A Window That Slides Over the Byte Stream

TCP numbers every byte of the stream with a sequence number, and both endpoints keep a small set of counters that carve the sequence space into regions (see TCP Sequence Numbers and Acknowledgements for how the numbers themselves work). On the sender, three variables matter (RFC 9293 §3.3.1, Table 2): SND.UNA (“send unacknowledged”) is the oldest byte the sender has transmitted but not yet had ACKed; SND.NXT (“send next”) is the next byte the sender will hand to the network; and SND.WND (“send window”) is the most recent window value the receiver advertised. On the receiver, RCV.NXT (“receive next”) is the next byte it expects, and RCV.WND (“receive window”) is how much beyond that it is willing to accept (RFC 9293 §3.3.1, Table 3).

The window is the span [SND.UNA, SND.UNA + SND.WND) of the byte stream. It divides into four regions: bytes already sent and acknowledged (to the left of SND.UNA), bytes sent but not yet acknowledged (SND.UNA up to SND.NXT — the data “in flight”), bytes the sender may send immediately (SND.NXT up to SND.UNA + SND.WND — the usable window), and bytes it may not send yet (beyond the right edge). As ACKs arrive, SND.UNA advances and the whole window slides rightward, uncovering fresh bytes to transmit. That sliding is the engine of TCP’s pipelining: the sender does not wait for each byte to be acknowledged before sending the next; it keeps up to a full window in flight and lets acknowledgements clock new data out — the property Nagle later called TCP’s “self-clocking” behaviour (RFC 896).

flowchart LR
  subgraph SEQ["Sender's view of the sequence space"]
    R1["① sent + ACKed<br/>(&lt; SND.UNA)"]
    R2["② sent, not yet ACKed<br/>in flight<br/>(SND.UNA .. SND.NXT)"]
    R3["③ usable window<br/>may send now<br/>(SND.NXT .. SND.UNA+SND.WND)"]
    R4["④ not yet allowed<br/>(&ge; SND.UNA+SND.WND)"]
    R1 --> R2 --> R3 --> R4
  end
  ACK["ACK arrives:<br/>advances SND.UNA<br/>→ window slides right →"] -.-> R2

The sliding window over the sender’s byte-numbered stream. What it shows: the receiver’s advertised window (SND.WND) sets the right edge; SND.NXT splits the window into data already in flight (region ②) and data the sender may still inject immediately (region ③, the usable window U = SND.UNA + SND.WND − SND.NXT). The insight to take: flow control is entirely about how far right the receiver lets the right edge go — a small SND.WND shrinks region ③ to nothing and stalls the sender even though the network is idle. The receiver moves that edge; the sender only obeys it.

The Sliding-Window Protocol, Mechanically

The protocol is a tight feedback loop, stated crisply in RFC 9293 §3.8: “When the sender creates a segment and transmits it, the sender advances SND.NXT. When the receiver accepts a segment, it advances RCV.NXT and sends an acknowledgment. When the data sender receives an acknowledgment, it advances SND.UNA” (RFC 9293). Every ACK carries two numbers that update the sender’s picture: the acknowledgement number (which byte the receiver next expects, moving SND.UNA) and the window field (how much room, moving the right edge via SND.WND).

The window field lives in the TCP header and is, per RFC 9293 §3.1, “the number of data octets beginning with the one indicated in the acknowledgment field that the sender of this segment is willing to accept.” Two subtleties are baked into the spec. First, “The window size MUST be treated as an unsigned number, or else large window sizes will appear like negative windows and TCP will not work (MUST-1)” — a signed-vs-unsigned bug here silently caps or corrupts throughput. Second, the header field is only 16 bits wide, so a raw window can never exceed 65 535 octets; the fix (window scaling) is a whole section below. RFC 9293 recommends that implementations keep 32-bit fields internally and “do all window computations with 32 bits (REC-1).”

A receiver defines exactly which incoming segments are acceptable in terms of its window (RFC 9293 §3.4): a segment’s sequence range must fall inside RCV.NXT ≤ SEG.SEQ < RCV.NXT + RCV.WND. Anything to the left is a duplicate (already received) and anything to the right of the window is dropped as out-of-buffer. There is one important exception the spec calls out: “when the receive window is zero no segments should be acceptable except ACK segments,” yet “A TCP receiver MUST process the RST and URG fields of all incoming segments, even when the receive window is zero (MUST-66)” — a zero window closes the data channel but never the control channel.

The window must never shrink

An easy design mistake is to let a receiver reduce its advertised window by moving the right edge left — for instance because it retroactively decided to reserve buffer. RFC 9293 forbids this: “A host MUST NOT shrink the window, i.e., move the right edge of the window to the left (MUST-31).” The reason is robustness: the sender may already have transmitted data that was inside the old, larger window; pulling the edge left would place bytes that are legitimately in flight outside the current window, where a strict receiver would discard them. A receiver that needs to advertise less space must instead let the edge stand and simply not advance it as data is acknowledged — the window narrows from the left as RCV.NXT catches up, which is safe.

Flow Control Is Not Congestion Control

This is the single most-confused point in TCP, and the reason the task brief insists on making it explicit. A TCP sender is governed by two independent limits, and the amount it may have outstanding is the smaller of the two:

in-flight bytes ≤ min(rwnd, cwnd)

The receive window (rwnd, the SND.WND the peer advertises) is set by the receiver and protects the receiver’s buffer. It is a flow-control signal — “don’t overrun me.” It is carried explicitly on the wire in every ACK. The congestion window (cwnd) is set by the sender’s own congestion-control algorithm and protects the network — “don’t overrun the path and cause packet loss.” It is never transmitted; it is purely sender-internal state inferred from ACK arrivals and losses. RFC 9293 keeps them cleanly separated: flow control is specified in §3.3–§3.4 and §3.8.6, while “A TCP endpoint MUST implement the basic congestion control algorithms slow start, congestion avoidance, and exponential backoff of RTO … (MUST-19)” points out to the congestion-control RFCs (RFC 5681 and successors) as a distinct concern (RFC 9293 §3.8.1–§3.8.2).

The two windows fail in different ways and are diagnosed differently. If rwnd is the binding limit, the receiving application is not calling recv() fast enough (or its buffer is too small) — the fix is on the receiver. If cwnd is the binding limit, the network is congested — the fix is a congestion-control story. A connection can be flow-control-limited with an idle network (rwnd tiny, cwnd huge), congestion-limited with a bored receiver (cwnd tiny, rwnd huge), or limited by neither. The theory of the cwnd half — additive-increase/multiplicative-decrease (AIMD), slow start, the sawtooth — is developed in Congestion Control Fundamentals and AIMD; the Linux implementation of both windows and the min(cwnd, rwnd) gate is in TCP Send and Receive Windows. This note owns only the receiver-protecting half.

Window Scaling — Rescuing the 16-Bit Field

The 16-bit window field was generous in 1981 and absurd today. A connection’s ceiling throughput is roughly its window divided by its round-trip time (RTT): to keep a pipe full you must have in flight one bandwidth-delay product (BDP) of data, where BDP = bandwidth × RTT. With a 64 KiB window on a 100 ms transcontinental path, throughput ≤ 65 535 bytes / 0.1 s ≈ 5.2 Mbit/s — hopeless on any modern link, regardless of how fat the pipe is. RFC 7323, TCP Extensions for High Performance (obsoleting the older RFC 1323), fixes this without widening the header, via the Window Scale option (RFC 7323 §2).

The option is three bytes: Kind = 3, Length = 3, and a one-byte shift.cnt (RFC 7323 §2.2). It “define[s] an implicit scale factor, which is used to multiply the window size value found in a TCP header to obtain the true window size.” Concretely, once a scale of S is in effect for a direction, every incoming window field is left-shifted before use: “SND.WND = SEG.WND << Snd.Wind.Shift” (RFC 7323 §2.3). A shift of 7 multiplies the advertised value by 128, lifting the 64 KiB ceiling to 8 MiB; the maximum shift is capped.

Three rules make window scaling behave. (1) SYN-only negotiation. The option “MAY be sent in an initial <SYN> segment,” and only if one was received may it be echoed in the <SYN,ACK>; “A Window Scale option in a segment without a SYN bit MUST be ignored” (RFC 7323 §2.2). The scale factor is therefore fixed for the life of the connection at handshake time — it cannot be renegotiated later. (2) Bilateral. “This option is an offer, not a promise; both sides MUST send Window Scale options in their <SYN> segments to enable window scaling in either direction” (RFC 7323 §2.2). If either endpoint omits it — or a middlebox strips it — scaling silently does not activate, and the connection is pinned to 64 KiB for its entire life. This is a classic, maddening cause of throughput that inexplicably tops out at a few Mbit/s on a long path. (3) The shift is capped at 14. “the shift count MUST be limited to 14 (which allows windows of 2^30 = 1 GiB)” (RFC 7323 §2.3). The cap exists to keep the window strictly less than half the 32-bit sequence space — “two times the maximum window size must be less than 2^31” — so that old duplicate segments from the wrapped sequence space can never be mistaken for in-window data. The maximum representable window is therefore about 1 GiB.

Zero Window and the Persist Timer

When the receiving application stops reading and the receive buffer fills, the receiver advertises a zero window: “stop, I have no room.” The sender halts. This creates a deadlock hazard, because the eventual window re-opening is announced by an ordinary ACK — and ACKs are not retransmitted. If that window-update ACK is lost, the sender waits forever for permission that already came and went, while the receiver waits forever for data that will never be sent.

TCP breaks the deadlock with the persist timer and zero-window probing, specified in RFC 9293 §3.8.6.1: “When the receive window is zero, the sending TCP peer MUST send acknowledgment segments periodically, or send a segment containing one octet of old data if the receiver has already seen this octet but the receiver’s advertised window prevents acceptance (MUST-32).” The probe is a single octet of already-sent data (so it is harmless if the receiver still cannot accept it), and it forces the receiver to reply with a fresh ACK carrying its current window. The probe interval uses exponential backoff — it doubles with each unanswered probe — so a persistently stalled connection generates probes at an ever-slower cadence rather than hammering the link. The vital robustness property: the connection persists as long as the receiver keeps answering the probes, even if every answer is another zero-window ACK. Only unanswered probes eventually tear the connection down. This is exactly why a healthy-but-buffer-stalled peer can legitimately sit in the zero-window state for a very long time without the connection dying — the design goal is deadlock-freedom, not liveness pressure on a slow reader. (The Linux realisation of this — tcp_send_probe0(), icsk_probes_out, the TCPWinProbe counter, and the interaction with tcp_retries2 / TCP_USER_TIMEOUT — is detailed in TCP Send and Receive Windows.)

Silly Window Syndrome and Its Avoidance

Silly Window Syndrome (SWS) is the pathology in which a TCP connection degenerates into shipping tiny segments forever, drowning in header overhead. David Clark named and analysed it in RFC 813 (1982). It arises from a self-perpetuating cycle: “whenever the acknowledgement of a small segment comes back, the useable window associated with that acknowledgement will cause another segment of the same small size to be sent” (RFC 813). Once the usable window has been fragmented into small pieces — because a natural data boundary caused the sender to split it — Clark observes there is “no natural way for those useable window allocations to be recombined; thus the breaking up of the useable window into small pieces will persist.” A single byte of data then rides in a segment with 40 bytes of header (20 IP + 20 TCP), a 4000 % overhead in the degenerate limit — the same “tinygram” waste Nagle attacked from the sender side (RFC 896).

Both endpoints defend against SWS, and the two defences are deliberately split across two notes to keep one idea per note:

Receiver-side SWS avoidance is a flow-control mechanism and belongs here. The receiver refuses to advertise a small new window increment; it holds the window artificially closed until it can offer a worthwhile chunk. Clark’s original rule (RFC 813) is to keep the window shut until it can be re-opened by “one half of the available space,” then advertise the whole space at once. RFC 9293 §3.8.6.2.2 codifies the modern form: the receiver should not advertise a larger window “until it can increase by at least min(MSS, half the receive buffer) (SHLD-15)” — where MSS is the Maximum Segment Size, the largest payload the connection carries in one segment. In practice this means a receiver that has drained only a few bytes advertises no growth (often continuing to advertise a small or zero window) until it can hand back at least a full segment’s worth of room, so the sender is never invited to emit a tinygram. This is the same defensive logic visible in Linux’s __tcp_select_window() returning zero rather than a sub-MSS window (see TCP Send and Receive Windows).

Sender-side SWS avoidance is Nagle’s algorithm, and it is the subject of its own note, Nagle’s Algorithm and Delayed Acknowledgement — cross-linked rather than duplicated here. The sender withholds a small segment while unacknowledged small data is already outstanding, coalescing further small writes until either a full segment accumulates or the outstanding data is ACKed. RFC 9293 §3.8.6.2.1 frames the sender’s decision as: send now if a maximum-sized segment can go, or if all queued (pushed) data can be sent because nothing is outstanding, or if at least a fraction Fs of the maximum window can be sent, or if an override timeout of roughly 0.1–1.0 seconds fires so that data is never buffered indefinitely (the classic value is Fs = 1/2; see the uncertainty flag at the top for why the exact modern text should be re-verified). The override timeout is what guarantees liveness: even a sender with only a trickle of data eventually flushes it rather than waiting forever for a window that never grows.

Failure Modes and Common Misunderstandings

  • Confusing rwnd with cwnd. The perennial error. “TCP slowed down” could be flow control (receiver’s rwnd) or congestion control (sender’s cwnd), and they have opposite fixes. If the peer’s advertised window is small, the receiving application is the bottleneck; if the sender’s cwnd is collapsing in a sawtooth, the network is. See Congestion Control Fundamentals and AIMD for the latter.
  • Throughput mysteriously capped near 5 Mbit/s on a long-RTT path. Almost always a stripped or missing Window Scale option — a middlebox removed the SYN option, or one side did not offer it, pinning the window at 64 KiB for the connection’s life. Because scaling is negotiated only in the SYN, there is no recovery; the connection must be re-established through a path that preserves the option.
  • Assuming a zero window means the connection is broken. A zero window is normal back-pressure. As long as probes are answered, the connection is healthy and simply throttled by a slow reader, not by loss. A steady stream of window probes points at the receiving application, not the network.
  • Believing the receiver can take back window it already granted. MUST-31 forbids shrinking the window (moving the right edge left). Buffer pressure is expressed by not advancing the edge, letting the window narrow from the left as data is acknowledged — never by retracting it.
  • Advertising sub-MSS windows and inducing tinygrams. A naive receiver that advertises every byte it drains re-creates Silly Window Syndrome. Receiver-side SWS avoidance (SHLD-15) exists precisely to suppress small window increments; a receiver that “helpfully” opens the window one byte at a time is the bug.
  • Treating the 16-bit window as the real limit. With scaling in effect the on-the-wire field is true_window >> scale; reading the raw field without applying the negotiated shift makes a 64 MiB window look like a few hundred bytes. Any tool or parser must apply the SYN-negotiated scale.

Alternatives and Boundaries

Flow control is not optional and has no real “alternative” within TCP — every conformant implementation runs the sliding window. What varies is how the receiver sizes the buffer that backs rwnd: a fixed hand-set buffer (SO_RCVBUF, which on Linux disables autotuning) versus kernel receive-buffer autotuning that grows the window to the path’s BDP automatically. That sizing decision, and the ss diagnostics for observing which window is binding, are owned by TCP Send and Receive Windows and deliberately not repeated here. At the transport-choice level, protocols that do not want TCP’s stream-and-window model at all — UDP, and QUIC’s per-stream flow control layered on UDP — are the subject of The User Datagram Protocol and QUIC Transport Protocol; QUIC notably re-implements a sliding-window flow-control scheme of its own, both per-stream and connection-wide, precisely because the idea is so fundamental that escaping TCP does not escape the need for it.

Production Notes

The BDP arithmetic is the number every performance engineer eventually memorises: to fill a 10 Gbit/s link at 30 ms RTT you need 10e9 bits/s × 0.03 s / 8 ≈ 37.5 MB of window — far beyond 64 KiB, which is why window scaling is mandatory at scale and why receive buffers on high-throughput hosts must be allowed to grow into the tens of megabytes. Operators of high-throughput services (Cloudflare’s TCP-tuning write-up is a good public example, cited in TCP Send and Receive Windows) raise the autotuning ceiling rather than pinning buffers, keeping the adaptivity while removing the cap. On the diagnostic side, wscale:X,Y in ss -i reports the negotiated send/receive shift factors; a wscale:0,0 on a high-RTT path stuck near 64 KiB is the fingerprint of a stripped option. A stream of TcpExtTCPWinProbe in nstat is the fingerprint of a persistently zero-windowed peer — a receiving application that has stopped reading. Both diagnostics are protocol-visible symptoms of the specification described in this note.

See Also

Graph View

Backlinks