vault

Traffic Control Overview

19 min read

Traffic Control Overview

Traffic control (universally abbreviated tc, after the userspace tool of the same name) is the Linux kernel subsystem that decides in what order, and how fast, packets leave a network interface — and, in a more limited way, what happens to packets as they arrive. It is the layer that sits between the IP/transport stack and the device driver on the transmit (TX) path: once the stack has built a packet and picked an output queue, the packet is enqueued onto a queueing discipline (qdisc) attached to that interface, and the qdisc — not the driver — is what shapes, schedules, prioritizes, polices, or drops it (tc(8); __dev_queue_xmit in net/core/dev.c, v6.12). Everything tc does is built from three primitives — qdiscs, classes, and filters — and the whole subsystem exists to solve one practical problem: a dumb first-in-first-out buffer in front of a bottleneck link fills up and adds enormous latency (bufferbloat), so the kernel needs a smarter place to hold packets where it can apply policy.

This is the orientation note for the traffic-control subsystem. It frames where tc sits, what it does, and how its pieces fit together; the mechanism of an individual qdisc is in Queueing Disciplines qdisc, the classful/classless distinction is in Classful and Classless Qdiscs, and the userspace tc command and its filter language are in The tc Tool and Filters. All code references are pinned to Linux 6.12 LTS (released 2024-11-17) and spot-checked against 6.18 LTS (2025-11-30); the core structures and the TX-path call sites are identical across both.

Mental Model — A Programmable Gate on the Egress Path

The right way to think about traffic control is as a configurable gate between the protocol stack and the wire. When an application calls send(), the data travels down through TCP/UDP and IP (see The Network Transmit Path), and the result is a struct sk_buff (the kernel’s packet buffer — see struct sk_buff) that the stack hands to the transmit path. At that hand-off, the kernel does not immediately push the packet to the hardware. Instead it enqueues the skb onto the interface’s root qdisc, then asks the qdisc to dequeue whatever it thinks should go next. The qdisc is therefore a policy object: a plain FIFO returns packets in arrival order, a priority qdisc returns high-priority packets first, a shaping qdisc holds packets back until a token bucket says a slot is free, and a fair-queueing qdisc interleaves competing flows so one bulk transfer cannot starve an interactive one.

flowchart TB
  APP["Application<br/>send()"] --> STACK["TCP/UDP + IP<br/>build sk_buff"]
  STACK --> DQX["__dev_queue_xmit()<br/>net/core/dev.c"]
  DQX -->|"egress tc hook"| EGR["sch_handle_egress()<br/>clsact egress / tc-BPF"]
  EGR --> ROOT["Root qdisc (egress)<br/>handle 1: — enqueue / dequeue"]

  subgraph TCBOX["Traffic control: the three building blocks"]
    ROOT -->|"filters classify"| CLS["Classes 1:10, 1:20 ...<br/>(classful qdisc only)"]
    CLS --> LEAF["Leaf qdiscs<br/>fq_codel / fifo per class"]
  end

  LEAF --> SDX["sch_direct_xmit()<br/>dequeue, validate (GSO/csum)"]
  SDX --> RING["Driver TX ring<br/>bounded by BQL"]
  RING --> NIC["NIC -> wire"]

  NICRX["NIC RX -> wire in"] -.->|"ingress tc hook"| INGR["sch_handle_ingress()<br/>ingress / clsact qdisc<br/>policing + tc-BPF only"]
  INGR -.-> STACKRX["up the RX stack"]

Where traffic control sits on the packet’s path. What it shows: on egress (the solid path, top to bottom), every transmitted packet passes through the root qdisc, optionally through classes selected by filters, to a leaf qdisc, and only then to the driver’s transmit ring; the dashed path shows the much smaller ingress hook, which can only filter/police/redirect incoming packets, not queue and reschedule them. The insight to take: tc is a gate on the egress side with full queue-and-reschedule power, and a thin inspection point on the ingress side — there is no symmetric “receive scheduler” because, by the time a packet has arrived, the kernel can only accept or drop it, not un-send it.

Where tc Sits on the Transmit Path

The concrete entry point is __dev_queue_xmit() in net/core/dev.c. After the stack picks a transmit queue (netdev_core_pick_tx), it reads that queue’s qdisc with q = rcu_dereference_bh(txq->qdisc) and, if the qdisc has an enqueue operation, calls __dev_xmit_skb(skb, q, dev, txq) (v6.12 dev.c). That single line is the boundary: above it is “the stack built a packet,” below it is “traffic control owns this packet now.” __dev_xmit_skb either enqueues the skb and runs the qdisc, or — in the common, uncontended case — short-circuits straight to transmission via the bypass fast path (the mechanics of enqueue, the bypass, and the lockless qdisc_run loop live in Queueing Disciplines qdisc).

Crucially, the egress traffic-control hook runs even earlier, before the qdisc. In the same __dev_queue_xmit, under CONFIG_NET_EGRESS, the kernel calls sch_handle_egress(skb, &rc, dev) (v6.12 dev.c). This is the attach point for the clsact qdisc’s egress side and for tc-BPF programs — eBPF attached at the traffic-control layer (cross-linked to Linux eBPF MOC). So a packet can be inspected, rewritten, classified, or redirected by a tc filter or a BPF program before it ever reaches the scheduling qdisc.

After the qdisc decides which skb goes next, sch_direct_xmit() (in net/sched/sch_generic.c) dequeues it, runs validate_xmit_skb_list() to apply software GSO segmentation and checksum fixups outside the qdisc lock, and calls dev_hard_start_xmit() to hand the frame to the driver’s ndo_start_xmit (v6.12 sch_generic.c). The full transmit pipeline — including the driver ring — is in The Network Transmit Path; tc’s job ends at the driver hand-off.

The Three Building Blocks: Qdiscs, Classes, Filters

The entire subsystem is assembled from three object types, and understanding their relationship is the core of “getting” tc. The definitions below are quoted from tc(8).

Qdisc (queueing discipline). “Whenever the kernel needs to send a packet to an interface, it is enqueued to the qdisc configured for that interface.” A qdisc is the scheduler object: it owns the queue(s), implements enqueue and dequeue, and embodies the policy. Every interface always has at least a root qdisc; if you configure nothing, the kernel attaches a default one (see below). Qdiscs come in two flavours — classless (a single internal policy, e.g. pfifo_fast, fq_codel, tbf) and classful (a tree of classes) — covered in Classful and Classless Qdiscs.

Class. “Some qdiscs can contain classes, which contain further qdiscs — traffic may then be enqueued in any of the inner qdiscs, which are within the classes.” A class is a named subdivision of a classful qdisc, into which a subset of traffic is steered, and which itself contains a child (leaf) qdisc. Classes are how you partition bandwidth: a Hierarchical Token Bucket (HTB) qdisc with three classes can guarantee, say, 50/30/20 percent of a link to three traffic categories. Classless qdiscs have no classes at all.

Filter. “A filter is used by a classful qdisc to determine in which class a packet will be enqueued.” A filter is a classifier-plus-action: it matches packets (by destination port, by an fwmark set by netfilter, by a BPF program, by flow hash) and returns a classid telling the qdisc which class the packet belongs in. Per tc(8), when a packet enters a classful qdisc, “it can be classified to one of the classes within” — filters are consulted first, then the IP Type-of-Service field, then the userspace-set priority in skb->priority. The filter language and the available classifiers/actions are in The tc Tool and Filters.

The relationship is strictly hierarchical: a classful qdisc holds classes, filters sort packets into those classes, and each class holds a child qdisc that re-applies the same model recursively until a leaf qdisc (classless) actually queues the packet.

Root Qdisc vs Ingress Qdisc — Egress Is King

There are two attach points on an interface, and they are profoundly asymmetric.

The root qdisc governs egress. It is the real scheduler: it can shape (delay), schedule (reorder), prioritize, classify, and drop. Per tc(8), traffic control’s four operations are shaping (rate-limiting transmission), scheduling (reordering for prioritization), policing (regulating an incoming rate), and dropping (discarding traffic over a limit). Egress gets all four — most importantly shaping, which is only meaningful when you control when a packet leaves, and only the sender controls that.

The ingress qdisc is, per tc(8), “a special qdisc as it applies to incoming traffic on an interface,” and it is “limited” — it can filter and police (and, with clsact/tc-BPF, redirect), but it cannot truly queue and reschedule, because a received packet cannot be un-received. You cannot “shape” inbound traffic at the receiver in the strict sense; the most you can do is police — drop or mark packets that arrive faster than a configured rate, hoping the sender’s congestion control backs off. In the kernel, the modern ingress path is the ingress qdisc (or its superset clsact, which adds an egress side too); ingress_init() requires sch->parent == TC_H_INGRESS and wires up a tc-BPF / classifier block rather than a real queue (v6.12 sch_ingress.c). Both carry the TCQ_F_INGRESS flag in struct Qdisc.

Uncertain

The claim that the ingress qdisc cannot queue/reschedule is true of the stock ingress/clsact qdiscs (they expose only filter/police/redirect, no real enqueue scheduler). Genuine inbound shaping is conventionally done by redirecting ingress traffic to an Intermediate Functional Block (ifb) device and applying a normal egress qdisc there — a well-known idiom but verified here only against tc(8) general wording, not a step-by-step ifb primary source. Reason: the ifb redirection mechanism was not separately fetched. To resolve: read tc-mirred(8) and the ifb driver docs. uncertain

The Handle / ClassID Scheme: major:minor

Every qdisc and class is named by a 32-bit identifier written as major:minor in tc syntax, where both halves are 16-bit hexadecimal numbers (tc(8)). In the kernel these are the handle (for a qdisc) and parent/classid fields of struct Qdisc, and the split is done by macros in include/uapi/linux/pkt_sched.h (v6.12):

#define TC_H_MAJ_MASK (0xFFFF0000U)        /* top 16 bits  = major */
#define TC_H_MIN_MASK (0x0000FFFFU)        /* low 16 bits  = minor */
#define TC_H_MAJ(h) ((h)&TC_H_MAJ_MASK)
#define TC_H_MIN(h) ((h)&TC_H_MIN_MASK)
#define TC_H_MAKE(maj,min) (((maj)&TC_H_MAJ_MASK)|((min)&TC_H_MIN_MASK))
 
#define TC_H_UNSPEC   (0U)            /* "no handle"                       */
#define TC_H_ROOT     (0xFFFFFFFFU)   /* the egress root anchor            */
#define TC_H_INGRESS  (0xFFFFFFF1U)   /* the ingress anchor                */
#define TC_H_CLSACT   TC_H_INGRESS    /* clsact shares the ingress handle  */

Reading this line by line: a handle is one 32-bit word; TC_H_MAJ masks off the high 16 bits (the major number, which identifies a qdisc), TC_H_MIN masks off the low 16 bits (the minor number, which identifies a class within that qdisc), and TC_H_MAKE glues a chosen major and minor back into one word. The conventions that follow from this:

  • A qdisc handle is major:0 — written 1: in tc (minor 0 means “the qdisc itself”). The root qdisc is conventionally given handle 1:.
  • A class is major:minor with a non-zero minor — e.g. 1:10, 1:20 — sharing its parent qdisc’s major number. So classes 1:10 and 1:20 both live under qdisc 1:.
  • TC_H_ROOT (all-ones, ffff:ffff) is the special parent meaning “attach me as the egress root.” TC_H_INGRESS (ffff:fff1) is the ingress anchor, and clsact deliberately reuses the same value — they occupy the same slot.

This major:minor tree is exactly what lets filters target classes: a filter installed on qdisc 1: with classid 1:10 means “packets matching this filter go to class 1:10.” The numbering is purely a naming scheme — it carries no built-in priority — but by convention lower minors are configured as higher-priority classes.

The BQL Interplay — Why tc Needs a Bounded Driver Ring

A subtle but critical point: traffic control can only control latency if the standing queue actually forms in the qdisc, not in the driver. Every modern NIC has a hardware transmit ring — a fixed-size array of descriptors the driver fills and the NIC drains. If the driver eagerly stuffs that ring with hundreds of packets, then a packet the qdisc carefully scheduled is still going to sit behind a pile of already-committed frames in the ring, where tc has no visibility or control. The qdisc’s fancy scheduling is wasted because the real queue moved downstream into dumb hardware FIFO — a microcosm of bufferbloat.

Byte Queue Limits (BQL) solves this. BQL dynamically caps the number of bytes (not packets) in flight to the driver ring, sizing the cap to just enough to keep the link busy and no more (see Byte Queue Limits and Buffer Bloat for the algorithm). The single insight to carry here: BQL bounds the driver ring so that backlog accumulates in the qdisc instead. With BQL holding the ring near-empty, when packets pile up they pile up in the qdisc — exactly where fq_codel (or HTB, or whatever policy you chose) can measure the delay, drop or mark the right packets, and keep latency low. BQL and the qdisc are complementary halves of the same anti-bufferbloat strategy: BQL keeps the queue out of the hardware, the qdisc manages the queue it now owns. In struct Qdisc, the bulk-dequeue helpers (qdisc_avail_bulklimit, qdisc_may_bulk) read the BQL state (netdev_queue_dql_avail) precisely so the qdisc dequeues only as many bytes as BQL says the ring can take (v6.12 sch_generic.h).

AQM and Bufferbloat — The Reason tc’s Defaults Changed

The historical motivation for the modern defaults is bufferbloat: oversized, unmanaged buffers throughout the network that, instead of dropping packets under load, hoard them — turning a momentary burst into seconds of added latency and wrecking interactive traffic (bufferbloat.net CoDel wiki). A tail-drop FIFO is the classic culprit: it only drops when completely full, by which point it is already adding maximum delay.

The fix is Active Queue Management (AQM) — a qdisc that watches the queue and signals congestion early, before the buffer is full, by dropping or ECN-marking packets. The dominant AQM in Linux is CoDel (Controlled Delay), which measures the time a packet spent queued (its sojourn time) rather than the queue’s byte length, and acts when that delay stays above a small target (5 ms) for at least one interval (100 ms). Pairing CoDel with per-flow fair queuing gives fq_codel, which both controls delay and prevents one greedy flow from hurting others. The CoDel/fq_codel mechanism is dissected in Queueing Disciplines qdisc; here it suffices that AQM is why a smart qdisc replaced the dumb FIFO as the practical default.

Who actually sets the default qdisc

This is a point that is constantly misstated, so it is worth being precise. The kernel’s compile-time default is pfifo_fast, not fq_codel. In net/sched/sch_generic.c the kernel hard-codes const struct Qdisc_ops *default_qdisc_ops = &pfifo_fast_ops; (v6.12, identical in v6.18), and CONFIG_DEFAULT_NET_SCH defaults to the string "pfifo_fast" unless a kernel builder explicitly enables NET_SCH_DEFAULT and picks another (v6.12 net/sched/Kconfig). The Kconfig help text says it outright: “Nearly all users can safely say no here, and the default of pfifo_fast will be used. Many distributions already set the default value via /proc/sys/net/core/default_qdisc.”

The reason almost every modern Linux box nonetheless shows fq_codel is userspace policy, not a kernel constant: the net.core.default_qdisc sysctl can be written at runtime, and systemd ships a sysctl drop-in — sysctl.d/50-default.conf — containing net.core.default_qdisc = fq_codel with the comment “Fair Queue CoDel packet scheduler to fight bufferbloat” (systemd 50-default.conf). Writing that sysctl calls set_default_qdisc() → qdisc_set_default(), which swaps the global default_qdisc_ops pointer (v6.12 sysctl_net_core.c and sch_api.c). From then on, attach_one_default_qdisc() instantiates fq_codel for each newly-created interface’s queues. So the chain is: kernel default pfifo_fast → systemd sysctl drop-in → runtime swap to fq_codel → seen on every interface.

Uncertain

Exactly which mechanism a given system uses to land on fq_codel is distro policy and varies: the common path is the systemd sysctl.d/50-default.conf drop-in verified above, but some distributions instead (or additionally) build their kernel with CONFIG_DEFAULT_NET_SCH="fq_codel", and the effective default also depends on init-system ordering (the sysctl applies only after systemd-sysctl runs). The hard-coded systemd default has itself been contested — for instance, a request to make it configurable because a kernel without the fq_codel module compiled in then logs an error setting the sysctl (systemd issue #15183, closed). Reason: the per-distro kernel .config and any distro-specific drop-ins were not individually fetched. To resolve: check cat /proc/sys/net/core/default_qdisc and the active sysctl --system sources on the specific system in question. uncertain

Configuration Walk-through

A minimal session showing each building block. (The full tc command surface and filter syntax are in The tc Tool and Filters.)

# 1. Inspect the current root qdisc on eth0. On a systemd box this is
#    typically fq_codel (handle assigned automatically), via mq on a
#    multiqueue NIC (one fq_codel per hardware TX queue).
tc qdisc show dev eth0
 
# 2. Replace the root with a classful HTB qdisc, handle 1:, default class 1:30.
#    "root" maps to TC_H_ROOT; "handle 1:" is major=1, minor=0.
tc qdisc add dev eth0 root handle 1: htb default 30
 
# 3. Carve out classes under qdisc 1: — note all share major 1.
tc class add dev eth0 parent 1:  classid 1:1  htb rate 100mbit          # parent shaper
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 60mbit ceil 100mbit  # priority
tc class add dev eth0 parent 1:1 classid 1:20 htb rate 30mbit ceil 100mbit  # bulk
tc class add dev eth0 parent 1:1 classid 1:30 htb rate 10mbit ceil 100mbit  # default
 
# 4. Give each leaf class its own AQM so latency is managed per partition.
tc qdisc add dev eth0 parent 1:10 fq_codel
tc qdisc add dev eth0 parent 1:20 fq_codel
 
# 5. A filter that steers traffic into a class by destination port.
#    classid 1:10 means "matching packets go to class 1:10".
tc filter add dev eth0 parent 1: protocol ip prio 1 \
    u32 match ip dport 22 0xffff flowid 1:10
 
# 6. The ingress side: attach clsact, then police inbound to 50mbit.
tc qdisc add dev eth0 clsact
tc filter add dev eth0 ingress matchall \
    action police rate 50mbit burst 100k conform-exceed drop

Line-by-line: step 2 installs a classful root qdisc, demonstrating the handle 1: (major:minor) scheme and the egress root anchor (TC_H_ROOT). Steps 3 builds the class tree — every class reuses major 1 and gets a distinct minor, forming the partition hierarchy described above. Step 4 attaches a leaf qdisc (fq_codel) per class so each partition gets AQM. Step 5 installs a filter that classifies SSH (port 22) into the priority class 1:10, the filter→classid mechanism in action. Step 6 shows the ingress asymmetry: clsact can only police (drop over-rate packets), not shape — there is no htb equivalent on ingress.

Failure Modes and Common Misunderstandings

“I added a fancy qdisc but latency is still terrible.” Almost always the driver ring (or a downstream device’s buffer) is where the queue is actually forming, not your qdisc — the BQL problem above. Without BQL (or on a virtual/over-buffered path), the qdisc never sees the backlog. Verify BQL is active (/sys/class/net/<dev>/queues/tx-*/byte_queue_limits/), and remember that shaping slightly below the true bottleneck rate is what moves the queue back into your qdisc.

“My egress shaping does nothing on a tunnel/bridge/virtual device.” Software devices may carry IFF_NO_QUEUE, in which case attach_one_default_qdisc() installs noqueue (packets go straight to transmit with no qdisc) (v6.12 sch_generic.c). You must explicitly add a qdisc, and even then the physical egress device downstream is where real shaping must happen.

“I configured ingress shaping and it does nothing.” You cannot shape inbound at the receiver — only police. Real inbound rate control needs redirection to an ifb device (see the uncertainty note above) or, better, control at the actual bottleneck.

Confusing the kernel default with the running default. As detailed above, pfifo_fast is the kernel constant; fq_codel is what systemd installs. tc qdisc show reports the running qdisc; sysctl net.core.default_qdisc reports the policy for new interfaces. They can legitimately differ (e.g., an interface created before the sysctl was applied).

Per-queue confusion on multiqueue NICs. A multiqueue NIC does not have one root qdisc — it has an mq root with one child qdisc per hardware TX queue. A tc qdisc add ... root on such a device replaces the whole arrangement; people are often surprised their classful root “disappeared” the mq structure. The mq mechanism is in Queueing Disciplines qdisc.

Alternatives and When to Choose Them

Within tc, the choice is mostly which qdisc (see Classful and Classless Qdiscs for the full comparison): fq_codel/fq for fairness-and-latency on a general host, tbf for simple rate-limiting, htb for hierarchical bandwidth partitioning, cake as a modern all-in-one for the home-router edge. Outside tc, the alternatives are at different layers: XDP (see XDP Express Data Path) for line-rate drop/redirect before the stack, useful for DDoS scrubbing where you do not want to queue at all; eBPF at the socket/cgroup layer for per-application policy; and in orchestrated environments, Kubernetes/CNI bandwidth plugins that ultimately program tc qdiscs under the hood (the orchestration view lives in Kubernetes MOC). Choose tc when you need queue management and scheduling on an interface; choose XDP when you need to not even build an skb; choose socket-layer controls when policy is naturally per-application.

Production Notes

The single most consequential production fact is the bufferbloat fix itself: shipping fq_codel (or cake) as the default qdisc, paired with BQL-aware drivers, dramatically cut latency-under-load across the Linux ecosystem — this is why systemd hard-codes the sysctl (its 50-default.conf comment reads “Fair Queue CoDel packet scheduler to fight bufferbloat”), despite ongoing debate about whether it is the right default for every workload — including a request to make it overridable when the fq_codel module is absent (systemd issue #15183, closed). On high-throughput servers the relevant tuning is usually mq + per-queue fq (with TCP pacing for BBR) rather than a single shaped root, because a single locked root qdisc becomes a scaling bottleneck — the lockless-qdisc story in Queueing Disciplines qdisc exists precisely to address that. For traffic shaping specifically (e.g., metering a VM or a tenant to a contracted rate), htb and tbf remain the workhorses, and the universal gotcha is to shape just under the real bottleneck so the managed queue stays in your qdisc.

See Also

Graph View

Backlinks