Segmentation Offloads (GSO, TSO)
A modern TCP stream is moved between machines in chunks no larger than the Maximum Segment Size (MSS) — typically about 1448 bytes on a 1500-byte-MTU Ethernet link after headers. If the kernel had to physically carve every
send()into MSS-sized packets at the top of the stack, each tiny packet would then have to be routed, firewalled, queued, and checksummed individually — paying the full per-packet cost of traversing the networking stack thousands of times for a single megabyte of data. Segmentation offload is the family of techniques that defers that carving as late as possible. TCP Segmentation Offload (TSO) pushes the split all the way into the Network Interface Card (NIC) hardware: the kernel hands the device one large buffer plus an MSS value, and the silicon emits the wire packets. Generic Segmentation Offload (GSO) is “a pure software offload” that does the same chopping in the kernel — just before the frame reaches the device driver — when the NIC cannot do it itself (kernel docs, v6.12). Either way, a single oversized super-packet carries the segmentation parameters —gso_size,gso_segs,gso_type— in theskb_shared_infotrailer and traverses the stack once. Herbert Xu, who wrote GSO, put the rationale in a single sentence when he posted it in 2006: “a lot of the savings in TSO come from traversing the networking stack once rather than many times for each super-packet. These savings can be obtained without hardware support” (netdev, 2006-06-20).
Version pin. Every field, constant, code excerpt, and default in this note was read directly from the v6.12 tag of the mainline tree — Linux 6.12, a maintained long-term-support branch — fetched from raw.githubusercontent.com and verified on 2026-09-04. Mainline at that date is v7.2; where the newer tree differs the difference is called out and dated, and where the v6.18 LTS documentation differs from v6.12 that is called out too. This note assumes you already know what an sk_buff is; the structure itself, its skb_shared_info trailer, its paged fragments, and the ip_summed/csum_start/csum_offset triple are the subject of struct sk_buff, and are cross-linked rather than re-explained here.
Mental Model: One Super-Packet, Split Late
The core idea is that a struct sk_buff (the “socket buffer,” universally abbreviated skb — the structure that carries every packet through the Linux stack, see struct sk_buff) is allowed to hold far more than one wire frame’s worth of payload. A 64 KB TCP write becomes a single skb whose skb_shinfo()->gso_size field records “split me into pieces this big.” That one fat skb is what flows through the IP routing lookup, the netfilter hooks, and the traffic-control queue. Only at the very bottom of the transmit path — inside validate_xmit_skb(), immediately before the driver’s transmit routine — does the kernel ask: can the NIC segment this itself? If yes (TSO), the super-packet is handed to the hardware intact. If no (GSO), the kernel calls skb_gso_segment() to expand the one skb into a list of MSS-sized skbs right there, and only those final frames go to the driver.
Two things follow from this that are worth holding onto before reading further. First, the abstraction is uniform and the hardware is optional: the same super-packet flows down the stack whether or not the NIC can segment, and the software path exists precisely so that a packet re-routed at the last moment onto a device without TSO can still be transmitted. Second, segmentation is not free even in hardware — it is merely moved. The kernel still has to compute how many segments there will be, account for the header bytes that will exist on the wire but do not exist in the super-packet, and arrange for the checksums to come out right. Most of the subtlety in this subsystem is in those bookkeeping obligations, not in the split itself.
flowchart TB APP["send(64 KB)"] --> SK["one super-packet skb<br/>skb->len = 65536<br/>gso_size = 1448 (the MSS)<br/>gso_type = SKB_GSO_TCPV4<br/>gso_segs = 46"] SK --> IP["IP route lookup — ONCE"] IP --> NF["netfilter OUTPUT + POSTROUTING — ONCE"] NF --> QD["qdisc enqueue/dequeue — ONCE<br/>(pkt_len inflated by<br/>gso_segs-1 header copies)"] QD --> VX{"validate_xmit_skb()<br/>netif_needs_gso(skb, features)?"} VX -->|"false: device can offload<br/>NETIF_F_TSO set and<br/>ip_summed is usable"| HW["driver gets ONE skb<br/>NIC silicon emits 46 frames"] VX -->|"true: device cannot,<br/>or checksum state is wrong"| SEG["skb_gso_segment()<br/>CPU builds 46 skbs<br/>headers copied, pages shared"] SEG --> DRV["driver gets 46 skbs<br/>46 descriptor setups"] HW --> WIRE["46 frames of ≤1500 bytes on the wire"] DRV --> WIRE
The transmit path with segmentation offload, from one write to the wire. What it shows: a single 64 KB write becomes one super-packet skb that traverses routing, netfilter, and the qdisc exactly once; the actual split into 46 roughly-1448-byte frames happens at the very last moment, either in NIC hardware (TSO) or in software (skb_gso_segment), and the branch between them is a single predicate evaluated per packet. The insight: the expensive part of packet processing is the per-packet stack traversal, not the byte copy — so the win comes from carrying many segments’ worth of bytes as one unit through the costly upper layers, and paying the split cost once, as late as possible. Note the qdisc box: because the queueing layer must shape based on what will actually be on the wire, it has to predict the segmentation rather than observe it.
The Four-Way Taxonomy: TSO, GSO, GRO, LRO
These four acronyms are routinely conflated, including in otherwise-good documentation, and getting them apart is the single most useful thing this note can do. They differ along two independent axes — direction (transmit or receive) and who does the work (hardware or software) — and, crucially, along a third property that only matters on the receive side: whether the transformation is lossless.
- TSO — TCP Segmentation Offload. Transmit side, hardware, TCP only. The kernel hands the driver one super-packet; the NIC emits the wire segments. Backed by the
NETIF_F_TSOandNETIF_F_TSO6feature bits, surfaced byethtoolastx-tcp-segmentationandtx-tcp6-segmentation. - GSO — Generic Segmentation Offload. Transmit side, software, any protocol with a registered handler. The kernel splits the super-packet itself in
skb_gso_segment(), immediately before the driver. Backed byNETIF_F_GSO, surfaced astx-generic-segmentation. GSO is not an alternative to TSO so much as its mandatory understudy: the v6.12 documentation states the ordering invariant outright — “Before enabling any hardware segmentation offload a corresponding software offload is required in GSO. Otherwise it becomes possible for a frame to be re-routed between devices and end up being unable to be transmitted.” - GRO — Generic Receive Offload. Receive side, software, and lossless by construction. Packets arriving in the same NAPI poll are merged into one large skb, but only when the merge can be undone exactly. Backed by
NETIF_F_GRO, surfaced asrx-gro. Covered in depth in Generic Receive Offload. - LRO — Large Receive Offload. Receive side, hardware (or an in-driver emulation), and lossy. Corbet’s write-up of Herbert Xu’s 2009 Japan Linux Symposium talk records the objection precisely: LRO “merges everything in sight. This transformation is lossy; if there are important differences between the headers in incoming packets, those differences will be lost. And that breaks things” (LWN, 2009). Backed by
NETIF_F_LRO, surfaced asrx-lro.
The mechanical difference between GRO and LRO is visible in three lines of include/linux/skbuff.h at v6.12, and it is the best single illustration of why LRO cannot be reversed:
static inline bool skb_warn_if_lro(const struct sk_buff *skb)
{
/* LRO sets gso_size but not gso_type, whereas if GSO is really
* wanted then gso_type will be set. */
const struct skb_shared_info *shinfo = skb_shinfo(skb);
if (skb_is_nonlinear(skb) && shinfo->gso_size != 0 &&
unlikely(shinfo->gso_type == 0)) {
__skb_warn_lro_forwarding(skb);
return true;
}
return false;
}Read that comment carefully. An LRO’d skb carries a gso_size — “there used to be several packets in here, this big” — but no gso_type, because the hardware did not preserve enough information to say what kind of segmentation would recreate the original stream. A GRO’d skb carries both. And a skb with gso_size but no gso_type cannot be handed to skb_gso_segment(), because there is no handler to dispatch to. That is exactly why ip_forward() in v6.12 drops such packets outright and logs, via __skb_warn_lro_forwarding(), "%s: received packets cannot be forwarded while LRO is enabled".
GRO, by contrast, is designed so the round trip is exact. dev_gro_receive() in net/core/gro.c (v6.12) stamps the coalesced skb with skb_shinfo(skb)->gso_size = skb_gro_len(skb) when it starts holding a flow, and the per-protocol gro_complete handler fills in gso_type. The v6.12 documentation states the resulting invariant as a design requirement: “Ideally any frame assembled by GRO should be segmented to create an identical sequence of frames using GSO, and any sequence of frames segmented by GSO should be able to be reassembled back to the original by GRO.” (The v6.12 text then names one exception, IPv4 ID handling when the Don’t-Fragment bit is set; that sentence has been removed from the v6.18 text, which now states the invariant unqualified.) One further detail worth knowing: if only a single packet ends up in the aggregate, napi_gro_complete() sets gso_size = 0 again, so a lone packet is never falsely marked as a GSO skb.
flowchart LR subgraph TX["TRANSMIT — one big skb becomes N frames"] direction TB T1["GSO<br/>software, in kernel<br/>NETIF_F_GSO / tx-generic-segmentation<br/>any protocol with a gso_segment handler<br/>ALWAYS available, mandatory fallback"] T2["TSO<br/>hardware, in the NIC<br/>NETIF_F_TSO / NETIF_F_TSO6<br/>TCP only<br/>requires TX checksum offload"] T1 -. "same super-packet,<br/>same gso_size/gso_type" .- T2 end subgraph RX["RECEIVE — N frames become one big skb"] direction TB R1["GRO<br/>software, at NAPI poll<br/>NETIF_F_GRO / rx-gro<br/>LOSSLESS: sets gso_size AND gso_type<br/>safe to forward, bridge, tunnel"] R2["LRO<br/>hardware or in-driver<br/>NETIF_F_LRO / rx-lro<br/>LOSSY: sets gso_size, NOT gso_type<br/>auto-disabled when forwarding is on"] R1 -. "GRO output is a legal<br/>GSO input; LRO output is not" .- R2 end RX ==>|"a forwarded packet arrives as one<br/>big skb and leaves as one big skb"| TX
The four offloads placed on their two real axes — direction and who does the work — with the lossless/lossy split that only exists on receive. What it shows: TSO and GSO are two implementations of one transmit abstraction; GRO and LRO are two implementations of one receive idea, but only GRO’s output is a legal input to the transmit abstraction. The insight: the reason LRO is discouraged is not that it is slower or buggier than GRO but that it is not invertible — it produces an aggregate the kernel cannot take apart again, which is fine for a pure endpoint and fatal for a router, a bridge, or a virtualization host. The dashed link between the two receive boxes is the whole argument for GRO’s existence.
Everything else that gets grouped with these four is a variant of one of them. UFO (UDP Fragmentation Offload) is transmit-side and deprecated; USO (UDP Segmentation Offload) is the modern UDP analogue of TSO; GSO_PARTIAL is a hybrid for tunnelled traffic; GRO_HW (rx-gro-hw) is hardware assistance for GRO that, unlike LRO, is still expected to be lossless. Each gets its own section below.
| TSO | GSO | GRO | LRO | |
|---|---|---|---|---|
| Direction | transmit | transmit | receive | receive |
| Performed by | NIC hardware | kernel CPU | kernel CPU (at NAPI poll) | NIC hardware / driver |
| Protocols | TCPv4, TCPv6 | any with a gso_segment handler (TCP, UDP, SCTP, ESP, tunnels) | TCP, UDP, and more | typically TCP only |
| Kernel feature bit | NETIF_F_TSO, NETIF_F_TSO6 | NETIF_F_GSO | NETIF_F_GRO | NETIF_F_LRO |
ethtool -k name | tx-tcp-segmentation | tx-generic-segmentation | rx-gro | rx-lro |
ethtool -K short flag | tso | gso | gro | lro |
| Reversible? | n/a (it is the split) | n/a | yes — lossless by design | no |
| Safe when forwarding? | yes | yes | yes | no — kernel force-disables it |
Sets gso_type? | consumes it | consumes it | yes | no |
The four offloads as an enumerable reference grid. What it shows: the feature-bit names, the ethtool spellings (the short flag comes from ethtool’s own off_flag_def[] table, the long name from the kernel’s netdev_features_strings[]), and the forwarding-safety column that separates LRO from the rest. The insight: if you remember only one row, remember the last three — gso_type is the field that makes an aggregate reversible, reversibility is what makes forwarding safe, and LRO is the only one of the four that fails both.
Where the Segmentation Parameters Live
All of the offload state for a packet lives in three fields of struct skb_shared_info, the trailer that sits at the end of an skb’s head buffer and is shared between an skb and its clones. The full structure and its role in the skb’s two-allocation design are the subject of struct sk_buff and skb_shared_info and Paged Fragments; here is the head of it at v6.12, with only the fields this note needs:
/* This data is invariant across clones and lives at
* the end of the header data, ie. at skb->end.
*/
struct skb_shared_info {
__u8 flags;
__u8 meta_len;
__u8 nr_frags;
__u8 tx_flags;
unsigned short gso_size;
/* Warning: this field is not always filled in (UFO)! */
unsigned short gso_segs;
struct sk_buff *frag_list;
union {
struct skb_shared_hwtstamps hwtstamps;
struct xsk_tx_metadata_compl xsk_meta;
};
unsigned int gso_type;
u32 tskey;
/* ... dataref and frags[] follow ... */
};gso_size is the target payload size of each emitted segment. For TCP it is the MSS. It is an unsigned short, so it tops out at 65535, and one specific value is reserved: GSO_BY_FRAGS (0xFFFF) means “the segment boundaries are not a fixed size — they are wherever the frag_list entries end,” which is how SCTP uses the GSO machinery to push a batch of correctly-padded chunks through the stack as one unit. Because gso_size doubles as the flag for “this is a GSO skb,” the test is trivially cheap:
static inline unsigned int skb_is_gso(const struct sk_buff *skb)
{
return skb_shinfo(skb)->gso_size;
}gso_segs is how many segments the super-packet will become. It is a u16, and the comment above it is a real warning rather than a formality: the field is not reliably populated for skbs that arrived from an untrusted source, which is why several code paths recompute it as DIV_ROUND_UP(len, mss) rather than trusting it. The combination of a 16-bit gso_segs and a minimum sensible MSS is what fixes the absolute ceiling on super-packet size, as include/linux/netdevice.h (v6.12) documents in a comment sitting between the two constants: GSO_MAX_SEGS is 65535, and GSO_MAX_SIZE is 8 * GSO_MAX_SEGS = 524,280 bytes, on the reasoning that “TCP minimal MSS is 8 (TCP_MIN_GSO_SIZE), and shinfo->gso_segs is a 16bit field.”
gso_type is a bitmask saying what kind of segmentation is wanted, so the code can dispatch to the right per-protocol handler and know which headers must be rewritten. At v6.12 the enumeration is:
enum {
SKB_GSO_TCPV4 = 1 << 0,
SKB_GSO_DODGY = 1 << 1, /* skb is from an untrusted source */
SKB_GSO_TCP_ECN = 1 << 2, /* the TCP segment has CWR set */
SKB_GSO_TCP_FIXEDID = 1 << 3, /* do not increment IPv4 ID per segment */
SKB_GSO_TCPV6 = 1 << 4,
SKB_GSO_FCOE = 1 << 5,
SKB_GSO_GRE = 1 << 6,
SKB_GSO_GRE_CSUM = 1 << 7,
SKB_GSO_IPXIP4 = 1 << 8, /* IPv4 or IPv6 inside IPv4 */
SKB_GSO_IPXIP6 = 1 << 9, /* IPv4 or IPv6 inside IPv6 */
SKB_GSO_UDP_TUNNEL = 1 << 10, /* VXLAN, GENEVE, ... */
SKB_GSO_UDP_TUNNEL_CSUM = 1 << 11,
SKB_GSO_PARTIAL = 1 << 12,
SKB_GSO_TUNNEL_REMCSUM = 1 << 13,
SKB_GSO_SCTP = 1 << 14,
SKB_GSO_ESP = 1 << 15, /* IPsec ESP */
SKB_GSO_UDP = 1 << 16, /* legacy UFO */
SKB_GSO_UDP_L4 = 1 << 17, /* USO — UDP payload GSO, NOT UFO */
SKB_GSO_FRAGLIST = 1 << 18,
};Three of these are not really “kinds of segmentation” at all but modifiers. SKB_GSO_DODGY marks a super-packet that arrived from userspace or a guest — through tuntap, virtio-net, or a packet socket — and therefore carries attacker-controlled gso_size and gso_segs; every consumer must re-derive rather than trust. SKB_GSO_TCP_ECN records that the original TCP header had the Congestion Window Reduced bit set, which matters because that bit must appear on exactly one emitted segment. SKB_GSO_TCP_FIXEDID suppresses the per-segment IPv4 identification increment.
Point-in-time note, verified 2026-09-04. At mainline v7.2 this enumeration has moved on in two ways worth recording. A new type
SKB_GSO_TCP_ACCECN = 1 << 19has appeared, for Accurate ECN. More disruptively,SKB_GSO_TCP_FIXEDIDhas been relocated from bit 3 to bit 30, with the old bit 3 renamed__SKB_GSO_TCP_FIXEDIDand a companionSKB_GSO_TCP_FIXEDID_INNERplaced at bit 31, carrying the comment “These indirectly map onto the same netdev feature.” The v6.18 LTS documentation describes the semantics of the new inner flag: “For encapsulated packets,SKB_GSO_TCP_FIXEDIDrefers only to the outer header.SKB_GSO_TCP_FIXEDID_INNERcan be used to specify the same for the inner header. Any combination of these two GSO types is allowed.” Everything else in this note’s v6.12 excerpts is unchanged at v7.2.
packet-beta 0-15: "gso_size (u16) — bytes of payload per segment; 0xFFFF = GSO_BY_FRAGS" 16-31: "gso_segs (u16) — segment count; untrusted when SKB_GSO_DODGY" 32-63: "gso_type (unsigned int) — SKB_GSO_* bitmask, see enum above"
The three segmentation fields drawn as a contiguous 12-byte window, in declaration order but with frag_list and the hwtstamps union elided so the trio can be seen together. What it shows: the entire contract between “the layer that decided to build a super-packet” and “the layer that will take it apart” is 96 bits, sitting in the shared trailer rather than in the skb itself. The insight: because these fields live in skb_shared_info and not in struct sk_buff, they are shared by every clone of the packet — which is exactly right, since segmentation geometry is a property of the data, not of any particular reference to it. It also means the split cost is bounded: whatever transformation the stack applies to a super-packet, this 12-byte descriptor is all that has to stay consistent with it. (Drawn with mermaid packet-beta; the renderer wraps a 64-bit span onto two 32-bit rows, which is a display artefact and not a structural boundary — see Drawing Wire Formats with Mermaid Packet Diagrams.)
Building the Super-Packet: Who Decides, and How Big
Before anything can be segmented, something has to build an over-sized skb in the first place. For TCP this happens in two stages: a per-socket capability negotiation that decides whether super-packets are legal on this route, and a per-transmission sizing decision that decides how big they should be right now.
The capability stage is sk_setup_caps() in net/core/sock.c (v6.12), called whenever a socket acquires or re-acquires a route. It is short and every line matters:
void sk_setup_caps(struct sock *sk, struct dst_entry *dst)
{
u32 max_segs = 1;
sk->sk_route_caps = dst->dev->features; /* start from the egress device */
if (sk_is_tcp(sk))
sk->sk_route_caps |= NETIF_F_GSO; /* TCP ALWAYS gets software GSO */
if (sk->sk_route_caps & NETIF_F_GSO)
sk->sk_route_caps |= NETIF_F_GSO_SOFTWARE;
if (unlikely(sk->sk_gso_disabled))
sk->sk_route_caps &= ~NETIF_F_GSO_MASK;
if (sk_can_gso(sk)) {
if (dst->header_len && !xfrm_dst_offload_ok(dst)) {
sk->sk_route_caps &= ~NETIF_F_GSO_MASK; /* IPsec: no GSO */
} else {
sk->sk_route_caps |= NETIF_F_SG | NETIF_F_HW_CSUM;
sk->sk_gso_max_size = sk_dst_gso_max_size(sk, dst);
max_segs = max_t(u32, READ_ONCE(dst->dev->gso_max_segs), 1);
}
}
sk->sk_gso_max_segs = max_segs;
sk_dst_set(sk, dst);
}Line by line: the socket’s capabilities start as a copy of the egress device’s; a TCP socket then unconditionally gains NETIF_F_GSO whether or not any hardware supports anything, which is the mechanical embodiment of “software GSO is always available”; having GSO implies the whole NETIF_F_GSO_SOFTWARE group (which is NETIF_F_ALL_TSO | NETIF_F_GSO_SCTP | NETIF_F_GSO_UDP_L4 | NETIF_F_GSO_FRAGLIST at v6.12); a route through an IPsec transform that cannot itself be offloaded strips all GSO, because the encryption has to see the final packets; and the surviving socket then records a byte ceiling and a segment ceiling that TCP will respect when building skbs.
The byte ceiling deserves a look, because it hides a protocol asymmetry:
static u32 sk_dst_gso_max_size(struct sock *sk, struct dst_entry *dst)
{
bool is_ipv6 = (sk->sk_family == AF_INET6 &&
!ipv6_addr_v4mapped(&sk->sk_v6_rcv_saddr));
u32 max_size = is_ipv6 ? READ_ONCE(dst->dev->gso_max_size) :
READ_ONCE(dst->dev->gso_ipv4_max_size);
if (max_size > GSO_LEGACY_MAX_SIZE && !sk_is_tcp(sk))
max_size = GSO_LEGACY_MAX_SIZE;
return max_size - (MAX_TCP_HEADER + 1);
}IPv4 and IPv6 have separate device ceilings (gso_ipv4_max_size and gso_max_size), because raising the limit past 64 KB requires protocol support that arrived for IPv6 first; and non-TCP sockets are clamped to GSO_LEGACY_MAX_SIZE (65536) regardless. Both are discussed in the size-ceilings section below.
The sizing stage is tcp_tso_segs() in net/ipv4/tcp_output.c, and it is the part most people do not know exists: the super-packet is not made as large as the hardware allows. It is sized from the connection’s current pacing rate, so that one TSO burst is roughly one pacing quantum of transmission time.
static u32 tcp_tso_autosize(const struct sock *sk, unsigned int mss_now,
int min_tso_segs)
{
unsigned long bytes;
u32 r;
bytes = READ_ONCE(sk->sk_pacing_rate) >> READ_ONCE(sk->sk_pacing_shift);
r = tcp_min_rtt(tcp_sk(sk)) >> READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_tso_rtt_log);
if (r < BITS_PER_TYPE(sk->sk_gso_max_size))
bytes += sk->sk_gso_max_size >> r;
bytes = min_t(unsigned long, bytes, sk->sk_gso_max_size);
return max_t(u32, bytes / mss_now, min_tso_segs);
}Walk the symbols. sk_pacing_rate is the connection’s current send rate in bytes per second, maintained by the congestion-control module. sk_pacing_shift defaults to 10 (net/core/sock.c, v6.12), and shifting a bytes-per-second rate right by 10 divides it by 1024 — so bytes starts out as roughly one millisecond’s worth of transmission at the current rate. Then a distance-dependent bonus is added: r is the connection’s minimum observed round-trip time shifted right by sysctl_tcp_tso_rtt_log, which defaults to 9 (that is, halved for every 512 µs of RTT), and sk_gso_max_size >> r is a full-size allowance that decays geometrically as the peer gets further away. The comment in the source spells out the intent: “For close peers, we rather send bigger packets to reduce cpu costs, because occasional losses will be repaired fast. For long distance/rtt flows, we would like to get ACK clocking with 1 ACK per ms.” The result is clamped to the device ceiling, divided by the MSS to get a segment count, and floored at sysctl_tcp_min_tso_segs (default 2). tcp_tso_segs() then applies one final cap, sk->sk_gso_max_segs.
Once the size is chosen, stamping it into the skb is three lines:
static int tcp_set_skb_tso_segs(struct sk_buff *skb, unsigned int mss_now)
{
if (skb->len <= mss_now) {
/* Avoid the costly divide in the normal non-TSO case. */
TCP_SKB_CB(skb)->tcp_gso_size = 0;
tcp_skb_pcount_set(skb, 1);
return 1;
}
TCP_SKB_CB(skb)->tcp_gso_size = mss_now;
tcp_skb_pcount_set(skb, DIV_ROUND_UP(skb->len, mss_now));
return DIV_ROUND_UP(skb->len, mss_now);
}Note where the MSS lives at this stage: in TCP_SKB_CB(skb)->tcp_gso_size, inside the 48-byte control buffer described in struct sk_buff, not yet in skb_shared_info. TCP keeps the segmentation intent in its own private scratch while the skb is still on the write queue and may be re-split by loss recovery; it is transferred into skb_shinfo()->gso_size on the way out. This also means TCP’s own accounting unit — tcp_skb_pcount(), used by congestion control to count packets in flight — is the segment count of the super-packet, not one.
flowchart TB CC["congestion control<br/>sets sk_pacing_rate"] --> A["bytes = sk_pacing_rate >> sk_pacing_shift<br/>shift = 10 ⇒ about 1 ms of transmission"] RTT["tcp_min_rtt()"] --> B["r = min_rtt >> tcp_tso_rtt_log (9)<br/>bytes += sk_gso_max_size >> r<br/>bonus halves per 512 us of RTT"] A --> C["bytes = min(bytes, sk_gso_max_size)"] B --> C C --> D["segs = max(bytes / mss, tcp_min_tso_segs)<br/>default floor = 2"] D --> E["segs = min(segs, sk_gso_max_segs)<br/>device segment ceiling"] E --> F["build one skb of segs x mss bytes<br/>TCP_SKB_CB->tcp_gso_size = mss<br/>tcp_skb_pcount = segs"] F --> G["on transmit: copy into<br/>skb_shinfo->gso_size / gso_segs / gso_type"]
How TCP decides how large a super-packet to build, at v6.12. What it shows: the size is a function of the pacing rate and the round-trip time, clamped by two device limits — not simply “as big as the NIC allows.” The insight: an unbounded TSO burst would defeat pacing and dump a millisecond-plus of back-to-back line-rate traffic into the network, so the kernel deliberately sizes the super-packet to about one pacing quantum, and deliberately shrinks it for distant peers so that ACK clocking survives. This is why a fast local connection and a transcontinental one, on the same NIC with the same gso_max_size, will show completely different average TSO sizes — and why “why are my TSO packets small?” is usually a pacing question, not an offload question.
Why Segmentation Rests on CHECKSUM_PARTIAL
TSO is not merely helped by checksum offload; it is impossible without it, and the v6.12 documentation says so plainly: “TCP segmentation is dependent on support for the use of partial checksum offload. For this reason TSO is normally disabled if the Tx checksum offload for a given device is disabled.”
The reason is arithmetic, not policy. Each emitted wire segment is an independent TCP segment with its own header, its own length, its own sequence number — and therefore its own checksum, computed over its own bytes. Those checksums cannot exist before the split, because the segments do not exist before the split. So whoever performs the segmentation must also compute the checksums, and if the hardware is doing the segmenting, the hardware must be doing the checksumming.
The mechanism that lets the kernel hand that job over is CHECKSUM_PARTIAL, the transmit-side value of the skb’s ip_summed field. The field itself, its four values, and the csum_start/csum_offset pair that accompanies CHECKSUM_PARTIAL are documented in struct sk_buff and in Checksum Offloads; what matters here is the contract it establishes. Under CHECKSUM_PARTIAL, the kernel has written a pseudo-header checksum — a partial sum over the source and destination addresses, the protocol number and the payload length — into the transport header’s checksum field, and has told the device where the transport header begins (csum_start) and where inside it the checksum field sits (csum_offset). The device finishes the job by summing the actual bytes and folding the result in. This composability is what makes the whole thing work: a partial sum over a changing length can be adjusted arithmetically rather than recomputed, so the segmenting agent — silicon or software — can produce a correct per-segment checksum by delta from the super-packet’s.
The v6.12 documentation adds the second precondition, which is about the device knowing where to write: “In order to support TCP segmentation offload it is necessary to populate the network and transport header offsets of the skbuff so that the device drivers will be able determine the offsets of the IP or IPv6 header and the TCP header. In addition as CHECKSUM_PARTIAL is required csum_start should also point to the TCP header of the packet.” Both requirements are enforced at runtime. tcp_gso_segment() bails out immediately if skb_checksum_start(skb) != skb_transport_header(skb), and netif_needs_gso() forces software segmentation whenever ip_summed is not in a state the hardware can finish:
static inline bool netif_needs_gso(struct sk_buff *skb, netdev_features_t features)
{
return skb_is_gso(skb) && (!skb_gso_ok(skb, features) ||
unlikely((skb->ip_summed != CHECKSUM_PARTIAL) &&
(skb->ip_summed != CHECKSUM_UNNECESSARY)));
}There is a third, quieter enforcement point in harmonize_features() (net/core/dev.c, v6.12), and it is the one that most cleanly expresses the dependency, because it strips the two feature groups together:
if (skb->ip_summed != CHECKSUM_NONE &&
!can_checksum_protocol(features, type)) {
features &= ~(NETIF_F_CSUM_MASK | NETIF_F_GSO_MASK);
}If the device cannot checksum this protocol, it loses not just its checksum features but every segmentation feature it had. That single line is the “TSO depends on checksum offload” rule expressed as code.
flowchart TB T["TCP builds super-packet<br/>len = 65536, gso_size = 1448"] --> P["ip_summed = CHECKSUM_PARTIAL<br/>csum_start → TCP header<br/>csum_offset = 16<br/>th->check = pseudo-header sum only"] P --> Q{"can the egress device<br/>checksum this protocol?"} Q -->|no| STRIP["harmonize_features():<br/>clears CSUM_MASK and GSO_MASK<br/>in one statement — BOTH groups die"] STRIP --> SW["netif_needs_gso() is true<br/>→ skb_gso_segment() in software<br/>→ then skb_checksum_help() per segment"] Q -->|yes| HW["super-packet handed to driver"] HW --> NIC["NIC splits at gso_size,<br/>and for EACH segment:<br/>fix length, fix seq,<br/>sum the bytes, fold into th->check"] SW --> WIRE2["N correct frames"] NIC --> WIRE2
The checksum-segmentation dependency, drawn as the decision it actually is. What it shows: CHECKSUM_PARTIAL is a promise deferred — the kernel writes only the part of the checksum that does not depend on the payload, and whoever splits the packet completes it per segment. The insight: segmentation and checksumming are the same job seen twice, because the only agent that knows a segment’s final bytes is the agent that created it. That is why harmonize_features() clears NETIF_F_CSUM_MASK and NETIF_F_GSO_MASK in one statement, and why ethtool -K eth0 tx off silently turns off TSO too — a behaviour that regularly surprises people who expected the two knobs to be independent.
The Decision Point: What Actually Forces a Software Split
The branch between hardware and software segmentation happens in validate_xmit_skb() (net/core/dev.c, v6.12), which runs after the qdisc has released the packet and immediately before the driver’s ndo_start_xmit. The full egress path around it is the subject of The Network Transmit Path; here is only the segmentation branch:
static struct sk_buff *validate_xmit_skb(struct sk_buff *skb, struct net_device *dev, bool *again)
{
netdev_features_t features;
features = netif_skb_features(skb); /* what can THIS device do for THIS skb? */
skb = validate_xmit_vlan(skb, features);
if (unlikely(!skb))
goto out_null;
skb = sk_validate_xmit_skb(skb, dev); /* kTLS offload hook */
if (unlikely(!skb))
goto out_null;
if (netif_needs_gso(skb, features)) {
struct sk_buff *segs;
segs = skb_gso_segment(skb, features); /* SOFTWARE split, here, now */
if (IS_ERR(segs)) {
goto out_kfree_skb;
} else if (segs) {
consume_skb(skb); /* free the super-packet */
skb = segs; /* replace it with the list of segments */
}
} else {
if (skb_needs_linearize(skb, features) && __skb_linearize(skb))
goto out_kfree_skb;
if (skb->ip_summed == CHECKSUM_PARTIAL) {
/* ... device cannot finish the checksum: do it in software ... */
if (skb_csum_hwoffload_help(skb, features))
goto out_kfree_skb;
}
}
skb = validate_xmit_xfrm(skb, features, again);
return skb;
/* ... error paths ... */
}Note the else if (segs) on the segmentation result: skb_gso_segment() may legitimately return NULL, meaning “this needed no segmentation after all,” in which case the original skb continues unchanged. That is not a degenerate case; it is the header-verification path for untrusted packets, explained below.
The interesting work is inside netif_skb_features(), which is where “the device advertises TSO” gets narrowed down to “the device can do TSO for this particular packet, right now.” It is a pipeline of independent narrowing steps, and any one of them can strip NETIF_F_GSO_MASK and thereby force a software split:
netdev_features_t netif_skb_features(struct sk_buff *skb)
{
struct net_device *dev = skb->dev;
netdev_features_t features = dev->features;
if (skb_is_gso(skb))
features = gso_features_check(skb, dev, features);
if (skb->encapsulation)
features &= dev->hw_enc_features;
if (skb_vlan_tagged(skb))
features = netdev_intersect_features(features,
dev->vlan_features | NETIF_F_HW_VLAN_CTAG_TX |
NETIF_F_HW_VLAN_STAG_TX);
if (dev->netdev_ops->ndo_features_check)
features &= dev->netdev_ops->ndo_features_check(skb, dev, features);
else
features &= dflt_features_check(skb, dev, features);
return harmonize_features(skb, features);
}and gso_features_check() is the segmentation-specific stage:
static netdev_features_t gso_features_check(const struct sk_buff *skb,
struct net_device *dev,
netdev_features_t features)
{
u16 gso_segs = skb_shinfo(skb)->gso_segs;
if (gso_segs > READ_ONCE(dev->gso_max_segs))
return features & ~NETIF_F_GSO_MASK; /* too many segments */
if (unlikely(skb->len >= netif_get_gso_max_size(dev, skb)))
return features & ~NETIF_F_GSO_MASK; /* too many bytes */
if (!skb_shinfo(skb)->gso_type) {
skb_warn_bad_offload(skb); /* gso_size but no gso_type! */
return features & ~NETIF_F_GSO_MASK;
}
if (!(skb_shinfo(skb)->gso_type & SKB_GSO_PARTIAL))
features &= ~dev->gso_partial_features;
if (skb_shinfo(skb)->gso_type & SKB_GSO_TCPV4) {
struct iphdr *iph = skb->encapsulation ? inner_ip_hdr(skb) : ip_hdr(skb);
if (!(iph->frag_off & htons(IP_DF)))
features &= ~NETIF_F_TSO_MANGLEID; /* may be fragmented: IDs must be real */
}
return features;
}That third test is the same LRO trap from the taxonomy section, seen from the transmit side: a packet with gso_size set but gso_type zero triggers skb_warn_bad_offload(), which dumps the skb and issues a WARN(1, ...) naming the driver and printing both the device’s feature mask and the socket’s route capabilities.
Collecting all of it, here is the complete list of things that force a software split of a packet that TCP built expecting hardware TSO:
flowchart TB START["super-packet arrives at<br/>validate_xmit_skb()"] --> F0["features = dev->features"] F0 --> G1{"gso_segs > dev->gso_max_segs?"} G1 -->|yes| STRIP["features &= ~NETIF_F_GSO_MASK"] G1 -->|no| G2{"skb->len ≥ gso_max_size<br/>(IPv4 and IPv6 ceilings differ)?"} G2 -->|yes| STRIP G2 -->|no| G3{"gso_type == 0?<br/>(an LRO'd or malformed skb)"} G3 -->|yes| WARN["skb_warn_bad_offload()<br/>WARN + skb_dump"] --> STRIP G3 -->|no| G4["clear TSO_MANGLEID if the<br/>IPv4 header lacks the DF bit"] G4 --> E1{"skb->encapsulation?"} E1 -->|yes| E2["features &= dev->hw_enc_features<br/>most NICs advertise far less here"] E1 -->|no| V1 E2 --> V1{"VLAN tag present?"} V1 -->|yes| V2["intersect with dev->vlan_features"] V1 -->|no| D1 V2 --> D1["driver's ndo_features_check()<br/>e.g. header too long, too many frags,<br/>unsupported tunnel depth"] D1 --> H1{"can_checksum_protocol(features, type)?"} H1 -->|no| H2["clears CSUM_MASK and GSO_MASK together"] --> STRIP H1 -->|yes| H3["illegal_highdma? strip NETIF_F_SG"] H3 --> DEC{"netif_needs_gso(skb, features)"} STRIP --> DEC DEC -->|"!skb_gso_ok OR<br/>ip_summed not PARTIAL/UNNECESSARY"| SOFT["skb_gso_segment()<br/>CPU splits now"] DEC -->|otherwise| HARD["hand the super-packet<br/>to ndo_start_xmit() — TSO"]
Every gate between “TCP built a super-packet” and “the NIC does the split,” at v6.12. What it shows: hardware TSO is not a property of the device but the conjunction of six independent narrowing steps, evaluated per packet, any one of which silently demotes the packet to a software split. The insight: this is why TSO can be “on” in ethtool -k and yet your workload never uses it. The three that bite most often in practice are the encapsulation step (hw_enc_features is usually a much smaller set than features, so a VXLAN or WireGuard-adjacent path loses offloads), the driver’s own ndo_features_check (which rejects packets whose header stack is deeper or longer than the descriptor format can express), and the checksum step (which takes segmentation down with it). None of these produce a log message; the only evidence is CPU time in skb_segment().
Two further conditions live outside this pipeline and are easy to forget. skb_gso_ok() also requires NETIF_F_FRAGLIST if the skb has a frag_list, so a fraglist-carrying aggregate is software-segmented on almost all hardware. And sk_setup_caps(), shown earlier, has already stripped NETIF_F_GSO_MASK from the socket entirely if the route goes through a non-offloadable IPsec transform — meaning the super-packet is never built in the first place on that path.
Inside skb_segment(): What Is Copied and What Is Shared
skb_gso_segment() is a thin wrapper in include/net/gso.h around __skb_gso_segment(skb, features, true) — the true being tx_path. __skb_gso_segment() (net/core/gso.c, v6.12) does three things before dispatching: it ensures the head is writable if the checksum needs initialising, it decides whether NETIF_F_GSO_PARTIAL is genuinely usable for this frame, and it resets the MAC header and length. Then it calls skb_mac_gso_segment(), which looks up the L3 protocol’s handler by walking net_hotdata.offload_base under RCU:
struct sk_buff *skb_mac_gso_segment(struct sk_buff *skb, netdev_features_t features)
{
struct packet_offload *ptype;
int vlan_depth = skb->mac_len;
__be16 type = skb_network_protocol(skb, &vlan_depth);
__skb_pull(skb, vlan_depth);
rcu_read_lock();
list_for_each_entry_rcu(ptype, &net_hotdata.offload_base, list) {
if (ptype->type == type && ptype->callbacks.gso_segment) {
segs = ptype->callbacks.gso_segment(skb, features);
break;
}
}
rcu_read_unlock();
__skb_push(skb, skb->data - skb_mac_header(skb));
return segs;
}For IPv4 that reaches inet_gso_segment(), which handles the IP header and recurses into the L4 handler; for TCP that is tcp4_gso_segment() → tcp_gso_segment(). Every one of those handlers eventually calls the same workhorse: skb_segment() in net/core/skbuff.c, which does the data carve and nothing protocol-specific.
The single most misunderstood thing about skb_segment() is what it does with the payload. It is widely assumed that software segmentation copies the data — that GSO is “TSO but you pay a memcpy.” On a scatter-gather-capable path that is simply false. Here is the loop body that builds each segment, with the essential lines kept:
/* one iteration per output segment */
nskb = __alloc_skb(hsize + doffset + headroom, GFP_ATOMIC, ...);
skb_reserve(nskb, headroom);
__skb_put(nskb, doffset);
__copy_skb_header(nskb, head_skb); /* metadata: the `headers` struct_group */
skb_headers_offset_update(nskb, skb_headroom(nskb) - headroom);
skb_reset_mac_len(nskb);
/* the HEADERS are copied, byte for byte, into every segment */
skb_copy_from_linear_data_offset(head_skb, -tnl_hlen,
nskb->data - tnl_hlen, doffset + tnl_hlen);
nskb_frag = skb_shinfo(nskb)->frags;
/* any linear payload still in the head buffer is copied */
skb_copy_from_linear_data_offset(head_skb, offset, skb_put(nskb, hsize), hsize);
while (pos < offset + len) {
if (unlikely(skb_shinfo(nskb)->nr_frags >= MAX_SKB_FRAGS)) {
net_warn_ratelimited("skb_segment: too many frags: %u %u\n", pos, mss);
err = -EINVAL;
goto err;
}
*nskb_frag = *frag; /* the PAGE DESCRIPTOR is copied ... */
__skb_frag_ref(nskb_frag); /* ... and a reference taken on the page */
size = skb_frag_size(nskb_frag);
if (pos < offset) { /* trim the leading edge of a straddling frag */
skb_frag_off_add(nskb_frag, offset - pos);
skb_frag_size_sub(nskb_frag, offset - pos);
}
skb_shinfo(nskb)->nr_frags++;
if (pos + size <= offset + len) { i++; frag++; pos += size; }
else { skb_frag_size_sub(nskb_frag, pos + size - (offset + len)); goto skip_fraglist; }
nskb_frag++;
}So the actual transformation is: a new, small head buffer is allocated per segment, holding only the headers; the headers are copied into it; and the payload pages are shared by reference, with the segments at the boundaries pointing into the middle of a page via adjusted (offset, size) pairs. __skb_frag_ref() takes a reference on each page so the original super-packet’s release does not free data the segments still point at. For a 64 KB TCP write built by sendmsg() with page fragments — or by sendfile()/splice(), where the pages belong to the page cache — this means software GSO copies about 46 × 66 bytes of headers, not 64 KB of payload.
The exception is the non-scatter-gather path. If features lacks NETIF_F_SG — a device that cannot DMA from multiple buffers, or a path where illegal_highdma() stripped SG — the code takes a different branch and does copy every byte, either with skb_copy_bits() or, when the checksum must also be computed, with skb_copy_and_csum_bits() in a single pass. This is exactly the “worst-case scenario” Herbert Xu measured in 2006: “where the NIC does not support SG and the user uses write(2) which means that we have to copy the data twice. … the cost of the extra copy is mostly offset by the reduction in the cost of going through the networking stack.”
flowchart TB subgraph BEFORE["BEFORE — one super-packet"] H0["head buffer<br/>headroom + MAC/IP/TCP headers<br/>~66 bytes of linear data"] S0["skb_shared_info<br/>gso_size=1448 gso_segs=46<br/>gso_type=SKB_GSO_TCPV4<br/>nr_frags=16"] P0["page 0"] & P1["page 1"] & PN["... page 15"] S0 --> P0 & P1 & PN end subgraph AFTER["AFTER — 46 segment skbs"] A1["seg 1: NEW head buffer<br/>headers COPIED<br/>frags → page 0, bytes 0-1447"] A2["seg 2: NEW head buffer<br/>headers COPIED<br/>frags → page 0, bytes 1448-2895"] AN["seg 46: NEW head buffer<br/>headers COPIED<br/>frags → page 15 tail"] end H0 -->|"skb_copy_from_linear_data_offset()<br/>headers copied 46 times<br/>~3 KB total"| A1 H0 --> A2 H0 --> AN P0 -->|"*nskb_frag = *frag<br/>__skb_frag_ref()<br/>PAGE SHARED, refcount++"| A1 P0 --> A2 PN --> AN S0 -->|"__copy_skb_header():<br/>priority, mark, hash, ip_summed,<br/>protocol, the whole headers group"| A1
What skb_segment() actually produces on a scatter-gather path. What it shows: each segment gets a freshly allocated head buffer with a byte-for-byte copy of the headers and a memcpy’d metadata block, but the payload pages are referenced, not copied — segments that begin or end mid-page carry an adjusted (page, offset, size) triple into the same page. The insight: software GSO is not “TSO plus a data copy.” Its real costs are 46 small allocations, 46 header copies, 46 atomic page references, and 46 driver descriptor setups — which is why it lands roughly midway between hardware TSO and no offload at all in the measurements below, rather than down at the no-offload level. It also explains the MAX_SKB_FRAGS failure mode: a super-packet whose pages are small and numerous can produce a segment needing more than MAX_SKB_FRAGS (default 17) fragments, at which point segmentation fails with -EINVAL and the ratelimited warning skb_segment: too many frags.
Two ownership details at the end of skb_segment() are easy to miss and matter for backpressure. First, __copy_skb_header() deliberately does not copy skb->sk or the destructor — see struct sk_buff for why — so the segments start out unowned. Second, the function then hands ownership to the last segment:
/* Following permits correct backpressure, for protocols using skb_set_owner_w().
* Idea is to tranfert ownership from head_skb to last segment.
*/
if (head_skb->destructor == sock_wfree) {
swap(tail->truesize, head_skb->truesize);
swap(tail->destructor, head_skb->destructor);
swap(tail->sk, head_skb->sk);
}The socket’s write-memory charge is therefore released when the last segment completes transmission, not when the super-packet is freed by the GSO engine. Without this, a socket would see its send buffer freed the moment segmentation finished — long before the bytes were actually on the wire — and TCP Small Queues would stop working. tcp_gso_segment() does the same thing with more care, explaining it in a comment: “The callback to TCP stack will be called at the time last frag is freed at TX completion, and not right now when gso_skb is freed by GSO engine.”
Fixing Up the Headers: tcp_gso_segment()
skb_segment() produces correctly-sized skbs with duplicated headers; those headers are all identical, which means all but the first are wrong. Making them right is the protocol handler’s job. tcp_gso_segment() in net/ipv4/tcp_offload.c (v6.12) is the reference implementation, and reading it is the fastest way to understand exactly what “producing an identical sequence of frames” requires.
struct sk_buff *tcp_gso_segment(struct sk_buff *skb, netdev_features_t features)
{
th = tcp_hdr(skb);
thlen = th->doff * 4;
if (thlen < sizeof(*th))
goto out;
if (unlikely(skb_checksum_start(skb) != skb_transport_header(skb)))
goto out; /* csum_start must point at the TCP header */
if (!pskb_may_pull(skb, thlen))
goto out;
oldlen = ~skb->len;
__skb_pull(skb, thlen);
mss = skb_shinfo(skb)->gso_size;
if (unlikely(skb->len <= mss))
goto out; /* nothing to split */
if (skb_gso_ok(skb, features | NETIF_F_GSO_ROBUST)) {
/* Packet is from an untrusted source, reset gso_segs. */
skb_shinfo(skb)->gso_segs = DIV_ROUND_UP(skb->len, mss);
segs = NULL;
goto out; /* verification only — do NOT segment */
}
copy_destructor = gso_skb->destructor == tcp_wfree;
ooo_okay = gso_skb->ooo_okay;
skb->ooo_okay = 0; /* all but the first must have it clear */
segs = skb_segment(skb, features); /* the data carve */
segs->ooo_okay = ooo_okay; /* restore on the first only */
if (skb_is_gso(segs))
mss *= skb_shinfo(segs)->gso_segs; /* GSO_PARTIAL: segments are multi-MSS */
delta = (__force __wsum)htonl(oldlen + thlen + mss);
skb = segs;
th = tcp_hdr(skb);
seq = ntohl(th->seq);
newcheck = ~csum_fold(csum_add(csum_unfold(th->check), delta));
while (skb->next) {
th->fin = th->psh = 0; /* only the LAST segment ends the data */
th->check = newcheck; /* same delta for every full-size segment */
if (skb->ip_summed == CHECKSUM_PARTIAL)
gso_reset_checksum(skb, ~th->check);
else
th->check = gso_make_checksum(skb, ~th->check);
seq += mss;
if (copy_destructor) { skb->destructor = gso_skb->destructor;
skb->sk = gso_skb->sk;
sum_truesize += skb->truesize; }
skb = skb->next;
th = tcp_hdr(skb);
th->seq = htonl(seq); /* monotonic sequence numbers */
th->cwr = 0; /* only the FIRST segment carries CWR */
}
/* ... final segment: recompute its checksum from its own real length ... */
/* ... transfer socket ownership to the last segment (TCP Small Queues) ... */
return segs;
}The per-segment edits are worth enumerating, because between them they are the correctness contract:
| Field | Rule | Why |
|---|---|---|
th->seq | first segment keeps the original; each subsequent one is previous + mss | the receiver reassembles the byte stream by sequence number |
th->fin | cleared on every segment except the last | only the true end of the stream may signal FIN |
th->psh | cleared on every segment except the last | PSH means “deliver now”; on a mid-stream segment it is noise |
th->cwr | cleared on every segment except the first | the ECN Congestion-Window-Reduced signal must be sent exactly once, and it belongs on the first segment (see SKB_GSO_TCP_ECN) |
th->check | delta-adjusted from the super-packet’s, then completed per segment | see below |
skb->ooo_okay | preserved on the first segment, cleared on the rest | ooo_okay says “this socket has nothing in flight, it is safe to move it to another TX queue”; that is only true before the burst starts |
IPv4 id | incremented per segment, unless SKB_GSO_TCP_FIXEDID | handled by inet_gso_segment(), one layer up |
IPv4 tot_len | rewritten per segment | handled by inet_gso_segment() |
The complete set of per-segment header rewrites for TCP over IPv4, split by which layer performs each. What it shows: producing “an identical sequence of frames” means editing seven distinct header fields with three different rules — monotone for the sequence number, first-only for CWR, last-only for FIN and PSH. The insight: every one of these rules is a place where a hardware TSO engine and the software path must agree exactly, or a packet capture taken with offloads on will differ from one taken with offloads off. When you see a bug report that says “it only reproduces with TSO enabled,” this table is the list of things to check.
The checksum handling repays a slow read. Because the TCP checksum is a one’s-complement sum, changing the length field changes the sum by a computable delta rather than requiring a fresh pass over the bytes. oldlen = ~skb->len is captured before the header is pulled; delta = htonl(oldlen + thlen + mss) is the difference between the super-packet’s total length and one segment’s; and newcheck is the super-packet’s checksum with that delta folded in. Every full-size segment shares the same newcheck because they all have the same length. Only the final segment, which may be short, gets its own delta computed from its actual length. If the device will finish the checksum, gso_reset_checksum() re-establishes CHECKSUM_PARTIAL state on the segment; if not, gso_make_checksum() completes it in software.
Finally, the NETIF_F_GSO_ROBUST branch near the top is the untrusted-input path, and it is the reason __skb_gso_segment() documents that it “may return NULL if the skb requires no segmentation. This is only possible when GSO is used for verifying header integrity.” When a super-packet arrives from a guest via virtio-net or from userspace via tuntap, it is tagged SKB_GSO_DODGY, and its gso_segs is attacker-controlled. If the egress device can offload the packet anyway (skb_gso_ok() with NETIF_F_GSO_ROBUST added), the handler does not split it — it merely recomputes gso_segs from the real length and returns NULL, letting the original skb proceed. The stack thus gets a cheap sanitisation pass on a hostile packet without paying for a segmentation it does not need. qdisc_pkt_len_init() does the same defensive recount, with an explicit comment /* Malicious packet. */ guarding against a header length that exceeds the packet length.
A 64 KB send() Descending to the Driver
Putting the pieces in order, here is one large write making its way to the wire on a NIC that supports TSO, and the same write on one that does not.
sequenceDiagram autonumber participant App as Application participant TCP as tcp_sendmsg / tcp_write_xmit participant IP as ip_queue_xmit / ip_output participant NF as netfilter POSTROUTING participant QD as qdisc — fq, fq_codel, ... participant DEV as validate_xmit_skb participant DRV as driver ndo_start_xmit participant NIC as NIC App->>TCP: send(fd, buf, 65536) Note over TCP: sk_setup_caps() already granted<br/>NETIF_F_GSO; sk_gso_max_size known TCP->>TCP: tcp_tso_segs(): pacing rate / 1024,<br/>plus RTT bonus, floor 2, cap sk_gso_max_segs TCP->>TCP: build ONE skb, 16 page frags,<br/>TCP_SKB_CB tcp_gso_size = 1448 TCP->>TCP: ip_summed = CHECKSUM_PARTIAL,<br/>csum_start → TCP header TCP->>IP: one skb, len 65536, gso_segs 46 IP->>IP: route lookup — ONCE IP->>NF: one skb NF->>NF: conntrack + rules evaluated — ONCE NF->>QD: one skb QD->>QD: qdisc_pkt_len_init(): pkt_len = 65536<br/>+ 45 × 66 = 68,506 shaping bytes QD->>DEV: one skb dequeued DEV->>DEV: netif_skb_features(): six narrowing steps alt NIC advertises TSO and nothing stripped it DEV->>DRV: ONE skb — netif_needs_gso() was false DRV->>NIC: one descriptor set + MSS NIC-->>NIC: silicon emits 46 frames,<br/>fixing seq / len / checksum per frame else a feature was stripped, or ip_summed is unusable DEV->>DEV: skb_gso_segment(): skb_segment()<br/>plus tcp_gso_segment() header fixups DEV->>DRV: 46 skbs, headers copied, pages shared DRV->>NIC: 46 descriptor sets end NIC-->>App: TX completion frees the LAST segment,<br/>whose destructor un-charges sk_wmem_alloc
One 64 KB write from send() to the wire, with both terminations of the final branch. What it shows: every stage above validate_xmit_skb handles exactly one object regardless of which branch is taken, and the two branches differ only in where the 46-way expansion happens and how many descriptor setups the driver performs. The insight: the qdisc step (7) is the subtle one — the queueing layer must shape on what will be on the wire, so it inflates pkt_len by (gso_segs - 1) × hdr_len to account for the 45 sets of Ethernet/IP/TCP headers that do not exist yet. Without that correction a token-bucket shaper would let a TSO’d flow overshoot its configured rate by the header overhead, roughly 4.5% at a 1448-byte MSS. And note the last step: the socket’s memory is released at the final segment’s TX completion, which is what keeps TCP Small Queues honest across the split.
The shaping correction is small enough to quote in full, because it is a piece of the mechanism almost no write-up mentions:
static void qdisc_pkt_len_init(struct sk_buff *skb)
{
const struct skb_shared_info *shinfo = skb_shinfo(skb);
qdisc_skb_cb(skb)->pkt_len = skb->len;
/* To get more precise estimation of bytes sent on wire,
* we add to pkt_len the headers size of all segments
*/
if (shinfo->gso_size && skb_transport_header_was_set(skb)) {
u16 gso_segs = shinfo->gso_segs;
unsigned int hdr_len = skb_transport_offset(skb); /* MAC + network */
if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6)))
hdr_len += __tcp_hdrlen(th); /* + transport */
else if (shinfo->gso_type & SKB_GSO_UDP_L4)
hdr_len += sizeof(struct udphdr);
if (unlikely(shinfo->gso_type & SKB_GSO_DODGY)) {
int payload = skb->len - hdr_len;
if (payload <= 0)
return; /* Malicious packet. */
gso_segs = DIV_ROUND_UP(payload, shinfo->gso_size);
}
qdisc_skb_cb(skb)->pkt_len += (gso_segs - 1) * hdr_len;
}
}UDP Segmentation Offload, UDP_SEGMENT, and QUIC
The UDP side has two distinct and constantly-confused offloads, and the kernel’s own header comment draws the line: NETIF_F_GSO_UDP_L4_BIT is annotated “... UDP payload GSO (not UFO)”.
- UFO — UDP Fragmentation Offload (
SKB_GSO_UDP, bit 16). The old mechanism. It takes one oversized UDP datagram and produces IP fragments of it: one logical datagram, split at the IP layer, with only the first fragment carrying a UDP header. The v6.12 documentation is explicit that this is finished: “UFO is deprecated: modern kernels will no longer generate UFO skbs, but can still receive them from tuntap and similar devices. Offload of UDP-based tunnel protocols is still supported.” - USO — UDP Segmentation Offload (
SKB_GSO_UDP_L4, bit 17). The live mechanism, and the UDP analogue of TSO. It takes one large payload and produces N independent UDP datagrams, each with its own complete UDP header. Willem de Bruijn, who wrote it, stated the distinction in the commit message: “UDP GSO is not UFO. UFO fragments a single large datagram. GSO splits a large payload into a number of discrete UDP datagrams.”
Both SKB_GSO_UDP_L4 and the UDP_SEGMENT socket option first appear in Linux 4.18, confirmed by checking include/linux/skbuff.h and include/uapi/linux/udp.h at the v4.17, v4.18 and v4.19 tags: absent at v4.17, present from v4.18.
USO matters enormously for QUIC and therefore for HTTP/3. QUIC runs over UDP, so a QUIC sender that wants to put a megabyte on the wire must otherwise issue hundreds of sendmsg() calls, one per datagram, each paying a full syscall plus a full stack traversal. UDP_SEGMENT collapses that into one call carrying one large buffer plus “cut it every N bytes.” De Bruijn’s own benchmark in the commit that added the socket option shows exactly this: bytes per syscall rose from 1470 to 61,818.
The userspace API
UDP_SEGMENT is socket option 103 at level SOL_UDP (include/uapi/linux/udp.h, v6.12; UDP_GRO is 104, its receive-side counterpart). It takes a u16 segment size and can be set two ways: with setsockopt() for the lifetime of the socket, or as a cmsg of type UDP_SEGMENT on an individual sendmsg() — the per-call form is __udp_cmsg_send() in net/ipv4/udp.c. The value is the payload size of each output datagram, excluding the 8-byte UDP header; udp_send_skb() computes datalen = len - sizeof(*uh) and then gso_segs = DIV_ROUND_UP(datalen, cork->gso_size). If the total payload is not an exact multiple, the last datagram is short.
int fd = socket(AF_INET, SOCK_DGRAM, 0);
uint16_t seg = 1200; /* one QUIC datagram's payload */
setsockopt(fd, SOL_UDP, UDP_SEGMENT, &seg, sizeof(seg));
/* one syscall, one skb, 50 datagrams of 1200 bytes on the wire */
sendto(fd, buf, 50 * 1200, 0, (struct sockaddr *)&dst, sizeof(dst));The four guards
udp_send_skb() (net/ipv4/udp.c, v6.12) enforces four preconditions before it will build a USO skb. Each has a distinct errno and a distinct reason, and knowing them turns a mystifying EINVAL into a one-line diagnosis:
if (cork->gso_size) {
const int hlen = skb_network_header_len(skb) + sizeof(struct udphdr);
if (hlen + cork->gso_size > cork->fragsize) return -EINVAL; /* 1 */
if (datalen > cork->gso_size * UDP_MAX_SEGMENTS) return -EINVAL; /* 2 */
if (sk->sk_no_check_tx) return -EINVAL; /* 3 */
if (is_udplite || dst_xfrm(skb_dst(skb))) return -EIO; /* 4 */
if (datalen > cork->gso_size) {
skb_shinfo(skb)->gso_size = cork->gso_size;
skb_shinfo(skb)->gso_type = SKB_GSO_UDP_L4;
skb_shinfo(skb)->gso_segs = DIV_ROUND_UP(datalen, cork->gso_size);
goto csum_partial; /* don't checksum here — segmentation will */
}
}- The segment must fit the path MTU.
cork->fragsizeis the MTU for this route; headers plus one segment must not exceed it. This is the guard that makesUDP_SEGMENTnot a way to send jumbo datagrams — every output datagram is a normal-sized datagram. - At most
UDP_MAX_SEGMENTSsegments per call, whereUDP_MAX_SEGMENTSis(1 << 7)= 128 at v6.12 (include/linux/udp.h). It was raised from 64 to 128 in 2024 for exactly the reason the commit message gives: “In general the USO should not be more restrictive than TSO. … with the minimal meaningful mss of 536 the maximal UDP packet will be divided to ~120 segments.” The commit also notes thatUDP_MAX_SEGMENTS“is kernel-only define and not available to user mode socket applications” — so userspace must discover the limit by hitting it. - UDP checksums must not be disabled.
SO_NO_CHECK/UDP_NO_CHECK6_TXand USO are mutually exclusive, because the segmentation path assumes it is completing a partial checksum. - No UDP-Lite and no IPsec. A route with an
xfrmtransform attached returnsEIO, because the transform must see the final datagrams.
Notice the last three lines: gso_size, gso_type and gso_segs are stamped, and then the code jumps over the ordinary checksum computation with the comment “Don’t checksum the payload, skb will get segmented.” This is CHECKSUM_PARTIAL in its purest form — the checksum for each output datagram simply does not exist yet, and cannot.
Version-dated behaviour change. Until Linux 6.11, a fifth guard rejected
UDP_SEGMENTon any egress device that did not advertise transmit checksum offload, which madeUDP_SEGMENTreturnEIOontun/tapdevices. Commit10154dbded6d(“udp: Allow GSO transmit from devices with no checksum offload”, Jakub Sitnicki, merged 2024-06-28) removed it, on the grounds that “the GSO stack has a software fallback for checksum calculation, which we can use. This way we don’t forceUDP_SEGMENTusers to handle theEIOerror and implement a segmentation fallback.” The commit warns of the cost: without checksum offload “the packet payload is read twice: first during thesendmsgsyscall when copying data from user memory, and then in the GSO stack for checksum computation.” v6.12, the version pinned here, includes this change.
The software path
__udp_gso_segment() in net/ipv4/udp_offload.c mirrors tcp_gso_segment() closely: the same -EINVAL on a mismatched csum_start, the same SKB_GSO_DODGY recount-and-return-NULL verification path, the same skb_segment() call, the same destructor transfer to the last segment. Its per-segment header work is much simpler than TCP’s, because UDP datagrams are independent: there are no sequence numbers to advance and no flags to clear — only the len field and the checksum to rewrite per datagram. One v6.12-specific guard is worth quoting, because it is a real-hardware limitation encoded in the stack:
/* We don't know if egress device can segment and checksum the packet
* when IPv6 extension headers are present. Fall back to software GSO.
*/
if (gso_skb->ip_summed != CHECKSUM_PARTIAL)
features &= ~(NETIF_F_GSO_UDP_L4 | NETIF_F_CSUM_MASK);flowchart TB subgraph UFO["UFO — SKB_GSO_UDP, bit 16, DEPRECATED"] U1["one 4000-byte UDP datagram"] --> U2["IP frag 1: IP hdr + UDP hdr + 1472 B<br/>IP frag 2: IP hdr + 1480 B (no UDP hdr)<br/>IP frag 3: IP hdr + 1048 B (no UDP hdr)"] U2 --> U3["receiver reassembles at IP layer<br/>→ ONE recvmsg() of 4000 bytes<br/>lose one fragment, lose the datagram"] end subgraph USO["USO — SKB_GSO_UDP_L4, bit 17, since v4.18"] S1["one sendmsg() of 3600 bytes<br/>UDP_SEGMENT = 1200"] --> S2["datagram 1: IP hdr + UDP hdr + 1200 B<br/>datagram 2: IP hdr + UDP hdr + 1200 B<br/>datagram 3: IP hdr + UDP hdr + 1200 B"] S2 --> S3["receiver gets THREE independent datagrams<br/>→ three recvmsg() of 1200 bytes<br/>(or one, if UDP_GRO is set)<br/>lose one, keep the other two"] end
UFO and USO on the same axis, which is the only way the difference stops being confusing. What it shows: UFO produces IP fragments of one datagram; USO produces N whole datagrams. The receive-side consequence is drawn deliberately — UFO’s output is one recvmsg(), USO’s is N. The insight: these are not two spellings of the same idea and they are not interchangeable. UFO changes the number of IP packets while keeping one datagram; USO changes the number of datagrams. A protocol like QUIC, which needs many small independent datagrams and cannot tolerate the all-or-nothing loss behaviour of IP fragmentation, wants USO and would be actively harmed by UFO — which is a large part of why UFO was deprecated and USO written.
Tunnels, GSO_PARTIAL, and Encapsulation
Encapsulated traffic — VXLAN, GENEVE, GRE, IPIP, SIT — puts a second full header stack in front of the packet. The v6.12 documentation explains the representation: “Currently only two levels of headers are supported. The convention is to refer to the tunnel headers as the outer headers, while the encapsulated data is normally referred to as the inner headers,” accessed through parallel skb_inner_*_header() accessors. Dedicated GSO types identify each encapsulation shape: SKB_GSO_IPXIP4, SKB_GSO_IPXIP6, SKB_GSO_GRE, SKB_GSO_GRE_CSUM, SKB_GSO_UDP_TUNNEL and SKB_GSO_UDP_TUNNEL_CSUM, with SKB_GSO_TUNNEL_REMCSUM for remote checksum offload.
The problem is that segmenting an encapsulated super-packet in software is expensive — every segment needs both header stacks rewritten — while very few NICs can do tunnel-aware TSO. GSO_PARTIAL is the compromise, and the documentation describes it as “a hybrid between TSO and GSO. What it effectively does is take advantage of certain traits of TCP and tunnels so that instead of having to rewrite the packet headers for each segment only the inner-most transport header and possibly the outer-most network header need to be updated. This allows devices that do not support tunnel offloads or tunnel offloads with checksum to still make use of segmentation.”
The mechanics are visible in skb_segment(). When NETIF_F_GSO_PARTIAL is in play, the function computes partial_segs = len / mss and then multiplies mss by that count, so the “segments” it produces are each a multiple of the real MSS. It then re-stamps every output skb as still being a GSO skb:
if (partial_segs) {
int type = skb_shinfo(head_skb)->gso_type;
unsigned short gso_size = skb_shinfo(head_skb)->gso_size;
type |= (features & NETIF_F_GSO_PARTIAL) / NETIF_F_GSO_PARTIAL * SKB_GSO_PARTIAL;
type &= ~SKB_GSO_DODGY;
for (iter = segs; iter; iter = iter->next) {
skb_shinfo(iter)->gso_size = gso_size;
skb_shinfo(iter)->gso_segs = partial_segs;
skb_shinfo(iter)->gso_type = type;
SKB_GSO_CB(iter)->data_offset = skb_headroom(iter) + doffset;
}
if (tail->len - doffset <= gso_size)
skb_shinfo(tail)->gso_size = 0;
else if (tail != segs)
skb_shinfo(tail)->gso_segs = DIV_ROUND_UP(tail->len - doffset, gso_size);
}So the software does the outer work — one set of tunnel headers per output skb — and hands the hardware skbs that are still marked as needing an inner-MSS split. __skb_gso_segment() guards the whole thing with a capability check, only claiming partial support when it actually helps:
/* Only report GSO partial support if it will enable us to
* support segmentation on this frame without needing additional work.
*/
if (features & NETIF_F_GSO_PARTIAL) {
netdev_features_t partial_features = NETIF_F_GSO_ROBUST;
partial_features |= dev->features & dev->gso_partial_features;
if (!skb_gso_ok(skb, features | partial_features))
features &= ~NETIF_F_GSO_PARTIAL;
}The documentation names the one field that cannot be handled this way: “The one exception to this is the outer IPv4 ID field. It is up to the device drivers to guarantee that the IPv4 ID field is incremented in the case that a given header does not have the DF bit set.”
flowchart LR A["encapsulated super-packet<br/>outer Eth/IP/UDP + VXLAN<br/>inner Eth/IP/TCP + 64 KB payload<br/>gso_size = inner MSS"] --> B{"does the NIC advertise<br/>tunnel-aware TSO?<br/>(tx-udp_tnl-segmentation)"} B -->|yes| C["full hardware TSO<br/>silicon rewrites BOTH header stacks<br/>per emitted frame"] B -->|"no, but tx-gso-partial<br/>and plain TSO"| D["GSO_PARTIAL:<br/>software emits a few skbs, each a<br/>MULTIPLE of the inner MSS,<br/>each with its own outer headers"] D --> E["each of those skbs is STILL a GSO skb<br/>gso_size = inner MSS, gso_segs = partial_segs<br/>hardware does the inner split"] B -->|"neither"| F["full software GSO:<br/>every segment gets both<br/>header stacks rewritten by the CPU"] C --> W["frames on the wire"] E --> W F --> W
The three ways an encapsulated super-packet can reach the wire. What it shows: GSO_PARTIAL is a genuine middle rung — the CPU pays for the outer headers only, and the NIC’s ordinary TCP segmentation engine does the inner split it already knows how to do. The insight: this is why ethtool -k on a modern NIC shows tx-gso-partial: on alongside a short list of tunnel offloads. The device does not need to understand VXLAN at all; it only needs to be told “split this at 1448 and fix the innermost TCP header,” and the kernel arranges for everything outside that to already be correct. Without GSO_PARTIAL, overlay networking on hardware without tunnel TSO falls all the way to the rightmost branch, and that gap is large enough to show up as a change in overlay-network throughput between kernel versions.
Feature Flags: How gso_type Maps onto NETIF_F_*
A netdev_features_t is a u64 bitmask of NETIF_F_*_BIT positions. The segmentation feature bits are not scattered through it: they occupy one contiguous run starting at NETIF_F_GSO_SHIFT, and include/linux/netdev_features.h (v6.12) carries a terse instruction above them — /**/NETIF_F_GSO_SHIFT, /* keep the order of SKB_GSO_* bits */. That ordering constraint exists so that testing “can this device do this kind of segmentation?” is a single shift and mask rather than a switch statement:
static inline bool net_gso_ok(netdev_features_t features, int gso_type)
{
netdev_features_t feature = (netdev_features_t)gso_type << NETIF_F_GSO_SHIFT;
/* check flags correspondence */
BUILD_BUG_ON(SKB_GSO_TCPV4 != (NETIF_F_TSO >> NETIF_F_GSO_SHIFT));
BUILD_BUG_ON(SKB_GSO_DODGY != (NETIF_F_GSO_ROBUST >> NETIF_F_GSO_SHIFT));
BUILD_BUG_ON(SKB_GSO_TCP_ECN != (NETIF_F_TSO_ECN >> NETIF_F_GSO_SHIFT));
/* ... sixteen more, one per SKB_GSO_* type ... */
BUILD_BUG_ON(SKB_GSO_FRAGLIST != (NETIF_F_GSO_FRAGLIST >> NETIF_F_GSO_SHIFT));
return (features & feature) == feature;
}Nineteen BUILD_BUG_ON() assertions — one per GSO type — make the correspondence a compile-time invariant. Add a new SKB_GSO_* type without adding the matching NETIF_F_GSO_*_BIT in the same position and the kernel will not build.
flowchart LR A["skb_shinfo(skb)->gso_type<br/>e.g. SKB_GSO_TCPV4 OR SKB_GSO_TCP_ECN<br/>= bit 0 OR bit 2 = 0x05"] -->|"shift left by NETIF_F_GSO_SHIFT"| B["0x05 shifted into the<br/>feature-bit run<br/>= NETIF_F_TSO OR NETIF_F_TSO_ECN"] C["dev->features<br/>(u64 NETIF_F_* bitmask)"] --> D{"(features & feature)<br/>== feature ?"} B --> D D -->|yes| E["net_gso_ok() true<br/>→ hardware can do<br/>ALL requested types"] D -->|no| F["net_gso_ok() false<br/>→ netif_needs_gso() true<br/>→ software split"] G["19 x BUILD_BUG_ON()<br/>SKB_GSO_X == NETIF_F_X >> SHIFT"] -.->|"enforces the<br/>bit-for-bit alignment<br/>at compile time"| B
How a per-packet gso_type becomes a per-device capability test. What it shows: the two enumerations are deliberately kept bit-for-bit aligned so that the whole capability check is one shift, one AND and one comparison — and the alignment is enforced by nineteen compile-time assertions rather than by convention. The insight: net_gso_ok() requires every requested bit to be present, not any. A packet asking for TCPv4 segmentation and ECN handling on a device that supports the first but not the second gets a software split, not a partial hardware one. This all-or-nothing rule is why SKB_GSO_TCP_ECN, SKB_GSO_TCP_FIXEDID and SKB_GSO_DODGY — which are modifiers rather than segmentation kinds — each need their own feature bit (tx-tcp-ecn-segmentation, tx-tcp-mangleid-segmentation, tx-gso-robust) for the hardware path to remain reachable.
Feature bits are grouped into named sets, and two of them govern how offloads propagate through virtual devices:
#define NETIF_F_ALL_TSO (NETIF_F_TSO | NETIF_F_TSO6 | \
NETIF_F_TSO_ECN | NETIF_F_TSO_MANGLEID)
/* List of features with software fallbacks. */
#define NETIF_F_GSO_SOFTWARE (NETIF_F_ALL_TSO | NETIF_F_GSO_SCTP | \
NETIF_F_GSO_UDP_L4 | NETIF_F_GSO_FRAGLIST)
/* If one device supports one of these features, then enable them
* for all in netdev_increment_features. */
#define NETIF_F_ONE_FOR_ALL (NETIF_F_GSO_SOFTWARE | NETIF_F_GSO_ROBUST | \
NETIF_F_SG | NETIF_F_HIGHDMA | \
NETIF_F_FRAGLIST | NETIF_F_VLAN_CHALLENGED)
/* If upper/master device has these features disabled, they must be disabled
* on all lower/slave devices as well. */
#define NETIF_F_UPPER_DISABLES NETIF_F_LRO
/* changeable features with no special hardware requirements */
#define NETIF_F_SOFT_FEATURES (NETIF_F_GSO | NETIF_F_GRO)NETIF_F_GSO_SOFTWARE is the set of things the kernel can always do itself, and its membership in NETIF_F_ONE_FOR_ALL is why a bridge or a bond advertises segmentation support if any member port does: netdev_increment_features() unions those bits across members. This is precisely the property Herbert Xu called out when he posted GSO in 2006 — “passing TSO packets through a bridge only works if all constiuents support TSO. With GSO, it provides a fallback so that we may enable TSO for a bridge even if some of its constituents do not support TSO. This provides massive savings for Xen as it uses a bridge-based architecture.” NETIF_F_UPPER_DISABLES is the mirror image and contains exactly one entry, LRO, for the reasons in the next section.
Reading and changing the state with ethtool
ethtool -k prints two levels: a handful of legacy “friendly” names, and beneath each the raw kernel feature strings. The friendly names come from ethtool’s own off_flag_def[] table (common.c in the ethtool userspace tree); the raw strings come from the kernel’s netdev_features_strings[] in net/ethtool/common.c. Knowing which is which prevents a lot of confusion:
ethtool -K flag | ethtool -k group line | Kernel feature string | Kernel bit |
|---|---|---|---|
tso | tcp-segmentation-offload | tx-tcp-segmentation / tx-tcp6-segmentation | NETIF_F_TSO / NETIF_F_TSO6 |
gso | generic-segmentation-offload | tx-generic-segmentation | NETIF_F_GSO |
gro | generic-receive-offload | rx-gro | NETIF_F_GRO |
lro | large-receive-offload | rx-lro | NETIF_F_LRO |
ufo | udp-fragmentation-offload | tx-udp-fragmentation | NETIF_F_GSO_UDP (deprecated) |
| — | — | tx-udp-segmentation | NETIF_F_GSO_UDP_L4 (USO) |
| — | — | tx-gso-partial | NETIF_F_GSO_PARTIAL |
| — | — | tx-udp_tnl-segmentation | NETIF_F_GSO_UDP_TUNNEL |
| — | — | tx-tcp-ecn-segmentation | NETIF_F_TSO_ECN |
| — | — | tx-tcp-mangleid-segmentation | NETIF_F_TSO_MANGLEID |
| — | — | rx-gro-hw | NETIF_F_GRO_HW |
| — | — | rx-udp-gro-forwarding | NETIF_F_GRO_UDP_FWD |
| — | — | tx-gso-list | NETIF_F_GSO_FRAGLIST |
The mapping from the three name spaces a practitioner meets: the short -K flag, the group heading -k prints, and the kernel’s own per-bit string. What it shows: only five offloads have a legacy short flag; the rest — including USO, GSO_PARTIAL and all the tunnel offloads — are addressed by their kernel string. The insight: ethtool -K eth0 tx-udp-segmentation off is the correct way to disable USO; there is no uso short flag and never was. Similarly, ethtool -K eth0 tso off turns off tx-tcp-segmentation and tx-tcp6-segmentation together, which is not always what you want when bisecting an IPv6-only problem.
# Inspect: the group lines come from ethtool, the indented ones from the kernel
$ ethtool -k eth0
tcp-segmentation-offload: on
tx-tcp-segmentation: on # NETIF_F_TSO — hardware TSO, IPv4
tx-tcp-ecn-segmentation: on # NETIF_F_TSO_ECN
tx-tcp-mangleid-segmentation: off # NETIF_F_TSO_MANGLEID
tx-tcp6-segmentation: on # NETIF_F_TSO6 — hardware TSO, IPv6
generic-segmentation-offload: on # NETIF_F_GSO — the software fallback
generic-receive-offload: on # NETIF_F_GRO
large-receive-offload: off [fixed] # NETIF_F_LRO — often absent entirely
tx-udp-segmentation: on # NETIF_F_GSO_UDP_L4 — USO, NOT UFO
tx-gso-partial: on # NETIF_F_GSO_PARTIAL
tx-udp_tnl-segmentation: on # tunnel-aware TSO
# Force the software path while keeping the super-packet abstraction
$ ethtool -K eth0 tso off
# Disable USO only, e.g. to test a QUIC stack's fallback path
$ ethtool -K eth0 tx-udp-segmentation off
# Diagnostic sledgehammer: every segment traverses the full stack
$ ethtool -K eth0 tso off gso off gro offA [fixed] suffix means the driver has placed the bit in dev->hw_features as unchangeable — usually because the hardware genuinely cannot do it. Attempting to set a fixed feature fails rather than silently doing nothing.
(The console block above is an illustrative rendering assembled from the kernel’s own feature strings, not a captured transcript from one specific NIC; the exact set of lines your card prints depends on its driver.)
MSS, MTU, and the Size Ceilings — Including BIG TCP
gso_size is a payload size, not a frame size, and forgetting that is a reliable source of off-by-a-header bugs. The kernel’s own helper spells out the arithmetic:
static unsigned int skb_gso_transport_seglen(const struct sk_buff *skb)
{
const struct skb_shared_info *shinfo = skb_shinfo(skb);
unsigned int thlen = 0;
if (skb->encapsulation) {
thlen = skb_inner_transport_header(skb) - skb_transport_header(skb);
if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6)))
thlen += inner_tcp_hdrlen(skb);
} else if (likely(shinfo->gso_type & (SKB_GSO_TCPV4 | SKB_GSO_TCPV6))) {
thlen = tcp_hdrlen(skb);
} else if (unlikely(skb_is_gso_sctp(skb))) {
thlen = sizeof(struct sctphdr);
} else if (shinfo->gso_type & SKB_GSO_UDP_L4) {
thlen = sizeof(struct udphdr);
}
/* UFO sets gso_size to the size of the fragmentation
* payload, i.e. the size of the L4 (UDP) header is already accounted for.
*/
return thlen + shinfo->gso_size;
}So for TCP, one segment on the wire is mac + ip + tcp_hdrlen + gso_size bytes; for USO it is mac + ip + 8 + gso_size; and for the deprecated UFO the UDP header is already inside gso_size, which is precisely the kind of inconsistency that makes UFO worth avoiding. skb_gso_validate_network_len() and skb_gso_validate_mac_len() build on this to answer “will this super-packet fit an MTU of N once split?”, and both correctly handle the GSO_BY_FRAGS case by walking the frag_list. ip_forward() uses the network-length variant so that a super-packet is not rejected for exceeding an MTU it will never actually hit.
There are then four separate ceilings on how large a super-packet may be, and they exist for different reasons:
| Constant / field | v6.12 value | What it limits |
|---|---|---|
GSO_MAX_SEGS | 65535 | segments per super-packet; a hard consequence of gso_segs being a u16 |
GSO_LEGACY_MAX_SIZE | 65536 | the default dev->gso_max_size; also the hard cap for non-TCP sockets |
GSO_MAX_SIZE | 8 * 65535 = 524,280 | the absolute ceiling netif_set_tso_max_size() will clamp to |
TSO_LEGACY_MAX_SIZE | 65536 | the default dev->tso_max_size — what the hardware is assumed to accept |
TSO_MAX_SEGS | 65535 (U16_MAX) | the default dev->tso_max_segs |
MAX_SKB_FRAGS | CONFIG_MAX_SKB_FRAGS, default 17, range 17–45 | page fragments per skb — the practical limit on super-packet size |
The size ceilings, their v6.12 values, and what each one is really about. What it shows: the limits come in two families — GSO_* are stack-side limits on the abstraction, TSO_* are what a driver declares its silicon can accept, and MAX_SKB_FRAGS is an orthogonal memory-layout limit that binds first in practice. The insight: 17 fragments of 4 KB pages is 68 KB, which is exactly enough for a 64 KB super-packet and not one byte more. That is not a coincidence: the default was chosen to make 64 KB work, and raising the super-packet ceiling above 64 KB is therefore also a fragment-count problem, which is why CONFIG_MAX_SKB_FRAGS exists and why its range tops out at 45.
The default ceilings are installed at device registration (net/core/dev.c, v6.12):
dev->gso_max_size = GSO_LEGACY_MAX_SIZE; /* 65536 */
dev->gso_ipv4_max_size = GSO_LEGACY_MAX_SIZE;
dev->gso_max_segs = GSO_MAX_SEGS; /* 65535 */
dev->gro_max_size = GRO_LEGACY_MAX_SIZE; /* 65536 */
dev->gro_ipv4_max_size = GRO_LEGACY_MAX_SIZE;
dev->tso_max_size = TSO_LEGACY_MAX_SIZE; /* 65536 */
dev->tso_max_segs = TSO_MAX_SEGS;A driver that knows its silicon accepts more calls netif_set_tso_max_size(), which clamps to GSO_MAX_SIZE and, if the new value is smaller than the current GSO ceiling, pulls that down too — so a driver can only ever loosen the hardware limit, never accidentally raise the stack’s above what the device can take. netif_inherit_tso_max() propagates both limits from a lower device to an upper one, which is how bonds, bridges and VLAN interfaces get sane values.
BIG TCP
Raising gso_max_size above 64 KB is the feature known as BIG TCP. The obstacle is not the kernel’s data structures but the wire format: as Corbet explains, “The length of an IP packet is stored in the IP header; for both IPv4 and IPv6, that length lives in a 16-bit field, limiting the maximum packet size to 64KB” (LWN, 2022). IPv6 has a standard escape hatch — RFC 2675 jumbograms, where the 16-bit payload-length field is set to zero and a hop-by-hop option carries a 32-bit length — which is why IPv6 support came first and why v6.12 has two separate device fields, gso_max_size (used for IPv6) and gso_ipv4_max_size.
The motivating arithmetic is the per-packet time budget. At 100 Gb/s with 1538-byte frames a system must handle “over eight-million packets per second. At that rate, CPU has all of about 120ns to do whatever is required to handle each packet, which is not a lot of time; a single cache miss can ruin the entire processing-time budget.” Dumazet’s own measurement with the patch set: “Enabling a packet size of 185,000 bytes increased network throughput by nearly 50% while also reducing round-trip latency significantly.”
Two practical caveats from the same article carry into v6.12. The fragment limit had to rise — “Current kernels limit the number of fragments stored in an SKB to 17, which is sufficient to store a 64KB packet in single-page chunks … the patch set raises the maximum number of fragments (to 45)” — but because “many interface drivers encode assumptions about the maximum number of fragments,” the increase became the build-time CONFIG_MAX_SKB_FRAGS rather than a flat change. And jumbo-packet generation is off by default: “These headers, per the IPv6 standard, are placed immediately after the IP header; that can confuse software that ‘knows’ that the TCP header can be found immediately after the IP header in a packet. The tcpdump utility has some problems in this regard; it also seems that there are a fair number of BPF programs in the wild that contain this assumption. For this reason, jumbo-packet handling is disabled by default.”
Turning it on is a per-device netlink attribute rather than an ethtool feature bit:
# Raise the super-packet ceiling for IPv6 and IPv4 independently
$ ip link set dev eth0 gso_max_size 185000
$ ip link set dev eth0 gro_max_size 185000
$ ip link set dev eth0 gso_ipv4_max_size 185000
$ ip link set dev eth0 gro_ipv4_max_size 185000
$ ip -d link show dev eth0 | grep -o 'gso_max_size [0-9]*'
gso_max_size 185000flowchart TB A["default: dev->gso_max_size = 65536<br/>(GSO_LEGACY_MAX_SIZE)"] --> B["sk_dst_gso_max_size():<br/>non-TCP sockets clamped to 65536<br/>regardless of the device value"] A --> C["TCP socket:<br/>sk_gso_max_size = ceiling - (MAX_TCP_HEADER + 1)"] C --> D["tcp_tso_autosize() may choose<br/>anything up to that"] E["ip link set dev X gso_max_size 185000"] --> F["dev->gso_max_size = 185000<br/>BIG TCP territory"] F --> G{"IPv6?"} G -->|yes| H["RFC 2675 jumbogram:<br/>IPv6 payload_len = 0,<br/>hop-by-hop option carries 32-bit length<br/>→ up to 4 GB in principle"] G -->|no| I["gso_ipv4_max_size,<br/>a separate field (IPv4 has no<br/>standard jumbogram escape)"] F --> J["needs CONFIG_MAX_SKB_FRAGS > 17<br/>(45 stores ~180 KB in 4 KB pages)"] H --> K["measured: 185,000-byte packets<br/>→ about +50% throughput<br/>and lower RTT (Dumazet)"]
The size ceiling as a chain from kernel default to BIG TCP. What it shows: the 64 KB default is a protocol limit inherited from a 16-bit IP length field, not a kernel one, and lifting it needs a wire-format escape hatch (RFC 2675 for IPv6), a bigger fragment budget, and per-device opt-in. The insight: the two separate device fields for IPv4 and IPv6 are the visible scar of that asymmetry, and the non-TCP clamp in sk_dst_gso_max_size() means BIG TCP is, at v6.12, genuinely TCP-only — a UDP socket on a BIG-TCP-configured device still gets 64 KB.
Forwarding, Bridging, and Where Offloads Get Turned Off
A machine that only originates and terminates traffic can use any offload it likes. A machine that forwards — a router, a bridge, a virtualization host, a Kubernetes node — cannot, and the kernel enforces this rather than trusting the operator.
The rule is one-sided and worth stating precisely: transmit-side segmentation offloads are safe when forwarding; receive-side LRO is not. GSO and TSO are safe because they are the inverse of an operation the kernel itself performed and can therefore reproduce exactly. LRO is unsafe because the hardware performed a merge the kernel cannot undo — the gso_type needed to dispatch a segmentation handler was never recorded.
The kernel therefore disables LRO automatically the moment forwarding is enabled. In net/ipv4/devinet.c (v6.12) this happens in three places: when an in_device is created on an interface where forwarding is already on, when the global net.ipv4.ip_forward sysctl is flipped on (inet_forward_change() walks every netdev), and when a per-interface forwarding sysctl is set. The function they all call recurses down the device stack:
void dev_disable_lro(struct net_device *dev)
{
struct net_device *lower_dev;
struct list_head *iter;
dev->wanted_features &= ~NETIF_F_LRO;
netdev_update_features(dev);
if (unlikely(dev->features & NETIF_F_LRO))
netdev_WARN(dev, "failed to disable LRO!\n");
netdev_for_each_lower_dev(dev, lower_dev, iter)
dev_disable_lro(lower_dev);
}The recursion is why NETIF_F_UPPER_DISABLES contains NETIF_F_LRO and nothing else: disabling LRO on a bond or bridge must disable it on every member, or a merged packet would still enter the stack. Installing an XDP program has the same effect — dev_xdp_install() calls both dev_disable_lro() and dev_disable_gro_hw() when a generic XDP program is attached, because XDP expects to see real frames.
If an LRO’d packet nonetheless reaches the forwarding path — for instance because a driver ignored the feature bit — ip_forward() drops it early, before any routing work:
int ip_forward(struct sk_buff *skb)
{
/* ... */
if (skb->pkt_type != PACKET_HOST)
goto drop;
if (unlikely(skb->sk))
goto drop;
if (skb_warn_if_lro(skb))
goto drop; /* ratelimited: "cannot be forwarded while LRO is enabled" */
/* ... */
}The bridge case is subtler and is where GSO earns its keep. A bridge’s feature set is computed by unioning NETIF_F_ONE_FOR_ALL across its ports, and because NETIF_F_GSO_SOFTWARE is in that set, the bridge advertises segmentation even when some ports cannot do it in hardware. A super-packet crossing the bridge is then segmented — or not — per egress port by validate_xmit_skb(), exactly as it would be on a physical device. This is the property that makes host-to-guest and guest-to-guest traffic on a Linux bridge fast: the two endpoints negotiate a large effective MSS, the packets never touch a wire, and no segmentation happens at all.
The remaining hazard is on the receive side of a forwarding node using GRO. GRO is safe, but coalescing packets that will immediately be re-segmented is pure overhead unless the aggregate survives to the transmit side — and for UDP, GRO on a forwarding path was historically not enabled, because a forwarded UDP aggregate is not obviously reversible into the same datagram boundaries. v6.12 exposes this as its own opt-in feature bit, NETIF_F_GRO_UDP_FWD (rx-udp-gro-forwarding), which is in NETIF_F_SOFT_FEATURES_OFF — the set of software features that default to off.
flowchart TB RX["frames arrive on eth0"] --> Q1{"LRO enabled on eth0?"} Q1 -->|"yes, and forwarding off"| L["hardware merges aggressively<br/>gso_size set, gso_type = 0<br/>fine for a pure endpoint"] Q1 -->|"forwarding turned on"| DIS["devinet.c: dev_disable_lro(dev)<br/>recurses to every lower device<br/>NETIF_F_UPPER_DISABLES = NETIF_F_LRO"] Q1 -->|no| G["GRO at NAPI poll:<br/>merges only reversibly,<br/>sets gso_size AND gso_type"] DIS --> G G --> FWD["ip_forward()<br/>skb_warn_if_lro(): gso_size but no gso_type → DROP<br/>skb_gso_validate_network_len() vs egress MTU"] FWD --> TX["egress device:<br/>validate_xmit_skb() re-splits<br/>in hardware or software"] L -.->|"if it ever reaches ip_forward()"| DROP["dropped + ratelimited warning"] TX --> OUT["identical frames back on the wire"]
What happens to a forwarded packet, and the two places the kernel intervenes. What it shows: enabling forwarding actively rewrites device features rather than merely warning, and there is a second, packet-level backstop in ip_forward() for anything that slips through. The insight: the super-packet abstraction is designed to survive a forwarding hop end to end — arrive as one big skb from GRO, leave as one big skb into GSO/TSO — and LRO is excluded from that design not because it is slow but because it breaks the round trip. This is also why a router or a Kubernetes node showing rx-lro: on is a configuration to investigate, and why “we turned on ip_forward and LRO turned itself off” is expected behaviour rather than a bug.
The Measured Win
The performance argument for segmentation offload is usually made with adjectives. It does not have to be: there are two public measurements from the people who wrote the code, taken eleven years apart, and they agree on the shape of the answer.
Herbert Xu, 2006, introducing GSO. Measuring through the loopback device — chosen because it “is a fairly good approximation of an SG-capable NIC” — at an MTU of 1500, throughput went from 3061.05 Mb/s to 3598.17 Mb/s, a 17.5% improvement, from software segmentation alone. He was careful to note that the measured gain understates the transmit-side saving: “The actual saving in transmission cost is in fact a lot more than that as the majority of the time here is spent on the RX side which still has to deal with 1500-byte packets.” He also measured the worst case — a NIC without scatter-gather, driven by write(2), so the data is copied twice — and found “the cost of the extra copy is mostly offset by the reduction in the cost of going through the networking stack.”
Willem de Bruijn, 2018, introducing UDP_SEGMENT. This one is more useful, because it measures five configurations under one methodology: perf stat -a -C 12 -e cycles ./udpgso_bench_tx -C 12 -4 -D "$DST" -l 4, with both the receive path and the benchmark pinned to a single core.
| Configuration | Throughput | Messages/s | Cycles measured | Derived cycles per MB | Relative efficiency |
|---|---|---|---|---|---|
| TCP with TSO | 3197 MB/s | 54,232 | 6,457,754,262 | ≈ 2.02 M | 7.5× |
| TCP with software GSO | 1765 MB/s | 29,939 | 11,203,021,806 | ≈ 6.35 M | 2.4× |
| TCP with neither | 739 MB/s | 12,548 | 11,205,483,630 | ≈ 15.16 M | 1.0× (baseline) |
| UDP, no offload | 876 MB/s | 14,873 | 11,205,777,429 | ≈ 12.79 M | 1.2× |
UDP with UDP_SEGMENT | 2139 MB/s | 36,282 | 11,204,374,561 | ≈ 5.24 M | 2.9× |
Segmentation offload measured as CPU cost per unit of data, from the UDP_SEGMENT commit message. The throughput, message-rate and cycle columns are verbatim from the commit; the last two columns are derived here by dividing cycles by megabytes and normalising against the no-offload TCP row. What it shows: relative to no offload at all, software GSO costs about 2.4× less CPU per byte and hardware TSO about 7.5× less — so GSO recovers roughly a third of the distance to TSO, and TSO is about 3.1× more efficient again than GSO. On the UDP side, UDP_SEGMENT is worth about 2.4× over ordinary sendmsg(). The insight: these are not three points on a line — hardware TSO is in a different class, because it removes the per-segment skb allocation and the per-segment driver descriptor setup as well as the per-segment stack traversal, whereas software GSO only removes the last of those.
Restating the middle column as “CPU per gigabit,” which is how capacity planning usually needs it: 1 Gb/s is 125 MB/s, so at these efficiencies a single stream costs roughly 0.25 GHz of a core per Gb/s with TSO, 0.79 GHz/Gb/s with software GSO, and 1.9 GHz/Gb/s with neither. On a 3 GHz core that is the difference between one core sustaining about 12 Gb/s, about 3.8 Gb/s, and about 1.6 Gb/s.
Two honest caveats. First, the TSO row consumed markedly fewer total cycles than the others (6.46 G against ~11.2 G), which means that run did not saturate the core — so its cycles-per-megabyte is a genuine efficiency figure but the four rows were not all measured under identical saturation. Second, the message-rate column shows the other half of the win, which the cycle counts partly hide: UDP_SEGMENT raised bytes per syscall from 1470 to 61,818, a 42× reduction in system-call count. For a QUIC implementation, that syscall reduction is often the dominant effect.
Uncertain
Verify: whether the efficiency ratios above still hold on current hardware and kernels. Reason: the numbers are from 2006 and 2018 and were measured on the hardware of their day; per-packet costs have moved since (page-pool recycling,
build_skb()drivers, XDP, and BIG TCP all change the constants), and no equally rigorous public re-measurement against a v6.12-era kernel was located while writing this note. To resolve: re-runtools/testing/selftests/net/udpgso_bench_txunderperf stat -e cycleson a v6.12 kernel with and withouttso/gso/tx-udp-segmentation, on the hardware in question. Treat the ratios as order-of-magnitude guidance, not as a specification. uncertain
Failure Modes and How to Diagnose
skb_warn_bad_offload in dmesg. The kernel dumps the skb and issues a WARN(1, "%s: caps=(%pNF, %pNF)\n", ...) naming the driver and printing both the device’s feature mask and the socket’s sk_route_caps. It is reached from three places in v6.12: gso_features_check() when gso_size is set but gso_type is zero; skb_checksum_help() when it is asked to checksum a GSO skb (which should never happen, since GSO skbs must be segmented first); and __skb_gso_segment() when segmentation completed but the checksum state is still wrong. All three mean some component built inconsistent offload metadata — a driver, a tunnel, a virtio guest, or a packet-injection tool. This is a correctness bug, not a tuning knob; the two feature masks in the message are the diagnostic, because they say what the device claimed it could do.
skb_segment: too many frags. A ratelimited net_warn_ratelimited() from skb_segment(), followed by -EINVAL and a dropped packet. It means one output segment needed more than MAX_SKB_FRAGS page fragments — typically because the super-packet was assembled from many small fragments (a zero-copy path, a guest, or an unusual sendmsg() iovec) rather than whole pages. Raising CONFIG_MAX_SKB_FRAGS addresses it at the cost of a larger skb_shared_info on every packet in the system.
received packets cannot be forwarded while LRO is enabled. Ratelimited, from __skb_warn_lro_forwarding(), and always accompanied by a silently dropped packet. It means a driver delivered an LRO-merged skb on a path that is forwarding. Fix with ethtool -K <dev> lro off, and check why dev_disable_lro() did not already do it — normally that means forwarding was enabled by a route or namespace path that did not go through devinet.c, or the driver ignores the feature bit.
Throughput collapse and a core pinned in softirq after ethtool -K ... gso off tso off. Expected: every segment now traverses the full stack. The diagnostic value is in the inverse direction — if disabling offloads makes a corruption or capture problem go away, the offload (or a buggy NIC) was implicated. Disabling gso alone while leaving tso on is rarely useful, since sk_setup_caps() re-adds NETIF_F_GSO for TCP sockets regardless; disabling tso alone is the more informative experiment, because it isolates the hardware path while keeping the super-packet abstraction.
Packet captures show frames far larger than the MTU. A tcpdump on the sending host routinely shows “TCP segments” of 30 or 60 KB. This is not a bug and not evidence of a wrong MTU: the AF_PACKET tap sits above the segmentation point, so it observes the super-packet, not the wire frames. The same capture taken on the receiving host, or on a mirror port, shows MSS-sized frames. If you need the sender’s real wire frames, disable tso and gso for the duration of the capture — accepting that you have changed the thing you are measuring. Conversely, oversized frames in a capture on the receive side mean GRO or LRO merged them before the tap.
EINVAL from sendmsg() on a UDP_SEGMENT socket. Four distinct causes, enumerated in the USO section above: the segment plus headers exceeds the path MTU, the payload needs more than 128 segments, transmit checksums are disabled on the socket, or (as EIO) the socket is UDP-Lite or the route carries an IPsec transform. The first is by far the most common, and it usually means the application computed its segment size against the interface MTU rather than the path MTU discovered for that destination.
Unexpectedly small TSO bursts. Usually not an offload problem at all but a pacing one: tcp_tso_autosize() derives the burst from sk_pacing_rate, so a connection that is congestion-limited, or one with a large min_rtt, deliberately builds smaller super-packets. ss -ti shows the pacing rate and minimum RTT per socket, and net.ipv4.tcp_min_tso_segs (default 2) sets the floor.
Offloads silently unused on an encapsulated path. netif_skb_features() masks against dev->hw_enc_features for any skb with skb->encapsulation set, and that mask is typically much smaller than dev->features. There is no warning. The symptom is CPU time in skb_segment() and tcp_gso_segment() on a host doing overlay networking; perf top is the tool, and ethtool -k will show whether tx-udp_tnl-segmentation and tx-gso-partial are actually on.
Common Misunderstandings
“GSO is just TSO done in software, so it costs a data copy.” Only on a non-scatter-gather path. On the normal path skb_segment() copies headers and references payload pages; the cost is allocations, atomic page references and driver descriptor setups, not bytes.
“TSO and GSO are alternatives; you pick one.” They are layers. sk_setup_caps() gives every TCP socket NETIF_F_GSO unconditionally, and the documentation makes the software offload a precondition for enabling a hardware one. TSO is an optimisation applied at the last moment to a packet that GSO could always have handled.
“GRO is the receive-side name for GSO, and LRO is the receive-side name for TSO.” The first half is nearly right; the second is wrong in the way that matters. GRO and GSO are inverses by design and the documentation states the round-trip requirement explicitly. LRO is not the receive-side TSO, because TSO’s output is exactly reproducible and LRO’s input is not recoverable — LRO sets gso_size without gso_type, and the kernel disables it whenever forwarding is on.
“UFO and UDP GSO are the same thing.” They are not, and the kernel comment says so in four words: “UDP payload GSO (not UFO).” UFO makes IP fragments of one datagram; USO makes N datagrams. UFO is deprecated for generation; USO is what QUIC uses.
“gso_size is the frame size.” It is the L4 payload size. Segment length on the wire is gso_size plus the L2, L3 and L4 headers — except for UFO, where the UDP header is already counted inside it.
“If ethtool -k says TSO is on, my packets are being segmented by the NIC.” Six independent narrowing steps run per packet in netif_skb_features(), and encapsulation, VLAN tags, checksum capability, fragment counts, size ceilings and the driver’s own ndo_features_check() can each demote the packet to a software split without logging anything.
“Turning off transmit checksum offload only affects checksums.” harmonize_features() clears NETIF_F_CSUM_MASK and NETIF_F_GSO_MASK in the same statement, and the documentation says “TSO is normally disabled if the Tx checksum offload for a given device is disabled.” The two knobs are not independent.
“A super-packet is as big as the device allows.” For TCP it is sized by tcp_tso_autosize() from the pacing rate and RTT, floored at tcp_min_tso_segs and only then capped by sk_gso_max_size and sk_gso_max_segs.
See Also
- struct sk_buff — the packet structure that becomes a super-packet; the
ip_summed/csum_start/csum_offsettriple, thecb[]scratch area that holdsTCP_SKB_CB->tcp_gso_size, and what__copy_skb_header()does and does not transfer - skb_shared_info and Paged Fragments — the trailer that holds
gso_size,gso_segsandgso_type, and thefrags[]array that segmentation shares by reference - Generic Receive Offload — the receive-side inverse, in depth
- Checksum Offloads —
CHECKSUM_PARTIALand the rest of theip_summedstate machine that TSO depends on - The Network Transmit Path — where
netif_skb_features(),validate_xmit_skb()andskb_gso_segment()sit in the egress pipeline - Queueing Disciplines qdisc — the layer whose
pkt_lenaccounting has to predict the post-segmentation wire size - Byte Queue Limits and Buffer Bloat — the other TX-side mechanism bounding the driver ring, and TCP Small Queues, whose correctness depends on the destructor transfer described above
- The TCP Protocol in Linux — the transport that builds TCP super-packets, and where pacing and congestion control set their size
- IP Routing Decision and Forwarding — the lookup that a super-packet pays for once instead of
gso_segstimes - The Netfilter Framework and Hooks — the hook points that likewise see one packet instead of many
- Drawing Wire Formats with Mermaid Packet Diagrams — the house style for the
packet-betadiagram above - MOC: Linux Networking Stack MOC (§5, The Transmit Path and Segmentation Offloads)