Checksum Offloads
Every Internet packet carries a checksum: a 16-bit one’s-complement sum that the receiver recomputes to detect corruption in transit. Computing it means touching every byte of the packet, which is precisely the work a CPU least wants to do at line rate — and it is arithmetic so simple that a few thousand gates on a Network Interface Card (NIC) can do it while the bytes stream past. Checksum offload is the negotiated hand-off of that arithmetic between the kernel and the card, and the entire negotiation is carried in one two-bit field of the packet buffer:
skb->ip_summed, which takes the valuesCHECKSUM_NONE,CHECKSUM_UNNECESSARY,CHECKSUM_COMPLETE, andCHECKSUM_PARTIAL. The single fact that makes this topic hard is that the same four constants mean different things on receive and on transmit — on receiveip_summedis a report from the device about work it already did; on transmit it is a request to the device for work still to do (include/linux/skbuff.h, “DOC: skb checksums”, v6.12). This note is the computation side of that story: the one’s-complement algorithm and the properties that make it offloadable, what the hardware is actually asked to compute, thecsum_start/csum_offsetcontract a driver must honour, how the stack validates a checksum on receive, why encapsulation is where all of this breaks, and why a broken offload looks to userspace like random data corruption rather than like a networking bug. Everything is verified against Linux 6.12, a maintained long-term-support branch (mainline is in the 7.x series as of 2026-09-04); anything newer is dated where it appears.
Scope — and the boundary with the sibling notes
Three notes in this vault touch
ip_summedand they divide the subject deliberately:
- struct sk_buff owns the fields: where
ip_summed,csum,csum_start,csum_offset,csum_level,csum_valid,csum_complete_swandcsum_not_inetlive in the structure, how they are packed, and how they are copied by a clone. It does not explain the arithmetic.- Segmentation Offloads GSO TSO owns segmentation:
gso_type,gso_size,skb_segment(), the per-segment header fix-ups,netif_needs_gso(), and the per-packet feature harmonisation that runs on the egress path. It explains why TSO cannot exist withoutCHECKSUM_PARTIAL, and stops there.- This note owns the checksum itself: the one’s-complement algorithm and its exploitable properties, what silicon actually computes, the
csum_start/csum_offsetcontract, receive-side validation and theCHECKSUM_COMPLETE-versus-CHECKSUM_UNNECESSARYquestion, the device feature flags and their configuration-time dependency graph, tunnelling (Local and Remote Checksum Offload), and diagnosis.Where the two must meet — the fact that a segmenting agent is necessarily also a checksumming agent — both notes state it and link to each other. Nothing here re-derives segmentation.
Mental Model: One Field, Two Directions
The whole machinery hangs on skb->ip_summed, a two-bit field of struct sk_buff (__u8 ip_summed:2;) whose four possible values are simply the integers 0 through 3 (skbuff.h, v6.12):
/* Don't change this without changing skb_csum_unnecessary! */
#define CHECKSUM_NONE 0
#define CHECKSUM_UNNECESSARY 1
#define CHECKSUM_COMPLETE 2
#define CHECKSUM_PARTIAL 3That comment is not decoration. skb_csum_unnecessary() — the predicate the whole receive path uses to ask “do I need to verify this?” — depends on the ordering of the constants, so renumbering them would silently break validation. It is the first hint that this is a field read by far more code than obviously touches it.
The direction rule is the thing to internalise before reading any of the code. On the receive path, a driver fills in ip_summed to describe what its device already did to the packet: nothing, a protocol-aware verification, or a protocol-agnostic running sum handed back for the stack to finish. On the transmit path, the stack fills in ip_summed to tell the driver what it must arrange before the frame hits the wire: nothing (the bytes are already final), or “sum this byte range and write the answer here.” A reader who tracks only the constant and not the direction will reach exactly the wrong conclusion about half the networking code — and, worse, so will a driver author.
flowchart TB subgraph RXD["RECEIVE — the device reports"] direction TB RXN["CHECKSUM_NONE = 0<br/>'I did nothing.'<br/>full checksum is in the packet,<br/>unverified. skb->csum undefined."] RXU["CHECKSUM_UNNECESSARY = 1<br/>'I parsed the headers and<br/>the checksum was good.'<br/>skb->csum still undefined.<br/>csum_level = verified layers - 1"] RXC["CHECKSUM_COMPLETE = 2<br/>'Here is the one's-complement sum<br/>of every byte I received,<br/>in skb->csum. I parsed nothing.'"] RXP["CHECKSUM_PARTIAL = 3<br/>'This never had a checksum.'<br/>from a local VM / veth peer,<br/>or set by GRO / remote csum offload"] end subgraph TXD["TRANSMIT — the stack requests"] direction TB TXN["CHECKSUM_NONE / _UNNECESSARY<br/>'The bytes in the packet<br/>are already final. Do nothing.'"] TXP["CHECKSUM_PARTIAL<br/>'Sum from head+csum_start to the end,<br/>fold, and store the 16 bits at<br/>head+csum_start+csum_offset.'"] TXC["CHECKSUM_COMPLETE<br/>never valid on output.<br/>A driver seeing it MUST treat<br/>the packet as CHECKSUM_NONE."] end RXC -.->|"stack folds pseudo-header in,<br/>compares against zero"| VERIFY["verdict: good / bad"] RXU -.->|"stack skips verification entirely"| VERIFY RXN -.->|"stack sums the whole payload<br/>in software"| VERIFY TXP ==>|"NETIF_F_HW_CSUM"| HW["NIC computes and inserts"] TXP ==>|"device cannot"| SW["skb_checksum_help()<br/>CPU sums the payload"]
The two-directional meaning of ip_summed in Linux 6.12. What it shows: four constants, eight meanings — the receive column is a report about the past, the transmit column is a request about the future, and only one value, CHECKSUM_PARTIAL, actually moves work onto the NIC. The insight to take: CHECKSUM_COMPLETE and CHECKSUM_PARTIAL are near-mirror images. COMPLETE says “I summed everything and understood nothing”; PARTIAL says “I understood everything and summed nothing.” Both are protocol-agnostic contracts with silicon, which is exactly why they are the two states that scale to protocols the hardware has never heard of — and why CHECKSUM_UNNECESSARY, the protocol-aware state, is the one that ossifies.
The rest of this note unpacks that diagram, starting one level below it: with the arithmetic that makes any of it possible.
The Internet Checksum, From First Principles
The checksum that TCP, UDP and the IPv4 header all use is specified once and reused everywhere. RFC 1071, Computing the Internet Checksum (Braden, Borman and Partridge, September 1988) states it in three steps:
- Adjacent octets are paired into 16-bit integers, and the one’s-complement sum of those integers is formed. One’s-complement addition means ordinary addition with an end-around carry: whenever the sum overflows out of bit 15, the carry is added back into bit 0.
- To generate a checksum, the checksum field itself is first cleared to zero, the sum is computed over the bytes concerned, and the one’s complement (bitwise NOT) of that sum is written into the checksum field.
- To check a checksum, the same sum is computed over the same bytes, this time including the checksum field. If the result is all ones —
0xFFFF, which is negative zero in one’s-complement arithmetic — the check passes.
Two definitions before the worked example. A fold is the reduction of a wider accumulator (32 or 64 bits) down to 16 bits by repeatedly adding the high half into the low half; it is how the end-around carries get paid off in bulk at the end instead of one at a time. A pseudo-header is a small block of fields that the transport checksum must cover but that do not live contiguously with the transport header: for IPv4/UDP it is the source address, destination address, a zero octet, the protocol number, and the UDP length (RFC 768, Postel, 28 August 1980). Its purpose is to bind the transport segment to the addresses it was sent between, so a misdelivered datagram fails its checksum.
A worked example, computed end to end
Take a UDP datagram from 192.0.2.1 to 198.51.100.9, source port 4660 (0x1234), destination port 53 (0x0035), carrying the two payload bytes "OK" (0x4F4B). The UDP length field counts the 8-byte header plus 2 bytes of payload, so it is 10 = 0x000A.
| Group | 16-bit words | One’s-complement sum of the group |
|---|---|---|
| Pseudo-header | C000 0201 (src) · C633 6409 (dst) · 0011 (zero + protocol 17) · 000A (UDP length) | 0xEC59 |
| UDP header + payload, checksum field zeroed | 1234 0035 000A 0000 4F4B | 0x61BE |
| Whole thing | all ten words above | 0x4E18 |
The transmitted checksum is ~0x4E18 = 0xB1E7. To verify, the receiver sums all ten words with 0xB1E7 in the checksum field and gets 0xFFFF. And note the group column: 0xEC59 +' 0x61BE — one’s-complement addition, so 0xEC59 + 0x61BE = 0x14E17, fold the carry back in to get 0x4E18 — reproduces the total exactly. That is not a coincidence; it is the entire basis of checksum offload. The sum splits into independent groups that can be computed by different agents at different times and combined afterwards with two integer additions.
The four properties, and which offload each one buys
RFC 1071 §2 enumerates the properties that make this checksum unusually cheap. Each one is load-bearing somewhere in the Linux offload machinery, and it is worth naming the correspondence explicitly, because otherwise the kernel code looks like a pile of arbitrary tricks.
flowchart LR subgraph PROPS["RFC 1071 property"] P1["A. Commutative and associative<br/>the sum splits into groups<br/>and recombines"] P2["B. Byte-order independent<br/>summing the swapped words<br/>gives the swapped sum"] P3["C. Parallel summation<br/>sum 32 or 64 bits at a time,<br/>fold once at the end"] P4["D. Incremental update<br/>change one word, adjust<br/>the checksum by the delta"] end subgraph USES["what Linux builds on it"] U1["CHECKSUM_PARTIAL<br/>kernel sums the pseudo-header,<br/>NIC sums the payload"] U2["do_csum() sums native-endian<br/>and byte-swaps once if the<br/>buffer started at an odd address"] U3["hardware sums a whole<br/>bus-width word per clock;<br/>CHECKSUM_COMPLETE needs<br/>no protocol knowledge"] U4["csum_replace2() / NAT<br/>rewrite an address without<br/>re-walking the payload"] end P1 --> U1 P2 --> U2 P3 --> U3 P4 --> U4 U1 --> LCO["...and Local Checksum Offload,<br/>which is property A applied<br/>to a tunnel's outer header"] U3 --> GRO["...and CHECKSUM_COMPLETE,<br/>which is what lets GRO<br/>merge segments cheaply"]
The four exploitable properties of the Internet checksum, each mapped to the Linux mechanism it makes possible. What it shows: every offload in this note is a direct consequence of one arithmetic property, not an independent invention — CHECKSUM_PARTIAL is associativity, CHECKSUM_COMPLETE is parallel summation, NAT’s cheap rewrites are incremental update. The insight to take: the Internet checksum was designed to be split, deferred and patched. A cryptographic hash has none of these properties, which is exactly why nobody offloads one this way — you cannot hand a NIC “the second half of a SHA-256.”
Property A — splitting. As long as the even/odd byte alignment of each group is respected, the sum can be computed in any order and split into arbitrary groups. This is CHECKSUM_PARTIAL: the kernel computes the pseudo-header group (which the NIC does not know about), the NIC computes the payload group (which the kernel does not want to touch), and the two combine.
Property B — byte-order independence. Summing 16-bit words in the wrong byte order produces the correct sum with its own two bytes swapped. So a little-endian CPU can load big-endian network data as native 16-bit or 32-bit words, sum them, and swap once at the end — or, since the result is stored back into network order, often not swap at all. Linux’s generic implementation exploits this directly (lib/checksum.c, v6.12):
static unsigned int do_csum(const unsigned char *buff, int len)
{
int odd;
unsigned int result = 0;
if (len <= 0)
goto out;
odd = 1 & (unsigned long) buff; /* did we start on an odd address? */
if (odd) {
#ifdef __LITTLE_ENDIAN
result += (*buff << 8); /* consume it into the HIGH byte */
#else
result = *buff;
#endif
len--;
buff++;
}
/* ... 16-bit align, then the 32-bit-at-a-time loop ... */
result = from32to16(result);
if (odd)
result = ((result >> 8) & 0xff) | ((result & 0xff) << 8); /* swap back */
out:
return result;
}Read the odd handling carefully, because it is property B in three lines. If the buffer does not start on an even address, the routine cannot pair bytes the way the wire pairs them. Rather than doing byte-at-a-time arithmetic for the whole buffer, it deliberately sums the misaligned pairing — which by property B yields the right answer with the bytes transposed — and then transposes the 16-bit result back at the end. One swap instead of a slow loop.
Property C — parallel summation. Because addition is associative, a machine can add whole 32-bit or 64-bit words and fold at the end. The inner loop above does exactly this, accumulating 32-bit words with a manually tracked carry (carry = (w > result) detects the overflow), and folding only after the loop. Hardware takes this to its conclusion: a NIC’s checksum unit is a wide adder tree that consumes a full bus word per clock as the frame streams through the MAC, which is why checksumming is essentially free in silicon and expensive in software. This is also why CHECKSUM_COMPLETE — “sum everything, understand nothing” — is the cheapest possible thing to ask a NIC for.
Property D — incremental update. Covered in its own section below, because getting it wrong has its own RFC.
Folding, and the two C types that keep it honest
The kernel encodes the fold-and-complement step in one branchless expression (include/asm-generic/checksum.h, v6.12):
static inline __sum16 csum_fold(__wsum csum)
{
u32 sum = (__force u32)csum;
return (__force __sum16)((~sum - ror32(sum, 16)) >> 16);
}Walk it with the example above. The unfolded accumulator is 0x00014E17. ~sum is 0xFFFEB1E8; ror32(sum, 16) rotates the halves to give 0x4E170001; the difference is 0xB1E7B1E7, and shifting right by 16 yields 0xB1E7 — the folded and complemented checksum, in one subtract and one shift, with no conditional carry handling at all. The >> 16 picks the high half specifically because the subtraction lands the correct answer in both halves.
The two __force-cast types in that signature are a real safety mechanism, not noise. Linux distinguishes:
| Type | Meaning | Complemented? | Folded? |
|---|---|---|---|
__wsum | a partial, still-accumulating sum | no | no — 32 bits wide |
__sum16 | a finished checksum, as it appears on the wire | yes | yes — 16 bits |
sparse, the kernel’s static analyser, treats these as incompatible bitwise types, so mixing a partial sum with a wire checksum is a build-time complaint rather than a corrupted packet at 3 a.m. Every conversion is explicit: csum_fold() turns __wsum into __sum16, and csum_unfold() widens the other way. skb->csum is a __wsum; the check field of a struct tcphdr is a __sum16. When reading this code, that distinction tells you at a glance whether a value is finished or still in flight.
The one exception: zero means “no checksum” in UDP
UDP has a wart that every implementation must handle. RFC 768 says: “If the computed checksum is zero, it is transmitted as all ones (the equivalent in one’s complement arithmetic). An all zero transmitted checksum value means that the transmitter generated no checksum.” So 0x0000 in a UDP checksum field is not a checksum of zero — it is the absence of a checksum, and a receiver must skip verification entirely. A sender whose arithmetic legitimately produces 0x0000 must therefore transmit 0xFFFF instead, which is numerically the same value in one’s-complement.
Linux names this substitution and applies it at every point that writes a computed checksum (include/net/checksum.h, v6.12):
#define CSUM_MANGLED_0 ((__force __sum16)0xffff)The software fallback path in skb_checksum_help() applies it with a GNU elvis operator, which is as terse as the kernel gets (net/core/dev.c, v6.12):
*(__sum16 *)(skb->data + offset) = csum_fold(csum) ?: CSUM_MANGLED_0;This matters for offload because a NIC must implement the same rule, and it is a classic place for cheap hardware to be subtly wrong: a card that writes a literal 0x0000 into a UDP checksum field has just told every receiver on the Internet not to check the datagram. It is also why the receive path has a “zero check” variant of its validation macro (skb_checksum_init_zero_check(), below) — the zero case has to be recognised before, not after, the sum is computed.
Where the Checksums Actually Live in a Packet
Before the offload contract can make sense, it helps to see the two checksum fields a plain TCP-over-IPv4 frame carries, and where the offload interface’s two numbers point. Here is the 40-byte IPv4-plus-TCP header region, bit-accurate, with the IPv4 header starting at bit 0 of this diagram:
packet-beta 0-3: "Ver" 4-7: "IHL" 8-13: "DSCP" 14-15: "ECN" 16-31: "IPv4 Total Length" 32-47: "Identification" 48-50: "Flg" 51-63: "Fragment Offset" 64-71: "TTL" 72-79: "Proto=6" 80-95: "IPv4 Header Checksum" 96-127: "Source Address" 128-159: "Destination Address" 160-175: "TCP Source Port" 176-191: "TCP Destination Port" 192-223: "TCP Sequence Number" 224-255: "TCP Acknowledgment Number" 256-259: "Off" 260-263: "Rsv" 264-271: "Flags" 272-287: "Window" 288-303: "TCP Checksum" 304-319: "Urgent Pointer"
The two checksums in an IPv4/TCP frame, drawn at bit accuracy. What it shows: the IPv4 Header Checksum at bits 80–95 covers only the 20 bytes of the IPv4 header and nothing else; the TCP Checksum at bits 288–303 covers the TCP header, the entire payload that follows this diagram, and a pseudo-header that is not in the packet at all. The insight to take: these two fields are treated completely differently by Linux. The IPv4 header checksum is never offloaded — “it is always done in software. This is OK because when we build the IP header, we obviously have it in cache, so summing it isn’t expensive. It’s also rather short” (checksum-offloads.rst, v6.12). Only the transport checksum, whose cost scales with the payload, is worth handing to hardware. Field semantics belong to The Internet Protocol Version 4 Header Field by Field and The Transmission Control Protocol Header Field by Field (and The User Datagram Protocol Header Layout for UDP); what matters here is the offset of the checksum field within the transport header — 16 bytes in for TCP, 6 bytes in for UDP — because that number becomes csum_offset.
Now the offload interface itself. csum_start and csum_offset are the only two numbers a driver receives, and the asymmetry between how they are measured is the single most common source of driver bugs, so it is worth drawing the pointer arithmetic literally. packet-beta cannot express this — it draws a header’s fields, not a pointer into a buffer that includes headroom — so this one falls back to an ASCII box diagram with byte offsets labelled:
skb->head skb->data skb->tail
| | |
v v v
+---------------+---------------+--------+----------------+--------------+
| headroom | Ethernet hdr | IPv4 | TCP hdr | payload |
| (e.g. 64 B) | 14 B | 20 B | 20 B | N bytes |
+---------------+---------------+--------+----------------+--------------+
|<------------- csum_start = 64 + 34 = 98 ------------->|
^
| TCP header starts
|
|<-- csum_offset = 16 -->|
^
| the 16-bit
| checksum field
| the NIC writes
The NIC is told: sum every byte from head + csum_start to the end of the
packet, fold to 16 bits, complement, and store the result at
head + csum_start + csum_offset. It is told nothing else.
The CHECKSUM_PARTIAL pointer contract, drawn against a real buffer. What it shows: csum_start is measured from skb->head, so it includes the headroom and is not an offset into the packet as it appears on the wire; csum_offset is measured from csum_start, not from head. The insight to take: these two numbers are all the hardware gets. It receives no protocol identity, no header lengths, no notion of TCP versus UDP versus SCTP — just “sum this range, write there.” That deliberate ignorance is what makes one NETIF_F_HW_CSUM engine able to serve every protocol, and it is why a driver must never try to re-derive the transport offset itself. ASCII is used here rather than packet-beta because the thing being drawn is buffer geometry, not a wire format; see sk_buff Memory Layout and Headroom for the head/data/tail model in full.
Two consequences follow immediately from that picture and are worth stating before the code.
First, because csum_offset cannot be negative, the checksum field is always inside the range the device sums. The documentation calls this out as a feature: “Because csum_offset cannot be negative, this ensures that the previous value of the checksum field is included in the checksum computation, thus it can be used to supply any needed corrections to the checksum (such as the sum of the pseudo-header for UDP or TCP).” The pseudo-header is folded in for free by the act of leaving its sum sitting in the field the device is about to overwrite. There is no separate channel for it, no descriptor field, no protocol knowledge — just a value the kernel parked in the packet.
Second, the interface can express exactly one checksum. “This interface only allows a single checksum to be offloaded. Where encapsulation is used, the packet may have multiple checksum fields in different header layers, and the rest will have to be handled by another mechanism such as LCO or RCO.” That single sentence is the root cause of nearly every tunnel-related offload bug, and the reason two additional mechanisms exist at all.
Transmit: The CHECKSUM_PARTIAL Contract
On transmit, the interesting state is CHECKSUM_PARTIAL; the other three all reduce to “do nothing.” The kernel’s own statement of the driver’s obligation is short enough to quote whole (skbuff.h, “Checksumming on transmit for non-GSO”, v6.12):
The driver is required to checksum the packet as seen by
hard_start_xmit()from&sk_buff.csum_startup to the end, and to record/write the checksum at offset&sk_buff.csum_start + &sk_buff.csum_offset. A driver may verify that thecsum_startandcsum_offsetvalues are valid values given the length and offset of the packet, but it should not attempt to validate that the checksum refers to a legitimate transport layer checksum — it is the purview of the stack to validate thatcsum_startandcsum_offsetare set correctly.When the stack requests checksum offload for a packet, the driver MUST ensure that the checksum is set correctly. A driver can either offload the checksum calculation to the device, or call
skb_checksum_help(in the case that the device does not support offload for a particular checksum).
Note the two halves. The driver may sanity-check the numbers against the packet length, but must not second-guess them — a driver that decides “this looks like TCP, so the checksum must go at byte 50” has broken every protocol the stack knows and the driver does not. And the obligation is absolute: a CHECKSUM_PARTIAL skb must never reach the wire with the checksum unresolved. There is no “I couldn’t, sorry” return code.
The division of labour, step by step
sequenceDiagram autonumber participant App as Application participant L4 as Transport layer<br/>tcp_v4_send / udp_send_skb participant Core as net/core/dev.c<br/>validate_xmit_skb participant Drv as Driver<br/>ndo_start_xmit participant NIC as NIC checksum engine App->>L4: send() — N bytes of payload Note over L4: 1. compute the PSEUDO-HEADER sum only:<br/>src IP + dst IP + proto + segment length.<br/>Payload is never touched. L4->>L4: th->check = folded pseudo-header sum Note over L4: 2. declare the contract<br/>ip_summed = CHECKSUM_PARTIAL<br/>csum_start = transport_header - head<br/>csum_offset = 16 for TCP, 6 for UDP L4->>Core: skb travels the egress path,<br/>qdisc, netfilter POSTROUTING Core->>Core: 3. features = netif_skb_features(skb)<br/>can this device finish the job? alt device advertises a usable checksum feature Core->>Drv: hand over the skb unchanged Drv->>NIC: DMA descriptor carries<br/>csum_start and csum_offset Note over NIC: 4. sum head+csum_start .. end of packet,<br/>fold, complement, and store at<br/>head+csum_start+csum_offset.<br/>The prefilled pseudo-sum is INSIDE<br/>that range, so it folds in for free. NIC-->>NIC: frame leaves with a correct checksum else device cannot Core->>Core: 4'. skb_csum_hwoffload_help()<br/>→ skb_checksum_help()<br/>CPU walks the whole payload Core->>Drv: skb now has ip_summed = CHECKSUM_NONE<br/>and a final checksum in the packet end
The transmit-side hand-off, from send() to the wire. What it shows: the kernel does the part that is small and requires protocol knowledge — the pseudo-header, twelve bytes of addresses and lengths that are already in cache — and the NIC does the part that is large and requires no knowledge at all. The insight to take: step 2 and step 4 communicate entirely through two integers and one value parked in the packet. There is no back-channel. If step 2 gets csum_start wrong by a VLAN tag’s four bytes, step 4 dutifully sums the wrong range and writes a wrong answer into the right place, and the packet leaves the machine corrupt with no error anywhere.
The prefill, arithmetically
Return to the worked example. The pseudo-header sum was 0xEC59 and the sum of the UDP header and payload with the checksum field zeroed was 0x61BE. Under CHECKSUM_PARTIAL the kernel writes 0xEC59 — the folded pseudo-header sum itself, not its complement — into the checksum field, and tells the device to sum from the start of the UDP header. The device therefore sums 0x1234 +' 0x0035 +' 0x000A +' 0xEC59 +' 0x4F4B, which is 0x61BE +' 0xEC59 = 0x4E18, complements it, and writes 0xB1E7. That is bit-for-bit the checksum computed from scratch in the earlier table.
The kernel’s UDP helper shows all four cases in fourteen lines, and is the clearest single piece of code in this subsystem (net/ipv4/udp.c, udp_set_csum, v6.12):
void udp_set_csum(bool nocheck, struct sk_buff *skb,
__be32 saddr, __be32 daddr, int len)
{
struct udphdr *uh = udp_hdr(skb);
if (nocheck) { /* (1) UDP checksum disabled */
uh->check = 0;
} else if (skb_is_gso(skb)) { /* (2) GSO: segmenter will finish it */
uh->check = ~udp_v4_check(len, saddr, daddr, 0);
} else if (skb->ip_summed == CHECKSUM_PARTIAL) { /* (3) already PARTIAL: this is LCO */
uh->check = 0;
uh->check = udp_v4_check(len, saddr, daddr, lco_csum(skb));
if (uh->check == 0)
uh->check = CSUM_MANGLED_0;
} else { /* (4) the normal offload request */
skb->ip_summed = CHECKSUM_PARTIAL;
skb->csum_start = skb_transport_header(skb) - skb->head;
skb->csum_offset = offsetof(struct udphdr, check);
uh->check = ~udp_v4_check(len, saddr, daddr, 0);
}
}Branch (4) is the common path and deserves a line-by-line reading. udp_v4_check(len, saddr, daddr, 0) expands to csum_tcpudp_magic(), which builds the pseudo-header sum and then calls csum_fold() — and csum_fold() complements. So udp_v4_check(...) returns the complement of the folded pseudo-header sum, and ~udp_v4_check(...) complements it back: the field ends up holding the plain folded pseudo-header sum, exactly as the worked example requires. csum_start is computed as skb_transport_header(skb) - skb->head, which is the pointer arithmetic from the ASCII diagram written out literally, and csum_offset is offsetof(struct udphdr, check) = 6, because the UDP header is source port, destination port, length, checksum — three 16-bit fields before the checksum. For TCP the equivalent constant is offsetof(struct tcphdr, check) = 16.
Branch (1) is the “UDP checksum is optional over IPv4” case (SO_NO_CHECK, or a tunnel configured with nocheck); branch (2) hands the job to the segmentation machinery, which is the seam described in Segmentation Offloads GSO TSO; branch (3) is Local Checksum Offload and is unpacked later in this note.
skb_partial_csum_set() — the guarded way to declare the contract
Code that sets up CHECKSUM_PARTIAL from untrusted offsets — most importantly virtio_net, which receives csum_start/csum_offset from a guest or a hypervisor — must go through a helper that bounds-checks them (net/core/skbuff.c, v6.12):
bool skb_partial_csum_set(struct sk_buff *skb, u16 start, u16 off)
{
u32 csum_end = (u32)start + (u32)off + sizeof(__sum16);
u32 csum_start = skb_headroom(skb) + (u32)start;
if (unlikely(csum_start >= U16_MAX || csum_end > skb_headlen(skb))) {
net_warn_ratelimited("bad partial csum: csum=%u/%u headroom=%u headlen=%u\n",
start, off, skb_headroom(skb), skb_headlen(skb));
return false;
}
skb->ip_summed = CHECKSUM_PARTIAL;
skb->csum_start = csum_start;
skb->csum_offset = off;
skb->transport_header = csum_start;
return true;
}Two checks, and each protects against a distinct failure. csum_start >= U16_MAX catches an offset that will not fit the 16-bit csum_start field — a silent truncation would point the device at a garbage address inside the buffer. csum_end > skb_headlen(skb) enforces something more subtle: the two bytes the device writes must land in the linear head area of the skb, not in a paged fragment. skb_headlen() is the length of the linear region only; a scatter-gather packet’s payload lives in pages hanging off skb_shared_info (see skb_shared_info and Paged Fragments). A device can sum across the fragments happily, because it walks the whole DMA scatter list — but the descriptor that says “write two bytes here” addresses a single location, and the stack’s own fallback path writes through skb->data. Allowing the checksum field into a fragment would break both. When a VLAN or tunnel insertion pushes the transport header out of the linear area, this is the check that fires, and the ratelimited warning above is what you see in dmesg.
Note also the last assignment: skb->transport_header = csum_start. The helper does not merely record the checksum contract, it defines where the transport header is, which is why the kernel’s rule is that csum_start and the transport header must agree. Segmentation enforces the same identity from the other side — tcp_gso_segment() refuses a packet where skb_checksum_start(skb) != skb_transport_header(skb).
Receive: What a Device Can Report, and How the Stack Uses It
On the receive path, the driver’s job is to describe accurately what its device did. The four states, quoted from the kernel’s own documentation block and then unpacked:
CHECKSUM_NONE — “Device did not checksum this packet e.g. due to lack of capabilities. The packet contains full (though not verified) checksum in packet but not in skb->csum. Thus, skb->csum is undefined in this case.” The stack must sum the payload itself. This is the baseline for a card with no receive-checksum capability, and also the honest answer for a packet the card could not parse — a fragmented datagram, an unrecognised tunnel, an IPv6 packet with extension headers the parser gave up on.
CHECKSUM_UNNECESSARY — “The hardware you’re dealing with doesn’t calculate the full checksum (as in CHECKSUM_COMPLETE), but it does parse headers and verify checksums for specific protocols. For such packets it will set CHECKSUM_UNNECESSARY if their checksums are okay. &sk_buff.csum is still undefined in this case though. A driver or device must never modify the checksum field in the packet even if checksum is verified.” The applicable protocols are enumerated: TCP over IPv4 and IPv6; UDP over IPv4 and IPv6 (including a zero UDP checksum, which the stack may then further validate); GRE, but “only if the checksum is present in the header”; SCTP’s CRC; and the FCoE CRC. Note what this state does not give you: any way to recheck the hardware’s verdict, and any usable value in skb->csum. You are trusting the card.
CHECKSUM_COMPLETE — “This is the most generic way. The device supplied checksum of the whole packet as seen by netif_rx() and fills in &sk_buff.csum. This means the hardware doesn’t need to parse L3/L4 headers to implement this.” No verdict, no protocol parsing: a raw one’s-complement sum of every byte the card received, handed back as a __wsum. The documentation is emphatic about the preference: “Even if device supports only some protocols, but is able to produce skb->csum, it MUST use CHECKSUM_COMPLETE, not CHECKSUM_UNNECESSARY.”
CHECKSUM_PARTIAL on receive is the odd one out — it means the packet arrived with a checksum that was never computed. “This may occur on a packet received directly from another Linux OS, e.g., a virtualized Linux kernel on the same host, or it may be set in the input path in GRO or remote checksum offload.” A veth pair, a virtio_net link between a guest and its host, a container’s bridge port: none of these traverse a physical medium, so computing a checksum to protect against corruption that cannot happen would be pure waste. The scoping rule is precise: “the checksum referred to by skb->csum_start + skb->csum_offset and any preceding checksums in the packet are considered verified. Any checksums in the packet that are after the checksum being offloaded are not considered to be verified.”
The validation decision tree
The receive-side validation is built from one macro family, and reading it as a decision tree makes the whole design legible (skbuff.h, v6.12):
flowchart TB START["transport layer calls<br/>skb_checksum_init / _validate<br/>e.g. udp4_csum_init()"] --> NEED{"__skb_checksum_validate_needed<br/>skb_csum_unnecessary(skb)<br/>or a zero UDP checksum?"} NEED -->|"yes — nothing to do"| DONE1["csum_valid = 1<br/>__skb_decr_checksum_unnecessary()<br/>consume one layer of csum_level<br/><b>zero bytes of payload touched</b>"] NEED -->|"no"| PSEUDO["compute the pseudo-header sum<br/>inet_compute_pseudo(skb, proto)<br/>— 12 bytes, already in cache"] PSEUDO --> ISCOMP{"ip_summed ==<br/>CHECKSUM_COMPLETE ?"} ISCOMP -->|"yes"| FOLD{"csum_fold(csum_add(psum, skb->csum))<br/>== 0 ?"} FOLD -->|"yes"| DONE2["csum_valid = 1, return 0<br/><b>two integer adds, no payload walk</b>"] FOLD -->|"no"| STASH ISCOMP -->|"no — CHECKSUM_NONE"| STASH["skb->csum = pseudo-header sum"] STASH --> SMALL{"complete requested,<br/>or skb->len <= CHECKSUM_BREAK (76) ?"} SMALL -->|"yes"| SW["__skb_checksum_complete()<br/>walk every byte in software"] SMALL -->|"no"| DEFER["return 0 and defer:<br/>pseudo-sum is parked in skb->csum,<br/>the sum happens during copy-to-user"] SW --> BAD{"recomputed sum == 0<br/>but hardware said bad ?"} BAD -->|"yes"| FAULT["netdev_rx_csum_fault()<br/>dmesg: 'hw csum failure'<br/>+ skb_dump + stack trace, once"] BAD -->|"no"| VERDICT["csum_valid = !sum<br/>drop the packet if sum != 0"]
Receive-side checksum validation in Linux 6.12, as a decision tree. What it shows: three of the four paths never touch the payload. CHECKSUM_UNNECESSARY short-circuits at the first gate; a good CHECKSUM_COMPLETE costs one csum_add and one csum_fold; only CHECKSUM_NONE — or a COMPLETE value that failed — forces a walk over every byte. The insight to take: the CHECKSUM_BREAK constant, 76 bytes, is where the kernel decides a packet is small enough that summing it now is cheaper than deferring. Below that threshold it sums immediately; above it, the pseudo-header sum is parked in skb->csum and the real work is folded into the copy to userspace, so the payload is dragged through cache exactly once instead of twice. That fusion of checksum-with-copy is RFC 1071’s “combine with data copying” technique, still earning its keep.
The first gate is worth reading directly, because it is subtler than its name:
static inline int skb_csum_unnecessary(const struct sk_buff *skb)
{
return ((skb->ip_summed == CHECKSUM_UNNECESSARY) ||
skb->csum_valid ||
(skb->ip_summed == CHECKSUM_PARTIAL &&
skb_checksum_start_offset(skb) >= 0));
}Three ways to be excused from verification. The obvious one is the hardware’s CHECKSUM_UNNECESSARY verdict. The second, csum_valid, is a sticky bit set by an earlier layer that already validated this packet — it stops a checksum being verified twice as a packet climbs through, say, a tunnel decapsulation. The third is the locally-generated case: a CHECKSUM_PARTIAL skb whose csum_start is at or after the start of the packet data is one this host (or a sibling guest) is about to fill in, so there is nothing to check. skb_checksum_start_offset() is skb->csum_start - skb_headroom(skb) — negative means the checksum point has been pushed behind the current skb->data, which happens when headers are stripped, and in that case the contract is no longer meaningful.
CHECKSUM_COMPLETE has upkeep
The one real cost of CHECKSUM_COMPLETE is that skb->csum is a sum over the packet as the device saw it, and the packet changes shape as it climbs the stack: Ethernet is pulled off, then IP, then a tunnel header. Every one of those skb_pull() operations invalidates the stored sum unless it is adjusted, so the receive path uses checksum-aware wrappers (skbuff.h, v6.12):
static inline void
__skb_postpull_rcsum(struct sk_buff *skb, const void *start, unsigned int len,
unsigned int off)
{
if (skb->ip_summed == CHECKSUM_COMPLETE)
skb->csum = csum_block_sub(skb->csum,
csum_partial(start, len, 0), off);
else if (skb->ip_summed == CHECKSUM_PARTIAL &&
skb_checksum_start_offset(skb) < 0)
skb->ip_summed = CHECKSUM_NONE;
}csum_block_sub() subtracts the sum of the bytes just removed — and the off argument matters, because of property A’s caveat about even/odd alignment. csum_block_sub calls csum_shift(), which rotates the sub-sum by 8 bits when the offset is odd, so that the bytes being subtracted line up with the same parity they had when they were added. Getting that rotation wrong produces a skb->csum that is off by a byte swap, which then fails validation for reasons that look like memory corruption.
This upkeep is real work — a csum_partial() over each stripped header — but it is work proportional to header size, not payload size, and it buys something CHECKSUM_UNNECESSARY cannot: a value the stack can still reason about after arbitrary decapsulation.
The kernel does not fully trust the hardware
When a CHECKSUM_COMPLETE value fails to validate, the stack does not simply drop the packet. __skb_checksum_complete() recomputes the sum in software and then runs a check whose logic is deliberately inverted (net/core/skbuff.c, v6.12):
sum = csum_fold(csum_add(skb->csum, csum));
/* This check is inverted, because we already knew the hardware
* checksum is invalid before calling this function. So, if the
* re-computed checksum is valid instead, then we have a mismatch
* between the original skb->csum and skb_checksum(). This means either
* the original hardware checksum is incorrect or we screw up skb->csum
* when moving skb->data around.
*/
if (likely(!sum)) {
if (unlikely(skb->ip_summed == CHECKSUM_COMPLETE) &&
!skb->csum_complete_sw)
netdev_rx_csum_fault(skb->dev, skb);
}If software says the packet is fine and hardware said it was not, one of the two lied, and the kernel says so loudly — netdev_rx_csum_fault() prints hw csum failure, dumps the skb, and dumps a stack trace, all wrapped in DO_ONCE_LITE() so a broken NIC cannot turn a packet flood into a log flood. The csum_complete_sw flag exists precisely to keep this honest: it records that a CHECKSUM_COMPLETE value was produced by software rather than by the device, so the kernel never blames a card for its own arithmetic. If you see hw csum failure in dmesg, that message is the kernel accusing your NIC or a driver of getting the sum wrong, and it is one of the few places in the stack that names hardware as the suspect.
UDP goes one step further and second-guesses a bad hardware verdict outright (net/ipv4/udp.c, udp4_csum_init, v6.12):
if (skb->ip_summed == CHECKSUM_COMPLETE && !skb->csum_valid) {
/* If SW calculated the value, we know it's bad */
if (skb->csum_complete_sw)
return 1;
/* HW says the value is bad. Let's validate that.
* skb->csum is no longer the full packet checksum,
* so don't treat it as such.
*/
skb_checksum_complete_unset(skb);
}Read that as institutional memory: cards have been wrong often enough that the receive path treats a hardware “bad” as a hypothesis to be tested, and only a software-computed failure as final.
Why CHECKSUM_COMPLETE Beats CHECKSUM_UNNECESSARY for Tunnels
The two receive states look interchangeable on a plain TCP-over-IPv4 packet: both mean “you do not have to sum the payload.” They diverge the moment a packet has more than one checksum in it, which today means the moment it comes out of a tunnel.
CHECKSUM_UNNECESSARY is a verdict about specific protocols the device recognised. To issue it, the card must contain a parser that walks the headers, identifies each layer, finds each checksum field, knows which pseudo-header each one covers, and computes them all. That parser is silicon, fixed at tape-out. When a new encapsulation appears — GENEVE after VXLAN, VXLAN-GPE after GENEVE, whatever comes after that — cards built before it exists cannot verify it, and fall back to CHECKSUM_NONE. This is protocol ossification in its purest form: the deployed hardware defines which protocols are cheap, and therefore which protocols get deployed.
That phrase is not this note’s coinage, and the argument was had in public. Jonathan Corbet’s Checksum offloads and protocol ossification (LWN, 8 December 2015) records the moment the Linux networking maintainers drew the line. Corbet’s framing of why tunnels are built on UDP is the sharpest statement of the problem: “of all the protocols out there, only two, TCP and UDP, are widely supported by network routers and protocol offload engines in network interfaces. Tunneling protocols use UDP because they have to if they are to get the performance they need.” Advanced hardware support “has the effect of setting protocols into stone,” and the article traces the consequences — Multipath TCP engineered to look like ordinary TCP, Stateless Transport Tunneling disguised behind TCP-shaped packets, SCTP and DCCP effectively undeployable.
The precipitating patch set was Intel’s, adding GENEVE offload to the i40e driver by generalising the kernel’s VXLAN-specific device operations into a tunnel-type multiplexer. Tom Herbert pushed back on the grounds that it “encourage[s] manufacturers to implement more protocol-specific awareness into their interfaces rather than implementing the protocol-independent mechanism,” and David Miller — then the networking maintainer — said he would not merge code heading that way, later softening to a position Corbet quotes directly: “Pushing back is different from blocking entirely… You’ll just have to bear with me, be patient, and survive my tantrum on this matter.” The protocol-independent mechanism he was holding out for is exactly the csum_start/csum_offset interface on transmit and CHECKSUM_COMPLETE on receive.
CHECKSUM_COMPLETE is not a verdict at all. It is “here is the sum of the bytes.” A card that implements it needs no parser, no protocol table, and no update: it works for VXLAN, for GENEVE, for a protocol invented next year, for a protocol the card’s designers would have refused to implement. The stack then does the protocol-aware part — the pseudo-header, the layer bookkeeping — in software, where it is cheap because those fields are already in cache and there are only a dozen bytes of them. Corbet’s summary of the receive half is worth keeping as the one-sentence version of this entire section: “the networking developers would rather that the interface simply calculate a checksum for the packet as a whole. It is then a relatively cheap operation for the kernel to ‘subtract out’ the portion of the checksum corresponding to the outer headers and arrive at the correct inner checksum.”
csum_level: counting how deep the trust goes
Because CHECKSUM_UNNECESSARY is protocol-aware, it needs a companion field to say how many layers were verified. That is skb->csum_level, defined as “the number of consecutive checksums found in the packet minus one that have been verified as CHECKSUM_UNNECESSARY.” The kernel documentation’s own worked example is the clearest possible illustration:
flowchart TB subgraph PKT["one received frame: IPv6 → UDP → GRE → IPv4 → TCP"] direction LR A["IPv6<br/><i>no checksum in IPv6</i>"] --> B["UDP<br/>checksum #1"] --> C["GRE<br/>checksum #2<br/><i>if present</i>"] --> D["IPv4<br/>header checksum<br/><i>not counted</i>"] --> E["TCP<br/>checksum #3"] end PKT --> Q{"which consecutive<br/>checksums did the<br/>device verify?"} Q -->|"UDP + GRE + TCP — all three"| L2["csum_level = 2<br/>(three verified, minus one)"] Q -->|"UDP only; GRE unsupported<br/>or GRE checksum bad"| L0["csum_level = 0<br/>the TCP checksum is NOT counted —<br/>the chain broke at GRE"] L2 --> USE2["each decapsulation step calls<br/>__skb_decr_checksum_unnecessary():<br/>level 2 → 1 → 0 → CHECKSUM_NONE"] L0 --> USE0["the tunnel layer consumes<br/>the one verified checksum;<br/>TCP must be summed in software"]
How csum_level tracks trust through a nested encapsulation. What it shows: the count is of consecutive verified checksums from the outside in. If the device could not verify GRE — either because it does not understand GRE or because the GRE checksum was actually bad — then the TCP checksum behind it does not count even if the device also verified it, because the chain of trust is broken in the middle. The insight to take: csum_level is a small integer that has to encode a fundamentally protocol-aware claim, and every decapsulation step spends one unit of it. CHECKSUM_COMPLETE needs none of this bookkeeping, because a sum over the whole packet remains a valid sum over the whole packet no matter how many layers you peel — which is the practical reason the kernel documentation tells partially-capable devices to report COMPLETE rather than UNNECESSARY.
The spending is done by a pair of helpers whose asymmetry tells the story:
static inline void __skb_decr_checksum_unnecessary(struct sk_buff *skb)
{
if (skb->ip_summed == CHECKSUM_UNNECESSARY) {
if (skb->csum_level == 0)
skb->ip_summed = CHECKSUM_NONE; /* trust exhausted */
else
skb->csum_level--;
}
}
static inline void __skb_incr_checksum_unnecessary(struct sk_buff *skb)
{
if (skb->ip_summed == CHECKSUM_UNNECESSARY) {
if (skb->csum_level < SKB_MAX_CSUM_LEVEL)
skb->csum_level++;
} else if (skb->ip_summed == CHECKSUM_NONE) {
skb->ip_summed = CHECKSUM_UNNECESSARY;
skb->csum_level = 0;
}
}
static inline void __skb_reset_checksum_unnecessary(struct sk_buff *skb)
{
if (skb->ip_summed == CHECKSUM_UNNECESSARY) {
skb->ip_summed = CHECKSUM_NONE;
skb->csum_level = 0;
}
}Every validation that succeeds consumes one level, and running out demotes the packet all the way to CHECKSUM_NONE — a fail-safe, not a fail-open. The increment side is what a tunnel driver calls after it has verified an inner checksum in software, promoting a CHECKSUM_NONE packet to UNNECESSARY so the layer above does not sum it again.
The conversion path
There is a third option the kernel takes where it can: turn one state into the other. skb_checksum_try_convert() upgrades a CHECKSUM_NONE-but-already-validated packet into a genuine CHECKSUM_COMPLETE:
static inline bool __skb_checksum_convert_check(struct sk_buff *skb)
{
return (skb->ip_summed == CHECKSUM_NONE && skb->csum_valid);
}
static inline void __skb_checksum_convert(struct sk_buff *skb, __wsum pseudo)
{
skb->csum = ~pseudo;
skb->ip_summed = CHECKSUM_COMPLETE;
}The assignment skb->csum = ~pseudo is property A again, run backwards, and it is worth pausing on because it looks like sleight of hand. If a transport checksum has been verified correct, then by construction the one’s-complement sum over the pseudo-header plus the transport segment is 0xFFFF. Therefore the sum over the transport segment alone must be the complement of the pseudo-header sum. So without touching a single payload byte the kernel can manufacture a valid skb->csum for the region and hand it to the layers above as a real CHECKSUM_COMPLETE. The documentation lists “CHECKSUM_UNNECESSARY conversion” among the things that “should be documented here but aren’t yet” in checksum-offloads.rst; the code is the only specification.
Uncertain
Verify: the original
netdevmailing-list discussion (the review thread, not the cover letter) of Local and Remote Checksum Offload. Edward Cree’s LCO cover letter of 7 January 2016 has since been read via LWN’s archive of it and is cited in the LCO section; what remains unread is the reviewer traffic on both series, and Tom Herbert’s RCO postings. Reason: both kernel.org git front-ends are behind an Anubis proof-of-work bot check as of 2026-09-04.lore.kernel.orgreturns HTTP 200 with a JavaScript challenge page titled “Making sure you’re not a bot!” rather than archive content.git.kernel.orgcgit is now blocked the same way — re-tested during this edit,.../ethtool.git/plain/common.cand.../log/?qt=grep&q=…both return HTTP 200 with a ~7.7 KB body carrying that same title. This matters because cgit was previously documented in this vault as the workaround forlore; it is not one any more, and a status-code check will not catch it. To resolve: fetch through a client that can execute the challenge, or use an alternative archive (marc.info,spinics.net). Working routes found during this edit and used here: kernel source atraw.githubusercontent.com/torvalds/linux/<tag>/<path>, commit metadata atgithub.com/torvalds/linux/commits/<tag>/<path>.atomand message text atgithub.com/torvalds/linux/commit/<sha>.patch, and userspaceethtoolsource from the release tarballs atmirrors.edge.kernel.org/pub/software/network/ethtool/. Every claim in this note is grounded in in-tree source or in-tree documentation read at v6.12; the one claim that previously rested on cgit — theoff_flag_def[]table below — was re-verified verbatim againstcommon.cin the ethtool 6.11 release tarball, which matches the quoted text exactly.#uncertain
Incremental Update: Rewriting a Packet Without Re-Summing It
Property D of the Internet checksum is the one that makes routers, firewalls and Network Address Translation (NAT) practical. If a middlebox changes one 16-bit word of a packet, it does not need to re-sum the packet; it can compute the delta to the checksum in a handful of instructions. This matters enormously to offload, because every NAT rewrite, every eBPF program that edits a header, and every skb_pull/skb_push in the stack has to keep the checksum state consistent — and “consistent” means something different in each ip_summed state.
The arithmetic has a history worth knowing, because the naive version is wrong in a way that took six years and two RFCs to pin down.
flowchart TB R1071["<b>RFC 1071 (1988)</b><br/>C' = C + (-m) + m'<br/><i>the sum, not the checksum field</i>"] R1071 --> GAP["not directly usable:<br/>C and C' are the internal sums,<br/>not the value stored in the header"] GAP --> R1141["<b>RFC 1141 (1990)</b><br/>HC' = HC + m + ~m'<br/><i>looks right, distributes the complement</i>"] R1141 --> BUG{"result is -0 (0xFFFF)<br/>where recomputation<br/>gives +0 (0x0000)"} BUG -->|"one's complement has TWO zeros,<br/>and the complement does not<br/>distribute over addition at zero"| R1624["<b>RFC 1624 (1994)</b><br/>HC' = ~(~HC + ~m + m')<br/><i>never assumes distribution</i>"] R1624 --> KERN["Linux: csum_replace2()<br/>*sum = ~csum16_add(csum16_sub(~(*sum), old), new)<br/>with the RFC number in the comment"]
Three RFCs and one off-by-a-representation bug. What it shows: RFC 1141’s shortcut is correct for every input except the boundary where the new checksum should be +0, and it produces -0 (0xFFFF) there instead — a value a correctly-computed IPv4 header checksum “can never contain,” per RFC 1624 §3. The insight to take: one’s-complement arithmetic has two representations of zero, and the complement operator does not distribute over addition when the result is zero. That is the whole bug. It is invisible in testing because a receiver that verifies per RFC 1071 — summing the checksum field in and comparing against -0 — accepts both values; only a receiver that recomputes and compares field-to-field rejects the packet. Which means the failure appears only against certain peers.
RFC 1624’s own worked example makes the boundary concrete. A 16-bit field m = 0x5555 changes to m' = 0x3285, and the one’s-complement sum of every other word in the header is 0xCD7A. The original checksum is therefore HC = ~(0xCD7A +' 0x5555) = ~0x22D0 = 0xDD2F. Recomputing from scratch after the change gives ~(0xCD7A +' 0x3285) = ~0xFFFF = 0x0000. RFC 1141’s formula gives 0xDD2F + 0x5555 + ~0x3285 = 0xFFFF — a checksum that “does not match that computed from scratch, and moreover can never obtain for an IP header.” RFC 1624’s formula gives ~(0x22D0 +' ~0x5555 +' 0x3285) = ~0xFFFF = 0x0000, correct.
Linux implements exactly the RFC 1624 equation, and cites it (include/net/checksum.h, v6.12):
/* Implements RFC 1624 (Incremental Internet Checksum)
* 3. Discussion states :
* HC' = ~(~HC + ~m + m')
* m : old value of a 16bit field
* m' : new value of a 16bit field
*/
static __always_inline void csum_replace2(__sum16 *sum, __be16 old, __be16 new)
{
*sum = ~csum16_add(csum16_sub(~(*sum), old), new);
}Unfold it against the equation: ~(*sum) is ~HC; csum16_sub(x, old) is csum16_add(x, ~old) so it contributes ~m; csum16_add(..., new) contributes m'; and the leading ~ complements the whole thing. Three one’s-complement additions, no memory traffic, no payload walk. Run the RFC’s numbers through it: ~0xDD2F = 0x22D0; csum16_sub(0x22D0, 0x5555) = 0x22D0 +' 0xAAAA = 0xCD7A; csum16_add(0xCD7A, 0x3285) = 0xFFFF; complement gives 0x0000. Correct, and identical to recomputation.
Where incremental update collides with offload
NAT does not just rewrite a field; it rewrites a field that is inside the pseudo-header. Changing a source IP address changes the value the transport checksum covers, so both the IPv4 header checksum and the TCP/UDP checksum must be adjusted. And the correct adjustment depends on which ip_summed state the packet is in — which is exactly what inet_proto_csum_replace4() encodes (net/core/utils.c, v6.12):
void inet_proto_csum_replace4(__sum16 *sum, struct sk_buff *skb,
__be32 from, __be32 to, bool pseudohdr)
{
if (skb->ip_summed != CHECKSUM_PARTIAL) {
csum_replace4(sum, from, to); /* (a) patch the packet's checksum */
if (skb->ip_summed == CHECKSUM_COMPLETE && pseudohdr)
skb->csum = ~csum_add(csum_sub(~(skb->csum), /* (b) and skb->csum too */
(__force __wsum)from),
(__force __wsum)to);
} else if (pseudohdr) /* (c) PARTIAL: patch only the */
*sum = ~csum_fold(csum_add(csum_sub(csum_unfold(*sum), /* prefilled partial */
(__force __wsum)from),
(__force __wsum)to));
}Three branches, three different truths about the same edit:
- (a) For a packet whose checksum field holds a finished checksum (
CHECKSUM_NONE,UNNECESSARY, orCOMPLETE), the field is patched by the RFC 1624 delta. - (b) Additionally, for
CHECKSUM_COMPLETE,skb->csumis a sum over the packet as received — and if the changed bytes are pseudo-header bytes, they are not in the packet regionskb->csumcovers, so the running sum must be adjusted separately and in the opposite sense. Miss this and validation further up the stack fails on a packet that is perfectly good. - (c) For
CHECKSUM_PARTIAL, the checksum field does not hold a checksum — it holds the prefilled pseudo-header partial, and the NIC has not run yet. So the only correct action is to patch the partial and leave everything else alone. Patching it as if it were a finished checksum, or worse, recomputing it, produces a packet the NIC then “completes” into garbage.
Branch (c) is where the classic bug lives. A tc-eBPF program or an out-of-tree netfilter module that calls a plain csum_replace4() on a CHECKSUM_PARTIAL skb has corrupted the packet, and the corruption manifests only after the NIC finishes the checksum, i.e. only on the wire, i.e. only as “the remote end dropped it” — with a local tcpdump showing nothing wrong, for reasons the diagnostics section returns to.
Device Feature Flags: NETIF_F_HW_CSUM and the Ones It Replaced
A driver advertises what its device can do in netdev->hw_features; the currently-enabled subset is netdev->features; what the administrator has asked for is netdev->wanted_features. The checksum-related bits, with their kernel names, their ethtool string names, and what each actually promises:
NETIF_F_* flag | ethtool feature name | Direction | Promise |
|---|---|---|---|
NETIF_F_HW_CSUM | tx-checksum-ip-generic | TX | “able to compute one IP (one’s complement) checksum for any combination of protocols or protocol layering,” per the csum_start/csum_offset interface |
NETIF_F_IP_CSUM | tx-checksum-ipv4 | TX | only “plain TCP or UDP packets over IPv4… unencapsulated packets of the form IPv4 + TCP or IPv4 + UDP.” Deprecated. |
NETIF_F_IPV6_CSUM | tx-checksum-ipv6 | TX | the same for IPv6, and “IPv6 extension headers are not supported with this feature.” Deprecated. |
NETIF_F_SCTP_CRC | tx-checksum-sctp | TX | offload of the SCTP CRC32c through the same csum_start/csum_offset plumbing, flagged by skb->csum_not_inet = 1 |
NETIF_F_FCOE_CRC | tx-checksum-fcoe-crc | TX | offload of the Fibre Channel over Ethernet CRC — with no flag in the skb to distinguish it, so a driver supporting both must inspect the headers |
NETIF_F_RXCSUM | rx-checksum | RX | “Driver (device) performs receive checksum offload. This flag is only used to disable the RX checksum feature for a device. The stack will accept receive checksum indication in packets received on a device regardless of whether NETIF_F_RXCSUM is set.” |
Two entries in that table need emphasis. NETIF_F_RXCSUM is not a capability the stack consults before believing a driver — it is purely an off-switch for the administrator. If a driver sets ip_summed = CHECKSUM_UNNECESSARY, the stack believes it, full stop; the flag exists so you can tell the driver to stop doing that. And NETIF_F_HW_CSUM is explicitly “a superset of NETIF_F_IP_CSUM + NETIF_F_IPV6_CSUM. It means that device can fill TCP/UDP-like checksum anywhere in the packets whatever headers there might be” (netdev-features.rst, v6.12) — so setting both is a contradiction, and the kernel says so at feature-negotiation time with netdev_warn(dev, "mixed HW and IP checksum settings.\n") before clearing the protocol-specific bits.
The preference for HW_CSUM is not aesthetic. Trace what happens to a packet on the egress path when only the protocol-specific flags are set (net/core/dev.c, skb_csum_hwoffload_help, v6.12):
int skb_csum_hwoffload_help(struct sk_buff *skb,
const netdev_features_t features)
{
if (unlikely(skb_csum_is_sctp(skb))) /* csum_not_inet set? */
return !!(features & NETIF_F_SCTP_CRC) ? 0 :
skb_crc32c_csum_help(skb);
if (features & NETIF_F_HW_CSUM)
return 0; /* the device can do anything */
if (features & (NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM)) {
if (vlan_get_protocol(skb) == htons(ETH_P_IPV6) &&
skb_network_header_len(skb) != sizeof(struct ipv6hdr))
goto sw_checksum; /* extension headers present */
switch (skb->csum_offset) {
case offsetof(struct tcphdr, check): /* 16 */
case offsetof(struct udphdr, check): /* 6 */
return 0;
}
}
sw_checksum:
return skb_checksum_help(skb); /* CPU sums the payload */
}The NETIF_F_HW_CSUM case is a single test and an early return. The protocol-specific case is a pattern match on csum_offset — the kernel is reduced to guessing whether this is TCP or UDP by looking at where the checksum field sits, because that is the only expressive power the old flags have. Anything else — an SCTP CRC on a card without NETIF_F_SCTP_CRC, an IPv6 packet carrying extension headers, an encapsulated packet whose inner checksum sits at some other offset — falls straight through to skb_checksum_help() and a full software sum. That is the concrete cost of the deprecated flags: they turn “the device can do this” into “the device can do this only if the packet looks like 2005.”
What ethtool -K actually toggles
ethtool -K <dev> tx off is not a single bit. The ethtool userspace tool carries a table mapping its short command names to a group of kernel feature names (common.c, ethtool 6.11 release tarball):
const struct off_flag_def off_flag_def[] = {
{ "rx", "rx-checksumming", "rx-checksum",
ETHTOOL_GRXCSUM, ETHTOOL_SRXCSUM, ETH_FLAG_RXCSUM, 0 },
{ "tx", "tx-checksumming", "tx-checksum-*",
ETHTOOL_GTXCSUM, ETHTOOL_STXCSUM, ETH_FLAG_TXCSUM, 0 },
{ "sg", "scatter-gather", "tx-scatter-gather*",
ETHTOOL_GSG, ETHTOOL_SSG, ETH_FLAG_SG, 0 },
{ "tso", "tcp-segmentation-offload", "tx-tcp*-segmentation",
ETHTOOL_GTSO, ETHTOOL_STSO, ETH_FLAG_TSO, 0 },
/* ... gso, gro, lro, rxvlan, txvlan, ntuple, rxhash ... */
};Note the third column: the short name tx maps to the wildcard tx-checksum-*. On the kernel side, the corresponding ETHTOOL_STXCSUM ioctl resolves that wildcard to a concrete mask (net/ethtool/ioctl.c, v6.12):
case ETHTOOL_GTXCSUM:
case ETHTOOL_STXCSUM:
return NETIF_F_CSUM_MASK | NETIF_F_FCOE_CRC |
NETIF_F_SCTP_CRC;
case ETHTOOL_GRXCSUM:
case ETHTOOL_SRXCSUM:
return NETIF_F_RXCSUM;with NETIF_F_CSUM_MASK being (NETIF_F_IP_CSUM | NETIF_F_IPV6_CSUM | NETIF_F_HW_CSUM). So ethtool -K eth0 tx off clears five distinct feature bits at once, including the two CRC offloads that have nothing to do with the Internet checksum. It then sets wanted_features (masked by hw_features, so asking for something the device cannot do returns -EOPNOTSUPP) and calls __netdev_update_features().
The dependency graph that -K tx off walks
__netdev_update_features() runs the requested set through netdev_fix_features(), which is where the configuration-time dependency graph lives. This is a different mechanism from the per-packet feature harmonisation described in Segmentation Offloads GSO TSO: harmonize_features() decides what to do with one skb on the egress path, while netdev_fix_features() decides what the device’s persistent feature set is even allowed to contain. Drawn out, it explains a large fraction of “I turned off one thing and three others disappeared” reports:
flowchart TB TXOFF["ethtool -K dev tx off<br/>clears NETIF_F_CSUM_MASK<br/>+ FCOE_CRC + SCTP_CRC"] --> FIX["netdev_fix_features()"] RXOFF["ethtool -K dev rx off<br/>clears NETIF_F_RXCSUM"] --> FIX SGOFF["ethtool -K dev sg off<br/>clears NETIF_F_SG"] --> FIX FIX --> D1["TSO requires HW_CSUM or IP_CSUM<br/>→ drop NETIF_F_TSO, NETIF_F_TSO_ECN"] FIX --> D2["TSO6 requires HW_CSUM or IPV6_CSUM<br/>→ drop NETIF_F_TSO6"] FIX --> D3["USO requires an IP or HW checksum<br/>→ drop NETIF_F_GSO_UDP_L4"] FIX --> D4["TLS TX offload requires a checksum<br/>→ drop NETIF_F_HW_TLS_TX"] FIX --> D5["TLS RX offload requires RXCSUM<br/>→ drop NETIF_F_HW_TLS_RX"] FIX --> D6["hardware GRO requires RXCSUM<br/>→ drop NETIF_F_GRO_HW"] FIX --> D7["all TSO requires SG<br/>→ drop NETIF_F_ALL_TSO"] FIX --> D8["HW_CSUM together with IP_CSUM/IPV6_CSUM<br/>is illegal → netdev_warn<br/>'mixed HW and IP checksum settings'"]
The configuration-time feature dependency graph rooted at the checksum flags, transcribed from netdev_fix_features() (net/core/dev.c, v6.12). What it shows: transmit checksum offload is a prerequisite for TCP segmentation offload, UDP segmentation offload and kernel TLS transmit offload; receive checksum offload is a prerequisite for hardware GRO and kernel TLS receive offload. Disabling a checksum silently disables all of them. The insight to take: this is the mechanism behind the most common surprise in Linux network tuning — ethtool -K eth0 tx off turns TSO off as a side effect, so a benchmark that “isolates the effect of checksum offload” has in fact measured checksum offload and segmentation offload together. Every one of these edges exists because the dependent offload must produce a correct checksum for output it generates, and it can only do that if the checksum engine is available. The kernel logs each demotion at netdev_dbg level, so dynamic_debug on net/core/dev.c will show you exactly which rule fired.
The kernel’s own comment on the GRO_HW edge is the clearest statement of the principle: “NETIF_F_GRO_HW implies doing RXCSUM since every packet successfully merged by hardware must also have the checksum verified by hardware. If the user does not want to enable RXCSUM, logically, we should disable GRO_HW.” Coalescing segments you have not verified would mean silently merging a corrupt segment into a good stream.
The Software Fallback, and Why It Is Expensive
The stack is written on the assumption that offload works. The documentation is explicit about where the one exception lives: “The stack should, for the most part, assume that checksum offload is supported by the underlying device. The only place that should check is validate_xmit_skb(), and the functions it calls directly or indirectly.” That function compares the offloads the skb wants against netdev->features and, for anything the device cannot do, performs it in software — for checksums, by calling skb_csum_hwoffload_help(skb, features).
The terminal function is skb_checksum_help(), and reading it in full is worth the space because every line is a guard against a specific historical disaster (net/core/dev.c, v6.12):
flowchart TB IN["skb_checksum_help(skb)"] --> G1{"ip_summed == CHECKSUM_COMPLETE ?"} G1 -->|"yes"| SET["goto out_set_summed:<br/>just demote to CHECKSUM_NONE.<br/>COMPLETE is meaningless on TX."] G1 -->|"no"| G2{"skb_is_gso(skb) ?"} G2 -->|"yes"| WARN["skb_warn_bad_offload()<br/>WARN(1, 'caps=...') + skb_dump<br/>return -EINVAL.<br/><i>a GSO skb must be segmented first;<br/>there is no single checksum to write</i>"] G2 -->|"no"| G3{"skb_has_shared_frag(skb) ?"} G3 -->|"yes"| LIN["__skb_linearize(skb)<br/><b>copy every paged fragment into<br/>one contiguous buffer</b><br/><i>'no frag could be modified by an<br/>external entity: checksum could be wrong'</i>"] G3 -->|"no"| OFF LIN --> OFF["offset = skb_checksum_start_offset(skb)"] OFF --> G4{"offset >= skb_headlen(skb)<br/>or offset+2 > skb_headlen(skb) ?"} G4 -->|"yes"| WARN2["WARN_ONCE + skb_dump<br/>return -EINVAL"] G4 -->|"no"| SUM["csum = skb_checksum(skb, offset,<br/>skb->len - offset, 0)<br/><b>walks every byte, including frags</b>"] SUM --> WRITE["*(__sum16 *)(skb->data + offset)<br/>= csum_fold(csum) ?: CSUM_MANGLED_0"] WRITE --> SET SET --> DONE["ip_summed = CHECKSUM_NONE<br/>the packet now carries a<br/>finished checksum"]
skb_checksum_help() in Linux 6.12, guard by guard. What it shows: the fallback is not merely “sum the payload” — it may first linearize the skb, copying every paged fragment into one contiguous allocation, and it refuses outright on a GSO packet. The insight to take: the linearize step is the hidden cost. A zero-copy sendfile() or MSG_ZEROCOPY send hands the stack pages it does not own and that userspace may still be writing to; summing them without copying would race, so the kernel copies. A single feature mismatch therefore converts a zero-copy send into a full payload copy plus a full payload sum — roughly the worst possible outcome, and one that shows up in perf top as skb_checksum and memcpy side by side, never as anything named “checksum offload is off.”
The skb_warn_bad_offload() path deserves its own note, because it is a diagnostic aimed squarely at driver authors. When it fires it prints the device’s driver name, a full hex dump of the skb, and — the useful part — both feature masks:
skb_dump(KERN_WARNING, skb, false);
WARN(1, "%s: caps=(%pNF, %pNF)\n",
name, dev ? &dev->features : &null_features,
skb->sk ? &skb->sk->sk_route_caps : &null_features);The two masks are dev->features (what the device says it can do now) and sk->sk_route_caps (what the socket believed the route could do when it built this packet). A mismatch between them is the entire bug class: something changed the device’s features — a bond member going down, a VLAN or tunnel netdev whose feature propagation is broken, an ethtool -K while the socket was live — after the socket had already committed to a large, CHECKSUM_PARTIAL, GSO-marked skb. Seeing caps= in dmesg means “the socket and the device disagreed about the offload contract,” and the two printed masks tell you which way.
The SCTP variant, skb_crc32c_csum_help(), is structurally identical but computes a CRC32c rather than a one’s-complement sum, writes it at csum_start + offsetof(struct sctphdr, checksum), and clears csum_not_inet at the end. It is the reason skb_csum_hwoffload_help() tests skb_csum_is_sctp(skb) first: an SCTP packet in CHECKSUM_PARTIAL state is asking for entirely different arithmetic through the same two fields.
Local Checksum Offload: Computing the Outer Checksum Without Reading the Payload
Everything up to this point has assumed a packet with one checksum. Encapsulation breaks that assumption, and the breakage is expensive in a specific way. When the kernel builds a Virtual Extensible LAN (VXLAN) or Generic Network Virtualization Encapsulation (GENEVE) frame, it wraps an entire inner Ethernet frame — inner MAC header, inner Internet Protocol (IP) header, inner Transmission Control Protocol (TCP) or User Datagram Protocol (UDP) header, and payload — inside an outer UDP datagram. The outer UDP checksum, by the rules of RFC 768, covers the outer pseudo-header, the outer UDP header, and every byte of that encapsulated frame. So the naive tunnel egress path computes the outer checksum by summing the whole payload in software — precisely the per-byte cost the entire offload apparatus exists to avoid — and does it in addition to whatever the device is going to do for the inner checksum.
Local Checksum Offload (LCO) eliminates that sum. The kernel documentation states the technique in one sentence: “LCO is a technique for efficiently computing the outer checksum of an encapsulated datagram when the inner checksum is due to be offloaded” (Documentation/networking/checksum-offloads.rst, v6.12). The word local means the whole trick happens on the transmitting host — unlike Remote Checksum Offload in the next section, it requires no cooperation from the receiver and no change to any wire protocol. It is a pure arithmetic identity, applied on the sending side, invisible to everyone else.
The provenance is worth recording, because it dates the feature and names the author. LCO arrived in a five-patch series posted by Edward Cree of Solarflare to David Miller and netdev on 7 January 2016, subject “[PATCH v2 net-next 0/5] Local Checksum Offload”, archived by LWN (LWN.net, Article 671457 — verified 2026-09-04 to be the genuine article and not a rate-limit page). The cover letter is short and tells you what the author cared about: “Tested with a VXLAN tunnel over a device that doesn’t support inner checksum offload (so the checksum will have been done in sw by validate_xmit_skb())”, and, among the changes from v1, “Wrote up some documentation covering TX checksum offload, LCO and RCO.” The series touched include/linux/skbuff.h (lco_csum() itself), net/ipv4/udp.c, net/ipv6/ip6_checksum.c, drivers/net/vxlan.c, net/ipv4/fou.c and net/ipv4/ip_tunnel_core.c, and created Documentation/networking/tx-offloads.txt — the 122-line file that, renamed, is the checksum-offloads.rst this note has been quoting throughout. The cover letter also notes the coverage gap frankly: “I think it now covers everything except GRE,” which is the ancestor of the IPv6-GRE gap flagged at the end of this section.
The identity
LCO rests on one fact, which the documentation states plainly and which is worth deriving in full because the derivation is the mechanism:
“if we have set up TX Checksum Offload with a start/offset pair, we know that after the device has filled in that checksum, the ones complement sum from
csum_startto the end of the packet will be equal to the complement of whatever value we put in the checksum field beforehand.”
Write it out. Let P be the 16-bit value the kernel prefilled into the inner checksum field — from the earlier section, that is the folded pseudo-header sum, not its complement. Let S be the one’s-complement sum of every byte from csum_start to the end of the packet as the buffer currently stands, with P sitting in the checksum field. The device’s job is to compute ~S and write it into that field. Call the resulting sum over the same region S′. Replacing one 16-bit word changes the sum by (new − old), so:
S' = S -' P +' ~S (replace one 16-bit word: subtract the old, add the new)
= S -' P -' S (property A: ~x == -x in one's-complement arithmetic)
= -P
= ~P
where +' and -' are one’s-complement addition and subtraction, as in the worked example earlier in this note.
The S terms cancel. The sum over the region from csum_start to the end of the packet, after the device has finished, is the complement of the value the kernel put there before the device ran — and the kernel knows that value, because it wrote it. It is therefore a known constant that costs zero payload reads to obtain. The bytes themselves never need to be looked at; whatever they are, they cancel out of the answer.
That is the entire trick. The outer UDP checksum needs the sum over [outer UDP header … end of packet]. Split that region at csum_start:
flowchart TB subgraph FRAME["one VXLAN frame on the transmit path, laid out in buffer order"] direction LR OIP["outer IPv4<br/>20 B"] --> OUDP["outer UDP<br/>8 B<br/><i>l4_hdr</i>"] --> VX["VXLAN<br/>8 B"] --> IETH["inner Ethernet<br/>14 B"] --> IIP["inner IPv4<br/>20 B"] --> ITCP["inner TCP<br/>20 B<br/><b>csum_start</b>"] --> PAY["payload<br/><b>up to 64 KB</b>"] end OUDP -.->|"region 1"| R1["<b>csum_partial(l4_hdr, csum_start − l4_hdr, partial)</b><br/>summed in software:<br/>outer UDP + VXLAN + inner Eth + inner IP<br/><b>50 bytes</b>"] ITCP -.->|"region 2"| R2["<b>never read</b><br/>known to equal ~P by the identity,<br/>where P is the prefilled<br/>pseudo-header sum"] R1 --> ADD["one's-complement add the two<br/>→ sum over [outer UDP … end of packet]"] R2 --> ADD ADD --> FOLD["udp_v4_check(len, saddr, daddr, …)<br/>adds the outer pseudo-header, folds, complements"] FOLD --> OUT["final outer UDP checksum,<br/>written into the packet now"]
How Local Checksum Offload splits the outer checksum’s coverage at csum_start. What it shows: the region the outer checksum must cover is cut in two at the point where the device’s work begins. Region 1 — a few dozen bytes of headers — is summed in software. Region 2 — the entire payload — is never touched, because the identity S' = ~P says exactly what it will sum to once the device has written the inner checksum. The insight to take: the cost of the outer checksum drops from O(packet length) to O(header length), and for a 64 KB Generic Segmentation Offload (GSO) super-packet that is a factor of roughly 1,300. This is why UDP-based tunnels can run with the outer checksum enabled at all; without LCO, “enable the outer UDP checksum” would mean “sum every byte twice.”
lco_csum(), line by line
The implementation is eight lines in include/linux/skbuff.h, v6.12, and the documentation notes that “all of the LCO implementations use a helper function lco_csum()”:
static inline __wsum lco_csum(struct sk_buff *skb)
{
unsigned char *csum_start = skb_checksum_start(skb);
unsigned char *l4_hdr = skb_transport_header(skb);
__wsum partial;
/* Start with complement of inner checksum adjustment */
partial = ~csum_unfold(*(__force __sum16 *)(csum_start +
skb->csum_offset));
/* Add in checksum of our headers (incl. outer checksum
* adjustment filled in by caller) and return result.
*/
return csum_partial(l4_hdr, csum_start - l4_hdr, partial);
}skb_checksum_start(skb) is skb->head + skb->csum_start — the pointer at which the device will begin summing, which for a tunnelled packet points at the inner transport header. skb_transport_header(skb) at this moment points at the outer UDP header, because the tunnel driver has already pushed the outer headers and reset the transport-header offset. The two pointers straddle exactly the header stack in the middle of the frame.
*(__sum16 *)(csum_start + skb->csum_offset) reads back the value the kernel itself prefilled into the inner checksum field — the P of the derivation. csum_unfold() widens the 16-bit value into a 32-bit __wsum without changing its numeric value, and the leading ~ complements it, producing ~P: the future sum of region 2, obtained by a single 16-bit load.
csum_partial(l4_hdr, csum_start - l4_hdr, partial) then sums region 1 — from the outer UDP header up to (not including) csum_start — seeded with ~P as the running total. csum_partial()’s third argument is an initial value, so this single call performs “sum region 1, then add region 2’s known total.” The length argument csum_start - l4_hdr is pure pointer arithmetic: for the VXLAN-over-IPv4 frame in the diagram it is 8 + 8 + 14 + 20 = 50 bytes, regardless of how large the payload is.
The caller in udp_set_csum() — branch (3) of the four-way switch quoted earlier — then does the rest:
} else if (skb->ip_summed == CHECKSUM_PARTIAL) { /* LCO */
uh->check = 0;
uh->check = udp_v4_check(len, saddr, daddr, lco_csum(skb));
if (uh->check == 0)
uh->check = CSUM_MANGLED_0;
}uh->check = 0 first, because region 1 includes the outer UDP checksum field itself, and a field that will be overwritten must contribute nothing to the sum that produces it. Then udp_v4_check(len, saddr, daddr, <the LCO sum>) folds in the outer pseudo-header — outer source and destination addresses, protocol, UDP length — and complements, producing the finished outer checksum, which is written into the packet immediately, in software. The CSUM_MANGLED_0 guard is the RFC 768 “zero means no checksum” rule discussed earlier, applied to the outer header.
Critically, skb->ip_summed, csum_start and csum_offset are left untouched. The skb is still CHECKSUM_PARTIAL, still pointing at the inner checksum. So the division of labour ends up being:
sequenceDiagram autonumber participant TCP as inner TCP/UDP<br/>(transport layer) participant TUN as tunnel driver<br/>(vxlan / geneve) participant LCO as udp_set_csum()<br/>+ lco_csum() participant NIC as NIC (plain<br/>TX csum offload) TCP->>TCP: build inner segment;<br/>ip_summed = CHECKSUM_PARTIAL<br/>csum_start = inner transport hdr<br/>write P = folded pseudo-header sum TCP->>TUN: hand off skb TUN->>TUN: push VXLAN + outer UDP + outer IP<br/>(csum_start still points at inner hdr) TUN->>LCO: udp_set_csum(…) LCO->>LCO: read back P (one 16-bit load) LCO->>LCO: sum ~50 B of headers, seeded with ~P LCO->>LCO: write FINAL outer UDP checksum Note over LCO: ip_summed / csum_start / csum_offset<br/>deliberately NOT changed LCO->>NIC: transmit NIC->>NIC: sum from csum_start to end,<br/>complement, store at csum_start+csum_offset Note over NIC: this writes the INNER checksum —<br/>and by the identity, doing so<br/>makes the OUTER one correct too
The LCO hand-off, in order. What it shows: the outer checksum is finished by software before the packet reaches the device, and the device is asked for exactly one ordinary, protocol-agnostic checksum — the inner one — using the same csum_start/csum_offset pair it would use for a non-tunnelled packet. The insight to take: LCO needs no tunnel awareness in the hardware whatsoever. A card that has never heard of VXLAN, that has one generic start/offset checksum engine, can carry fully checksummed VXLAN traffic at line rate. This is the “protocol-independent mechanism” David Miller was holding out for, working exactly as designed — and it is why the ossification argument is not merely a complaint but a design with a concrete alternative behind it.
The self-consistency is worth stating explicitly because it looks circular and is not: the outer checksum written now is correct only after the device later writes the inner one. Between those two moments the packet on the wire would fail both checks. That interval is entirely inside the transmit path, and nothing observes the buffer in between — but it is the reason LCO cannot be combined with anything that inspects or captures the outer checksum after udp_set_csum() and before the device runs. A packet captured by tcpdump on the egress path shows the outer checksum as correct and the inner checksum as the raw pseudo-header sum, which pcap tooling reports as “incorrect.” This is the single most common false alarm in tunnel debugging, and it is the same “checksum incorrect — should be 0x…” artefact that plain CHECKSUM_PARTIAL produces, now appearing one layer deeper.
Nesting, and what it costs
The documentation addresses stacked tunnels directly: “LCO can safely be used for nested encapsulations; in this case, the outer encapsulation layer will sum over both its own header and the ‘middle’ header. This does mean that the ‘middle’ header will get summed multiple times, but there doesn’t seem to be a way to avoid that without incurring bigger costs (e.g. in SKB bloat).”
The reason the recursion is safe is that the identity is applied at each layer against whatever csum_start currently is, and each layer’s csum_partial() call simply covers a slightly longer stretch of headers. The reason it is mildly wasteful is that the middle headers fall inside region 1 of every enclosing layer, so an n-deep stack sums the innermost headers n times. Since the headers are tens of bytes and are already in L1 cache, the cost is measured in nanoseconds; the alternative — caching per-layer partial sums in the skb — would enlarge struct sk_buff, which the networking maintainers treat as close to sacred (see struct sk_buff for why every bit in that structure is contested).
LCO’s coverage in-tree is not uniform, and the documentation is explicit about the gap: it is used in udp_set_csum() and udp6_set_csum() for UDP-based tunnels, and in net/ipv4/ip_gre.c:build_header() for IPv4 Generic Routing Encapsulation (GRE), but “it is not currently performed when constructing an IPv6 GRE header; the GRE checksum is computed over the whole packet in net/ipv6/ip6_gre.c:ip6gre_xmit2(), but it should be possible to use LCO here as IPv6 GRE still uses an IP-style checksum.” That is a documented, unfixed performance gap: IPv6 GRE with checksums enabled sums the entire payload in software on every transmit, where IPv4 GRE does not.
Uncertain
Verify: whether the IPv6 GRE LCO gap described in
checksum-offloads.rstis still open in mainline after v6.12. Reason: in-tree documentation goes stale, and this passage reads as a long-standing “TODO” rather than a current statement of fact; the note’s version pin is the v6.12 long-term-support branch, and mainline is in the 7.x series as of 2026-09-04. To resolve: readnet/ipv6/ip6_gre.c:ip6gre_xmit2()at a current mainline tag and check whether it callslco_csum()or still calls a whole-packetcsum_partial(). The claim as written is verified for v6.12 against the v6.12 documentation and is dated accordingly.#uncertain
Remote Checksum Offload: Making the Receiver Derive the Inner Checksum
LCO removes the software cost of the outer checksum on transmit. Remote Checksum Offload (RCO) attacks the other half of the problem: the inner checksum, on a card that cannot parse the tunnel. Where LCO is a private arithmetic optimisation with no wire visibility, RCO is a genuine protocol change — the sender writes metadata into the encapsulation header, and a receiver that does not understand that metadata will mis-handle the packet. The kernel documentation is blunt about the consequence: “It does, however, involve a change to the encapsulation protocols, which the receiver must also support. For this reason, it is disabled by default.”
The mechanism was specified by Tom Herbert in two Internet-Drafts, both read in full for this note: the generic Remote checksum offload for encapsulation (draft-herbert-remotecsumoffload-00, 27 August 2014, Informational) and the VXLAN binding Remote checksum offload for VXLAN (draft-herbert-vxlan-rco-00, 1 December 2014, Experimental). Both expired without becoming RFCs — February and June 2015 respectively — which is itself a fact worth carrying: RCO is a shipping Linux feature whose only specification is an expired individual draft. The abstract states the goal precisely: it “provides checksum offload of encapsulated packets using rudimentary offload capabilities found in most Network Interface Card (NIC) devices.”
The idea
Recall the two receive states. A dumb card can offload the outer UDP checksum, because that is an ordinary UDP datagram as far as it is concerned. What it cannot do is find and verify the inner TCP checksum buried 42 bytes further in. RCO’s observation is that it does not have to — the receiving host can derive the inner checksum from the outer one, arithmetically, for free, if the sender tells it where to look.
The draft’s receiver algorithm is four steps:
- “Receive packet and validate outer checksum following normal processing.”
- “Deduce full checksum for the IP packet. This is directly provided if device returns the packet checksum in
CHECKSUM_COMPLETE. If the device returnedCHECKSUM_UNNECESSARY, then the complete checksum can be trivially derived as either zero (GRE) or the bitwise not of the outer pseudo header (UDP).” - “From the packet checksum, subtract the checksum computed from the start of the packet (outer IP header) to the offset in the packet indicated by checksum start in the meta data. The result is the deduced checksum to set in the checksum field of the encapsulated transport packet.”
- “Write the resultant checksum value into the packet at the offset provided by checksum offset in the meta data.”
Step 3 is property C — a one’s-complement sum over a region is the sum over its parts, so subtracting the prefix leaves the suffix — used in the one direction the earlier sections have not yet exercised. The sender’s contribution is only the metadata: a csum_start/csum_offset pair, the exact same two numbers the transmit contract uses, shipped across the wire instead of down to a device.
The wire format
The generic draft defines the metadata as a plain 32-bit field, “a pair of checksum start and checksum offset values,” and then leaves the binding to each encapsulation. VXLAN’s binding is tighter, because the VXLAN header has no spare 32 bits — it steals the low byte of the Virtual Network Identifier (VNI) word (include/net/vxlan.h, v6.12):
packet-beta 0-3: "R R R R" 4: "I" 5-9: "R R R R R" 10: "C" 11-31: "Reserved" 32-55: "VXLAN Network Identifier (VNI) — 24 bits" 56: "O" 57-63: "Csum start / 2 — 7 bits"
The 8-byte VXLAN header with the Remote Checksum Offload option, drawn at bit accuracy from the ASCII diagram in include/net/vxlan.h. What it shows: RCO occupies exactly one flag bit and one byte. C (bit 10, VXLAN_HF_RCO) says the option is present; O is the offset selector; the low seven bits carry the checksum start, halved. The insight to take: the seven-bit field is the whole design constraint. Csum start is stored divided by two, so it can express only even offsets from 0 to 254 (VXLAN_MAX_REMCSUM_START = 0x7f << 1 = 254 bytes), and the offset is not a number at all but a single bit selecting UDP’s checksum field or TCP’s. Everything RCO can and cannot carry follows from having had one spare byte to work with.
The constants make the encoding literal:
#define VXLAN_HF_RCO cpu_to_be32(BIT(21)) /* the 'C' flag */
#define VXLAN_RCO_MASK cpu_to_be32(0x7f) /* Last byte of vni field */
#define VXLAN_RCO_UDP cpu_to_be32(0x80) /* Indicate UDP RCO (TCP when not set *) */
#define VXLAN_RCO_SHIFT 1 /* Left shift of start */
#define VXLAN_MAX_REMCSUM_START (0x7f << VXLAN_RCO_SHIFT)and the decoders are one line each: vxlan_rco_start() returns be32_to_cpu(vni_field & VXLAN_RCO_MASK) << VXLAN_RCO_SHIFT, and vxlan_rco_offset() returns offsetof(struct udphdr, check) (6) or offsetof(struct tcphdr, check) (16) depending on the VXLAN_RCO_UDP bit. Bit 21 of the big-endian flags word is the C position in the ASCII picture the header file itself carries; the file cites the draft URL directly.
Transmit: vxlan_build_skb() and the four-condition gate
On transmit, the sending VXLAN device must decide whether RCO is even expressible for this packet (drivers/net/vxlan/vxlan_core.c, v6.12):
if ((vxflags & VXLAN_F_REMCSUM_TX) &&
skb->ip_summed == CHECKSUM_PARTIAL) {
int csum_start = skb_checksum_start_offset(skb);
if (csum_start <= VXLAN_MAX_REMCSUM_START &&
!(csum_start & VXLAN_RCO_SHIFT_MASK) &&
(skb->csum_offset == offsetof(struct udphdr, check) ||
skb->csum_offset == offsetof(struct tcphdr, check)))
type |= SKB_GSO_TUNNEL_REMCSUM;
}Four conditions, each one a direct consequence of the one-byte encoding: the administrator enabled VXLAN_F_REMCSUM_TX for this remote destination; the packet is in CHECKSUM_PARTIAL (there is an offload request to relay in the first place); csum_start fits in 254 bytes and is even; and csum_offset is one of the two values the single O bit can name. A packet with a long inner header stack — stacked VLAN tags, IPv6 extension headers, IP options — silently fails the first arithmetic test and falls back to ordinary processing. RCO is best-effort and unobservable when it declines, which matters when reasoning about why a benchmark did not improve.
If the gate passes, the header is written and — for a non-GSO packet — the inner checksum request is cancelled outright:
if (type & SKB_GSO_TUNNEL_REMCSUM) {
unsigned int start;
start = skb_checksum_start_offset(skb) - sizeof(struct vxlanhdr);
vxh->vx_vni |= vxlan_compute_rco(start, skb->csum_offset);
vxh->vx_flags |= VXLAN_HF_RCO;
if (!skb_is_gso(skb)) {
skb->ip_summed = CHECKSUM_NONE;
skb->encapsulation = 0;
}
}skb->ip_summed = CHECKSUM_NONE is the whole point and is worth staring at. The inner checksum field is left holding the pseudo-header prefill P, and nobody — not the kernel, not the device — will ever replace it. skb->encapsulation = 0 erases the fact that this is a tunnel at all, so that no downstream code tries to apply inner/outer feature harmonisation to it (see Segmentation Offloads GSO TSO for what encapsulation normally drives).
The packet then reaches udp_tunnel_xmit_skb(), which calls udp_set_csum() — and because ip_summed is now CHECKSUM_NONE, it takes branch (4), the plain one: set CHECKSUM_PARTIAL, point csum_start at the outer UDP header, csum_offset = 6, prefill the outer pseudo-header sum. The card is now looking at what appears to be an ordinary, unremarkable UDP datagram with a single checksum to compute.
flowchart LR subgraph TX["transmitting host"] A["inner TCP builds segment<br/>CHECKSUM_PARTIAL → inner hdr<br/>field holds P = pseudo sum"] --> B{"REMCSUM_TX enabled<br/>AND csum_start ≤ 254<br/>AND even<br/>AND offset is 6 or 16 ?"} B -->|"no"| Z["ordinary path:<br/>LCO for the outer sum,<br/>device does the inner one"] B -->|"yes"| C["write C flag + start/2 + O bit<br/>into the VXLAN header"] C --> D["<b>ip_summed = CHECKSUM_NONE</b><br/>encapsulation = 0<br/><i>inner checksum abandoned,<br/>field still holds P</i>"] D --> E["udp_set_csum() branch 4:<br/>CHECKSUM_PARTIAL → <b>outer</b> UDP hdr"] end E ==>|"one plain UDP datagram<br/>as far as the card can tell"| F["NIC: sum from outer UDP hdr<br/>to end of frame, write outer csum"] F ==> G(("wire")) subgraph RX["receiving host"] G ==> H["NIC verifies the outer UDP csum<br/>→ CHECKSUM_COMPLETE or<br/>CHECKSUM_UNNECESSARY"] H --> I["vxlan_remcsum():<br/>read C flag, decode start/offset"] I --> J["skb_remcsum_process()"] end
RCO end to end. What it shows: between the two dashed hand-offs the frame is, to every piece of silicon that touches it, a plain UDP datagram with exactly one checksum. The tunnel-awareness that a CHECKSUM_UNNECESSARY-style card would need has been replaced by seven bits of metadata and arithmetic at the far end. The insight to take: the inner checksum is computed by nobody, on either host, in the common case — not by the sender’s CPU, not by the sender’s NIC, not by the receiver’s NIC, and (as the next subsection shows) not by the receiver’s CPU either. That is why the technique is worth a protocol change; it is not a shifting of cost but an elimination of it.
Receive: skb_remcsum_process() and its two very different modes
The receiving VXLAN device calls vxlan_remcsum(), which decodes the metadata and hands off (include/linux/skbuff.h, v6.12):
static inline void skb_remcsum_process(struct sk_buff *skb, void *ptr,
int start, int offset, bool nopartial)
{
__wsum delta;
if (!nopartial) {
skb_remcsum_adjust_partial(skb, ptr, start, offset);
return;
}
if (unlikely(skb->ip_summed != CHECKSUM_COMPLETE)) {
__skb_checksum_complete(skb);
skb_postpull_rcsum(skb, skb->data, ptr - (void *)skb->data);
}
delta = remcsum_adjust(ptr, skb->csum, start, offset);
/* Adjust skb->csum since we changed the packet */
skb->csum = csum_add(skb->csum, delta);
}ptr is (void *)(vxlan_hdr(skb) + 1) — the first byte after the VXLAN header, i.e. the start of the inner Ethernet frame — and start/offset are the decoded metadata, relative to ptr. The nopartial flag distinguishes two completely different behaviours.
The default path (nopartial == false) does no arithmetic at all. It calls:
static inline void skb_remcsum_adjust_partial(struct sk_buff *skb, void *ptr,
u16 start, u16 offset)
{
skb->ip_summed = CHECKSUM_PARTIAL;
skb->csum_start = ((unsigned char *)ptr + start) - skb->head;
skb->csum_offset = offset - start;
}Three assignments. This is the CHECKSUM_PARTIAL-on-receive state described earlier in this note, reached deliberately: the skb is marked as “everything up to csum_start is verified; the 16 bits at csum_start + csum_offset are not filled in and need not be.” The inner transport layer, on finding CHECKSUM_PARTIAL, accepts the segment without summing it. The inner checksum field on the wire still holds the sender’s prefill and is never corrected — which is exactly why the receiver must understand RCO, and why an RCO-unaware receiver would compute a checksum failure on every packet. The kernel’s own documentation of CHECKSUM_PARTIAL on receive names this case in its list of origins: “it may be set in the input path in GRO or remote checksum offload.”
The nopartial path (VXLAN_F_REMCSUM_NOPARTIAL) actually derives and writes the checksum, because some consumers cannot cope with a CHECKSUM_PARTIAL skb — a packet about to be forwarded out of a bridge to a virtual machine, say, or handed to a CHECKSUM_COMPLETE-expecting consumer. It first forces a genuine CHECKSUM_COMPLETE (summing in software if the device did not provide one), rebases skb->csum onto ptr with skb_postpull_rcsum(), and then calls the arithmetic core in include/net/checksum.h, v6.12:
static __always_inline __wsum remcsum_adjust(void *ptr, __wsum csum,
int start, int offset)
{
__sum16 *psum = (__sum16 *)(ptr + offset);
__wsum delta;
/* Subtract out checksum up to start */
csum = csum_sub(csum, csum_partial(ptr, start, 0));
/* Set derived checksum in packet */
delta = csum_sub((__force __wsum)csum_fold(csum),
(__force __wsum)*psum);
*psum = csum_fold(csum);
return delta;
}Walk it. On entry csum is the one’s-complement sum from ptr to the end of the packet. csum_sub(csum, csum_partial(ptr, start, 0)) removes the bytes from ptr up to ptr + start — the inner Ethernet and IP headers — leaving the sum over the inner transport segment alone. That segment currently contains P, the pseudo-header sum, in its checksum field. csum_fold() folds to 16 bits and complements, so csum_fold(csum) = ~(sum-of-segment-with-P-in-place) = ~(P +' sum-of-segment-with-zero), which is by definition the correct transport checksum. It is written into *psum. The returned delta is the difference between the new and old 16-bit values, added back into skb->csum by the caller so that the CHECKSUM_COMPLETE value stays truthful about a packet that has just been edited — property D, incremental update, applied to the kernel’s own bookkeeping.
Note the asymmetry in cost. The default path is three stores. The nopartial path sums the inner headers, and if the device did not supply CHECKSUM_COMPLETE it sums the entire packet first. VXLAN_F_REMCSUM_NOPARTIAL therefore converts RCO from a free optimisation into something that can be slower than not using RCO at all.
| LCO — Local Checksum Offload | RCO — Remote Checksum Offload | |
|---|---|---|
| What it saves | the software sum of the outer checksum on transmit | the computation of the inner checksum, on both hosts |
| Wire-visible? | no — pure sender-side arithmetic | yes — a flag and a byte in the encapsulation header |
| Receiver must cooperate? | no | yes, or every packet appears corrupt |
| Default state | always on, wherever implemented | off; per-tunnel VXLAN_F_REMCSUM_TX / _RX |
| Hardware requirement | generic start/offset TX offload | generic start/offset TX offload + any RX checksum |
| Specification | none — an in-tree technique, documented in checksum-offloads.rst | two expired Internet-Drafts (2014/2015) |
| Failure when unsupported | n/a — invisible | inner checksum appears wrong on every packet |
| Encoding limits | none | csum_start ≤ 254 and even; offset ∈ {6, 16} (VXLAN) |
| Interaction with TSO | fine; the segmenter fixes each segment | draft §3.4: device replicates the metadata per segment |
LCO versus RCO. What it shows: they solve adjacent halves of the same problem and are routinely confused, but they sit on opposite sides of the most important line in protocol design — whether the wire format changes. The insight to take: LCO is free and universal because it asks nothing of anyone else; RCO is off by default and largely undeployed because it asks the whole path to agree. That difference, not the arithmetic, is why one is everywhere and the other is a flag you have to know to look for.
Both drafts’ Security Considerations sections say, in full, “Remote checksum offload should not impact protocol security.” That is a strikingly thin analysis for a mechanism in which a sender-supplied offset causes a receiver to write bytes into a packet buffer, and the kernel treats it accordingly: vxlan_remcsum() guards the decoded offset with pskb_may_pull(skb, offset + sizeof(u16)) before skb_remcsum_process() touches anything, and the seven-bit encoding bounds start at 254 by construction. The bound is a property of the encoding, not of the validation — which is worth remembering for any future binding of RCO to an encapsulation with a wider metadata field.
Failure Modes: Why a Broken Offload Never Looks Like a Networking Bug
Checksum offload has a signature that makes it uniquely miserable to diagnose: when it goes wrong, nothing in the failure resembles a checksum problem. The checksum is the mechanism that detects corruption, so a checksum that is computed wrongly does not report corruption — it creates it, or it hides it. The symptom lands on whoever is furthest from the cause.
flowchart TB Q0{"which side is broken?"} Q0 -->|"transmit"| TX["the packet leaves with a<br/><b>wrong or absent</b> checksum"] Q0 -->|"receive"| RX["the device claims a checksum<br/>is good when it is not"] TX --> TX1["peer's stack discards the segment"] TX1 --> TXS["<b>symptom:</b> TCP stalls, retransmits<br/>forever, connection resets;<br/>UDP simply vanishes.<br/>Both hosts' logs are clean."] TXS --> TXC["<b>counter, on the PEER:</b><br/>Tcp: InCsumErrors / InErrs<br/>Udp: InCsumErrors<br/>(/proc/net/snmp)"] TXC --> TXD["<b>bisect:</b> ethtool -K <dev> tx off<br/>if it heals, it is TX offload"] RX --> RX1["corrupt bytes are accepted<br/>and delivered to the application"] RX1 --> RXS["<b>symptom:</b> silent data corruption —<br/>a bad byte in a file transfer,<br/>a garbled database row.<br/><b>No counter anywhere moves.</b>"] RXS --> RXD["<b>bisect:</b> ethtool -K <dev> rx off<br/>forces software validation;<br/>errors now appear in InCsumErrors"] TX --> ALT["<b>or:</b> the offload was requested<br/>from a device that cannot do it"] ALT --> ALTS["skb_warn_bad_offload() in dmesg:<br/><code>caps=(dev-features, sk_route_caps)</code><br/>plus a full skb hex dump"]
The two failure directions and their utterly different signatures. What it shows: a transmit-side fault is loud but reported on a machine you may not own; a receive-side fault is silent and reported nowhere at all. The insight to take: the asymmetry dictates the debugging order. For a stall, look at the peer’s counters and suspect TX. For corruption with clean counters everywhere, suspect RX — and note that “clean counters” is not evidence of health, it is the expected reading for the worst failure mode in this subsystem.
The false alarm: tcpdump on the sender
Before any of the real failures, the one that is not a failure. tcpdump on a sending host taps the packet at ptype_all — inside __netif_receive_skb_core’s transmit twin, dev_queue_xmit_nit(), which runs before the driver hands the buffer to the device. At that point a CHECKSUM_PARTIAL packet’s checksum field holds the pseudo-header prefill, not the checksum, so tcpdump -vv prints:
IP host.51234 > peer.443: Flags [P.], ... cksum 0xec59 (incorrect -> 0xb1e7), length 1448
0xec59 is the prefill; 0xb1e7 is what tcpdump computed itself and what the NIC will shortly write. This is correct operation. It appears on essentially every outbound packet from every modern Linux host, and it is the single most reported non-bug in the subsystem. The tells that it is benign: it appears only on locally originated packets, only in captures taken on the sending host, and the “should be” value is what actually arrives at the peer. tcpdump -K (or --no-checksum-verification) suppresses the check entirely. Under LCO the same artefact appears on the inner checksum of a tunnelled packet, one layer deeper, which is why tunnel debugging generates a fresh wave of the same false report.
Failure 1 — a genuinely broken transmit offload
A driver or firmware bug that writes the checksum at the wrong offset, sums the wrong range, or omits the pseudo-header produces packets that every receiver in the world discards. The sending host sees nothing: its own counters record successful transmission, ethtool -S shows no errors, dmesg is silent. The peer sees InCsumErrors climbing. On a machine you control, the counters live in /proc/net/snmp and are surfaced by netstat -s; this host, after 214 million received TCP segments, reads:
Tcp: ... InSegs 214540288 ... InErrs 341 OutRsts 402954 InCsumErrors 0
IpExt: ... InCsumErrors: 3
Udp: ... InCsumErrors 0
Three facts to read out of that. First, Tcp: InCsumErrors is 0 across 214 million segments — a healthy path, and the baseline against which any nonzero reading is alarming. Second, InErrs is 341 while InCsumErrors is 0: the two are not the same counter. tcp_v4_do_rcv() bumps both on a checksum failure — TCP_INC_STATS(net, TCP_MIB_CSUMERRORS); TCP_INC_STATS(net, TCP_MIB_INERRS); (net/ipv4/tcp_ipv4.c, v6.12) — so InErrs is the superset and the difference between them is non-checksum errors, here 341 malformed or truncated segments. Comparing the two is the cheap first diagnostic: if InErrs climbs and InCsumErrors does not, the problem is not a checksum. Third, IpExt: InCsumErrors: 3 counts IPv4 header checksum failures, which is a different checksum entirely (16 bits over 20 bytes of header, validated by ip_fast_csum() in ip_rcv_core()), and three failures in 264 million packets is background noise from an unreliable link, not an offload bug.
The kernel-side gate is tcp_checksum_complete() (include/net/tcp.h, v6.12), and it is a compact statement of everything in the receive section of this note:
static inline bool tcp_checksum_complete(struct sk_buff *skb)
{
return !skb_csum_unnecessary(skb) &&
__skb_checksum_complete(skb);
}Short-circuit evaluation is the offload: if skb_csum_unnecessary() is true — ip_summed is CHECKSUM_UNNECESSARY, or CHECKSUM_COMPLETE with csum_valid already set — the second operand is never evaluated and no byte of payload is read. Only when the device gave nothing usable does __skb_checksum_complete() walk the packet. A drop from this path is tagged SKB_DROP_REASON_TCP_CSUM and fires the tcp_bad_csum tracepoint, so perf trace -e tcp:tcp_bad_csum or a bpftrace one-liner will show the offending packets live, with the socket, rather than only a counter.
Failure 2 — a receive offload that lies
This is the dangerous one. A device that reports CHECKSUM_UNNECESSARY for a packet whose checksum is in fact wrong causes the kernel to skip validation entirely and hand corrupt bytes to the application. There is no counter for “the hardware lied,” because the only instrument that could detect it is the checksum the hardware told us not to compute.
The kernel’s partial defences were covered earlier — netdev_rx_csum_fault() and the CHECKSUM_COMPLETE cross-check, which catch a disagreeing device but not a confidently wrong one. The operational defence is a bisect: ethtool -K <dev> rx off clears NETIF_F_RXCSUM, which forces the driver to report CHECKSUM_NONE and the stack to sum every packet in software. If corruption stops, the device’s receive offload is the culprit. If corruption continues but InCsumErrors now climbs, the corruption is happening on the wire or upstream and the offload was merely hiding it — which is an equally useful answer and a much more common one. Either way the experiment converts a silent failure into a counted one, which is the whole objective.
Historically this class of bug has been serious enough that vendors ship errata for it, and it is the reason the kernel’s guidance to driver authors is so insistent that a device which cannot fully verify a protocol must report CHECKSUM_COMPLETE or CHECKSUM_NONE rather than guessing CHECKSUM_UNNECESSARY.
Uncertain
Verify: specific named NIC errata in which a device reported
CHECKSUM_UNNECESSARYfor packets whose checksum was invalid. Reason: the failure mode is described in general terms in the kernel’s documentation and is the stated rationale for thenetdev_rx_csum_fault()warning, but pinning it to a named part and driver commit requires thenetdevlist archives, andlore.kernel.orgis behind an Anubis proof-of-work bot check as of 2026-09-04. To resolve: search GitHub’s mirror of the kernel history for driver commits touchingCHECKSUM_UNNECESSARYwith “errata” or “hardware bug” in the message —github.com/torvalds/linux/commits/<tag>/<path>.atomfor the full 40-character SHAs andgithub.com/torvalds/linux/commit/<sha>.patchfor verbatim messages, sincegit.kernel.orgcgit is itself behind the bot check as of 2026-09-04 — or read the vendor specification updates directly. The mechanism as described is verified against in-tree source and documentation; the incident history is not.#uncertain
Failure 3 — the feature-mismatch path, and what caps= is telling you
The third class is neither a hardware nor a driver bug but a bookkeeping failure: a socket built a large CHECKSUM_PARTIAL packet on the belief that the egress device could finish it, and by the time the packet reached the device that was no longer true. Its fingerprint is the skb_warn_bad_offload() output described in the software-fallback section: a driver name, a hex dump, and the two feature masks caps=(dev->features, sk->sk_route_caps). The mask pair is the diagnosis. Common causes, all of which change a device’s features underneath a live socket:
- an
ethtool -Kissued while connections are open; - a bonding or team slave changing state, so the master recomputes its features from the intersection of its members;
- a VLAN, bridge, or tunnel netdev whose feature propagation to the underlying device is wrong or incomplete — the classic container-networking version, where a
vethpair advertisesNETIF_F_HW_CSUM(as this machine’s loopback does,tx-checksum-ip-generic: on [fixed]) and the packet later escapes onto a physical NIC that does not; - a route change that moves an established flow onto a different, less capable device.
The last two are the reason CHECKSUM_PARTIAL packets escaping a virtual device onto a physical one is a recurring bug family. It is worth being precise about what does not go wrong here: validate_xmit_skb() catches the mismatch and skb_checksum_help() fixes it in software, so the packet is correct. What you lose is performance — and, as the software-fallback section showed, potentially a full payload copy from __skb_linearize() on top of the sum. A mismatch that fires on every packet of a bulk transfer can halve throughput while every counter reads clean and dmesg shows a single WARN_ONCE.
A diagnostic sequence that works
- Confirm the direction. Corruption or loss seen by the peer → suspect TX. Corruption seen locally with clean counters → suspect RX.
- Read the counters on the receiving side:
netstat -s | grep -i csum, or/proc/net/snmpdirectly. CompareInCsumErrorsagainstInErrs; if only the latter moves, stop looking at checksums. - Bisect with
ethtool -K, one direction at a time —tx off, retest, back on; thenrx off, retest. Remember from the feature-dependency section thattx offalso drags Scatter-Gather and every segmentation offload down with it, so a “fix” fromtx offimplicates a set of offloads, not just the checksum;ethtool -K <dev> tx-checksum-ip-generic off(ortx-checksum-ipv4 off) isolates it more precisely. - Read
dmesgforcaps=lines and forbad partial csum: csum=…/… headroom=… headlen=…fromskb_partial_csum_set(). - Capture on both ends. A capture on the sender that shows “incorrect” and a capture on the receiver that shows correct is the normal picture. A capture on the receiver that shows “incorrect” is a real transmit fault.
- Trace, do not guess.
perf trace -e tcp:tcp_bad_csum, and theskb:kfree_skbtracepoint with its drop reason, name the packet and the exact reason.SKB_DROP_REASON_TCP_CSUMandSKB_DROP_REASON_UDP_CSUMare unambiguous.
Alternatives: The Four States as a Design Space
There is no “alternative to checksum offload” in the sense of a competing technology; the design space is entirely internal, and it consists of choosing which of the four ip_summed states a device reports or requests. The choice is the interesting part, and the kernel’s preferences are not neutral.
| State | Direction | What it means | Hardware cost | Kernel cost | When it is the right answer |
|---|---|---|---|---|---|
CHECKSUM_NONE | both | nothing was done | zero | full software sum | honest fallback; the only correct report for a packet the device could not parse |
CHECKSUM_UNNECESSARY | RX | “I parsed it and it is valid” | a protocol parser in silicon, fixed at tape-out | zero, plus csum_level bookkeeping per decapsulation | plain TCP/UDP over IPv4/IPv6 on a card that already has the parser |
CHECKSUM_COMPLETE | RX | “here is the sum of all bytes” | a single accumulator, no parsing | one pseudo-header add per layer, from cache | the kernel’s stated preference; the only state that survives an unknown encapsulation |
CHECKSUM_PARTIAL | TX | “sum from here, store there” | one generic start/offset engine | prefill the pseudo-header sum | the only transmit state worth having; also the prerequisite for TSO |
CHECKSUM_PARTIAL | RX | “this was never summed and need not be” | none | none | veth, virtio_net, GRO, and the RCO receive path |
The design space, and the fact that it is not symmetric. What it shows: on transmit there is really one useful state, because csum_start/csum_offset is fully general — there is nothing a protocol-specific transmit offload can do that the generic one cannot. On receive there are two genuine competitors, and they differ in what kind of thing the device is claiming. The insight to take: the kernel documentation’s rule — “Even if device supports only some protocols, but is able to produce skb->csum, it MUST use CHECKSUM_COMPLETE, not CHECKSUM_UNNECESSARY” — is not a style preference. It is the single lever the kernel has against protocol ossification, and every card that ships with a parser instead of an accumulator narrows the set of protocols that can ever be deployed cheaply.
The same asymmetry shows up in the feature flags. NETIF_F_HW_CSUM is the generic transmit capability; NETIF_F_IP_CSUM and NETIF_F_IPV6_CSUM are the protocol-specific ancestors that the kernel has been trying to retire for years. Reading a real device is instructive. This machine’s Realtek 5 Gigabit Ethernet controller, driven by r8169, reports:
Features for enp191s0:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
tx-checksum-ip-generic: off [fixed] is NETIF_F_HW_CSUM absent and unavailable, while tx-checksum-ipv4 and tx-checksum-ipv6 are on: this 2024-era consumer part still exposes the protocol-specific pair rather than the generic engine. Contrast the same query against the loopback device, which is pure software and therefore has no excuse:
rx-checksumming: on [fixed]
tx-checksumming: on
tx-checksum-ipv4: off [fixed]
tx-checksum-ip-generic: on [fixed]
tx-checksum-ipv6: off [fixed]
tx-checksum-sctp: on [fixed]
Exactly the inverse: NETIF_F_HW_CSUM on, the protocol-specific flags off. The ossification argument in the tunnels section is not abstract — it is legible in two ethtool -k outputs on any Linux machine, and the direction of travel is that virtual devices are generic and physical ones still are not. (Both readings taken 2026-09-04 on Linux 7.1.8; the flag semantics are as defined in net/ethtool/common.c at v6.12, and the [fixed] marker means the driver did not place the bit in hw_features, so ethtool -K cannot change it.)
For the encapsulated case the alternatives are the two techniques of the preceding sections, and the choice between them is not really a performance question:
- LCO alone — free, invisible, always on, no cooperation needed. This is what essentially all VXLAN and GENEVE deployments actually run.
- LCO + RCO — additionally elides the inner checksum, but requires the receiver to speak the same extension, is off by default, is bounded by a seven-bit offset field, and rests on expired drafts.
- Hardware tunnel offload (
NETIF_F_GSO_UDP_TUNNEL_CSUMand friends, plus a card that parses VXLAN) — fastest where it exists, and precisely the ossification the maintainers pushed back against, because it works only for the encapsulations that were fashionable when the silicon was designed.
Production Notes
Never turn checksum offload off as a “fix.” It is the first thing suggested in every forum thread and it is almost always the wrong action, because ethtool -K <dev> tx off walks the dependency graph described earlier and takes Scatter-Gather, TSO, and GSO down with it. Throughput on a 10 Gigabit link can fall by an order of magnitude, and the change is not obviously reversible across a reboot or a driver reload. Use it as a bisection step — off, retest, on — not as a remedy. If a specific offload really must stay disabled, disable the narrowest flag that reproduces the fix (tx-checksum-ip-generic, not tx) and record why.
The [fixed] marker is the most useful column in ethtool -k. It means the driver never put the bit into dev->hw_features, so the feature is not merely off — it cannot be turned on, and any runbook step that says “enable it” will silently do nothing. Both outputs above show several. Checking for [fixed] before writing an automation step saves a class of no-op configuration management.
Virtual devices advertise checksum offload aggressively, and they are telling the truth. veth, virtio_net, lo, and bridge ports set NETIF_F_HW_CSUM because a packet that never crosses a physical medium cannot be corrupted in transit, so computing a checksum for it is pure waste. This is correct and desirable — a CHECKSUM_PARTIAL skb travelling between two containers on one host is never summed by anyone. It becomes a problem only at the boundary where such a packet reaches a real NIC, which is why container networking has a long history of checksum bugs concentrated at exactly that seam. When debugging container traffic, establish first which hop the packet was on when it broke, because the correct behaviour on one side of the veth is a bug on the other.
In virtual machines, CHECKSUM_PARTIAL crosses a trust boundary. virtio_net receives csum_start and csum_offset from the other side of the hypervisor interface and must run them through skb_partial_csum_set(), which bounds-checks both. The ratelimited bad partial csum: csum=… message in a host’s dmesg is a guest or a device model sending offsets the host rejects — a correctness win, but also a signal worth alerting on, since a stream of them means every packet from that guest is being dropped or repaired.
Watch for perf top showing csum_partial or do_csum high in a profile. That is the software fallback running, and on a machine whose NIC advertises checksum offload it always means something upstream is defeating it: a feature mismatch, an skb with shared fragments forcing __skb_linearize(), a GSO packet being segmented in software, or an encapsulation the LCO path does not cover. The profile is often the only signal, because — unlike a drop — the fallback produces no counter and no log line. It is worth internalising that this subsystem’s normal failure is a performance regression with no error anywhere, and that a CPU profile is therefore a first-class checksum-offload diagnostic rather than a last resort.
Tunnel MTU interacts with all of this. LCO’s saving is proportional to payload size, so it matters most for GSO super-packets; a tunnel configured with a small MTU, or a path that forces per-packet transmission, spends proportionally more time in header arithmetic and gets less benefit. Conversely, an RCO deployment that never triggers because inner headers exceed the 254-byte csum_start bound will show no improvement at all, and there is no counter that says so — only reading the VXLAN flags on a capture will tell you whether the C bit is actually being set.
See Also
- struct sk_buff — where
ip_summed,csum,csum_start,csum_offset,csum_level,csum_valid,csum_complete_swandcsum_not_inetphysically live, and how a clone copies them - Segmentation Offloads GSO TSO — segmentation,
gso_type/gso_size,skb_segment(), and why TSO is impossible withoutCHECKSUM_PARTIAL - skb_shared_info and Paged Fragments — the paged fragments that force
__skb_linearize()in the software fallback - The Network Transmit Path —
validate_xmit_skb()and whereskb_csum_hwoffload_help()is called - The Network Receive Path — where a driver sets
ip_summedon receive, and whereptypedispatch hands the packet to a protocol that will test it - Generic Receive Offload — GRO, which both consumes and produces checksum state while coalescing
- Network Device Drivers and net_device —
dev->features/hw_features, where the flags this note reads actually live - Tunnel and Overlay Interfaces — VXLAN and GENEVE, the encapsulations whose headers carry the Remote Checksum Offload option
- XDP Express Data Path — the bypass that runs before any of this, and must maintain checksum state itself if it edits a packet
- The TCP Protocol in Linux / The UDP Protocol in Linux — the transport layers that prefill the pseudo-header sum and validate on receive
- Drawing Wire Formats with Mermaid Packet Diagrams — the house style for the
packet-betadiagram of the VXLAN RCO header above - MOC: Linux Networking Stack MOC