vault

Connection Tracking conntrack

15 min read

Connection Tracking conntrack

Connection tracking (the nf_conntrack subsystem, historically ip_conntrack) is the part of netfilter that makes the firewall stateful. Without it, every packet is judged in isolation; with it, the kernel remembers flows — it records the first packet of every connection in a hash table keyed by a tuple (source/destination address, source/destination port, and protocol), then recognizes every subsequent packet as belonging to that flow and labels it with a state (NEW, ESTABLISHED, RELATED, or — as a non-stored match bit — INVALID). This is the basis of both stateful firewalling (“allow replies to connections I started, without an explicit rule for the return traffic”) and NAT (which stores its address rewrite in the conntrack entry so both directions are translated consistently). Everything below pins to Linux 6.12 LTS source (net/netfilter/nf_conntrack_core.c, as of 2026-06).

Conntrack registers as a netfilter hook function (nf_conntrack_in, at priority -200 on PRE_ROUTING/LOCAL_OUT) — it is a consumer of the hook framework, so read The Netfilter Framework and Hooks first for where it sits in the pipeline. The address-rewriting layer that builds on it is Network Address Translation NAT.

Mental Model: One Entry, Two Tuples, Both Directions

A conntrack entry is a single struct nf_conn object that represents one bidirectional flow but stores it as two tuples — the original direction (the way the connection was opened) and the reply direction (the inverse). Both tuples are inserted into the same hash table, so a packet arriving from either side finds the same nf_conn. The genius of the design is the inverse tuple: for a TCP connection 10.0.0.1:5000 → 1.1.1.1:443, the original tuple is {src 10.0.0.1:5000, dst 1.1.1.1:443, TCP} and the reply tuple is its inverse {src 1.1.1.1:443, dst 10.0.0.1:5000, TCP}. When the SYN-ACK comes back, its tuple equals the reply tuple of the existing entry, so the kernel instantly knows it is the return traffic of a known flow — and labels it ESTABLISHED rather than treating it as a brand-new connection that a firewall would have to explicitly permit.

flowchart TB
  P1["First packet:<br/>SYN 10.0.0.1:5000 -> 1.1.1.1:443"] --> LOOK{"tuple in<br/>hash table?"}
  LOOK -->|"no match"| NEW["init_conntrack():<br/>allocate nf_conn<br/>ctinfo = IP_CT_NEW"]
  NEW --> STORE["store ORIGINAL tuple<br/>+ inverted REPLY tuple<br/>(unconfirmed)"]
  STORE --> CONF["packet survives rules<br/>confirm hook inserts<br/>both tuples into table"]
  P2["Reply packet:<br/>SYN-ACK 1.1.1.1:443 -> 10.0.0.1:5000"] --> LOOK2{"tuple in<br/>hash table?"}
  LOOK2 -->|"matches REPLY tuple"| EST["set IPS_SEEN_REPLY<br/>ctinfo = ESTABLISHED_REPLY<br/>future packets = ESTABLISHED"]

The lifecycle of one connection in conntrack. What it shows: the first packet finds no matching tuple, so a new nf_conn is allocated with the original tuple and its computed inverse (reply) tuple, marked IP_CT_NEW, and only confirmed (inserted into the live hash) after it survives the ruleset; the reply packet’s tuple matches the stored reply tuple, flipping the connection to seen-reply / ESTABLISHED. The insight to take: conntrack stores a flow as two tuples precisely so a single hash lookup on any packet — from either side — lands on the same entry, which is what makes stateful matching and consistent two-way NAT possible.

The Tuple — What Identifies a Flow

The key is struct nf_conntrack_tuple (include/net/netfilter/nf_conntrack_tuple.h), whose source comment states it plainly: *“A tuple' is a structure containing the information to uniquely identify a connection. ie. if two packets have the same tuple, they are in the same connection; if not, they are not."* It is split into a **manipulable** part (src — what NAT may rewrite) and a **fixed** part (dst`):

struct nf_conntrack_tuple {
	struct nf_conntrack_man src;   /* manipulable: src addr (u3), src port (u), l3num */
	struct {
		union nf_inet_addr u3;     /* dst address */
		union { __be16 all; struct { __be16 port; } tcp; /* ...udp/icmp/sctp/gre... */ } u;
		u_int8_t protonum;         /* L4 protocol: TCP/UDP/ICMP/... */
		u_int8_t dir;              /* direction (original vs reply) */
	} dst;
};

So the identifying fields are the classic 5-tuple — source IP, source port, destination IP, destination port, and the Layer-4 protocol number — plus the Layer-3 protocol (l3num, e.g. AF_INET/AF_INET6) and a direction byte. For protocols without ports (ICMP, GRE) the u union holds a protocol-specific identifier instead: ICMP uses {type, code, id}, GRE uses a key. The split into src (manipulable) and dst (fixed) exists for NAT: SNAT rewrites src, DNAT rewrites dst, and conntrack records what the rewrite was so it can be reversed on the reply.

The States

The state a packet is given is the enum ip_conntrack_info value (ctinfo) stored on the skb via nf_ct_set(skb, ct, ctinfo). The enum (include/uapi/linux/netfilter/nf_conntrack_common.h) is:

enum ip_conntrack_info {
	IP_CT_ESTABLISHED,       /* 0: part of an established connection */
	IP_CT_RELATED,           /* 1: related to an existing connection */
	IP_CT_NEW,               /* 2: started a new connection to track */
	IP_CT_IS_REPLY,          /* 3: >= this indicates reply direction */
	IP_CT_ESTABLISHED_REPLY = IP_CT_ESTABLISHED + IP_CT_IS_REPLY,
	IP_CT_RELATED_REPLY = IP_CT_RELATED + IP_CT_IS_REPLY,
	IP_CT_UNTRACKED = 7,     /* (in-kernel) deliberately not tracked */
};

How ctinfo is computed lives in resolve_normal_ct() (nf_conntrack_core.c), and reading it is the clearest possible explanation of the states:

if (NF_CT_DIRECTION(h) == IP_CT_DIR_REPLY) {
	ctinfo = IP_CT_ESTABLISHED_REPLY;     /* matched the reply tuple */
} else {
	unsigned long status = READ_ONCE(ct->status);
	if (likely(status & IPS_SEEN_REPLY))
		ctinfo = IP_CT_ESTABLISHED;       /* we've seen both directions */
	else if (status & IPS_EXPECTED)
		ctinfo = IP_CT_RELATED;           /* expected by a helper */
	else
		ctinfo = IP_CT_NEW;               /* original dir, no reply yet */
}
  • NEW — the packet matches a conntrack entry that has only ever seen traffic in the original direction (no reply yet). In practice this is the first packet of a flow: resolve_normal_ct finds no existing tuple, calls init_conntrack() to allocate the nf_conn, and the entry has not yet seen a reply, so ctinfo is IP_CT_NEW.
  • ESTABLISHED — the connection has seen traffic both ways. The IPS_SEEN_REPLY status bit is set the first time a packet matches the reply tuple (in nf_conntrack_in, test_and_set_bit(IPS_SEEN_REPLY_BIT, ...)). After that, both directions are ESTABLISHED. The comment in the source is exact: “Once we’ve had two way comms, always ESTABLISHED.”
  • RELATED — the packet belongs to a different flow that is logically connected to an existing one. The classic case is an ICMP “destination unreachable” error carrying the headers of a tracked flow, or an FTP data connection on a separate port that the FTP control connection’s helper predicted. RELATED is set when the matched entry has IPS_EXPECTED (it was created from an expectation — see Helpers below).
  • INVALID — not a stored state. This is the trap. INVALID is not a value in enum ip_conntrack_info; it is a match bit (NF_CT_STATE_INVALID_BIT = 1 << 0, in nf_conntrack_common.h) that the iptables/nftables ctstate match reports when a packet could not be associated with any tracked connection. In nf_conntrack_in(), the paths that bump the per-namespace invalid statistic and return NF_ACCEPT without attaching a conntrack to the skb (e.g. a packet whose L4 header cannot be parsed, a checksum failure when nf_conntrack_checksum is on, or a TCP segment that the TCP tracker judges out-of-window) are what userspace rules see as INVALID. So NEW/ESTABLISHED/RELATED are ctinfo values carried on the skb; INVALID and UNTRACKED are ctstate match bits the firewall tests — a distinction the kernel headers make crisp but secondary docs routinely blur.

A typical stateful firewall ruleset is therefore: ct state established,related accept (let replies and related traffic through with no per-service rule), ct state new rules to decide what new connections are allowed, and ct state invalid drop to discard packets that fit no flow.

Mechanical Walk-through: nf_conntrack_in

The hook entry point is nf_conntrack_in() (nf_conntrack_core.c), registered at priority -200. Step by step for a packet:

  1. Already tracked? nf_ct_get(skb, &ctinfo) checks if a conntrack is already attached (e.g. loopback, or an UNTRACKED packet from the raw table’s NOTRACK); if so it returns NF_ACCEPT immediately.
  2. Find the L4 protocol. get_l4proto() parses down to the transport header; if it cannot, the invalid counter is bumped and the packet is accepted untracked.
  3. ICMP special-case. ICMP/ICMPv6 errors are handled by nf_conntrack_handle_icmp, which can associate the error with the flow it references (this is how an ICMP error becomes RELATED).
  4. Resolve the tuple. resolve_normal_ct() computes the packet’s tuple, hashes it (hash_conntrack_raw), and looks it up with __nf_conntrack_find_get. On a miss it calls init_conntrack() to allocate a new nf_conn (with the original tuple and its computed inverse reply tuple). It then sets ctinfo via the logic shown above and attaches the conntrack to the skb with nf_ct_set.
  5. Protocol state machine. nf_conntrack_handle_packet() dispatches to the per-protocol tracker (nf_conntrack_tcp_packet, nf_conntrack_udp_packet, etc.), which validates the packet against the protocol’s state machine (for TCP: is this a valid segment for the current TCP state?) and refreshes the timeout via nf_ct_refresh_acct. A protocol tracker can return -NF_REPEAT (TCP reopen of a closed connection) which loops back to step 4 to build a fresh entry, or a drop.
  6. Mark seen-reply. If this packet matched the reply tuple, IPS_SEEN_REPLY is set and an IPCT_REPLY event is cached.

Crucially, a new entry is not yet in the live hash table — it is unconfirmed. It is only confirmed (__nf_conntrack_confirm, registered at priority INT_MAX on POST_ROUTING/LOCAL_IN) once the packet has survived the entire ruleset and is about to leave the box. This deferral is deliberate: if a firewall rule drops the first packet, no half-formed entry pollutes the table, and only the original direction triggers confirmation (if (CTINFO2DIR(ctinfo) != IP_CT_DIR_ORIGINAL) return NF_ACCEPT;).

The Conntrack Table: Size, Hashing, and Limits

The flow store is a hash table of struct nf_conntrack_tuple_hash nodes (two per nf_conn — one per direction), with these tunables (Documentation/networking/nf_conntrack-sysctl.rst):

  • nf_conntrack_buckets — number of hash buckets. Default is total RAM / 16384, clamped to [1024, 262144]. Writable only in the initial namespace.
  • nf_conntrack_max — the hard cap on tracked entries. Defaults to nf_conntrack_buckets. The doc is explicit and worth internalizing: because each connection occupies two tuple slots (original + reply), a maxed-out table with max == buckets has an average hash chain length of 2, not 1.
  • nf_conntrack_count — read-only current count.

When a new entry would exceed nf_conntrack_max, __nf_conntrack_alloc() does not simply fail — it first calls early_drop(), which scans a hash bucket for an entry that has not yet seen a reply (!IPS_ASSURED) and evicts it to make room. Only if early-drop finds nothing does allocation fail with -ENOMEM, the packet is dropped, the drop stat increments, and the kernel logs the infamous nf_conntrack: table full, dropping packet (net_warn_ratelimited). A background garbage-collection worker (gc_worker) also walks the table periodically, expiring timed-out entries and, when pressure is high (above 95% of max), early-dropping non-assured flows.

Timeouts per Protocol

Every flow has a timeout that is refreshed on each packet (nf_ct_refresh_acct); when it expires, the gc worker reaps the entry. Timeouts are per-protocol and, for TCP, per-state (nf_conntrack-sysctl.rst):

  • TCP established: nf_conntrack_tcp_timeout_established — default 432000 s (5 days). This is the famous one: a long-idle TCP connection stays tracked for five days by default, which is why connection-heavy workloads exhaust the table.
  • TCP syn_sent 120 s, syn_recv 60 s, fin_wait 120 s, time_wait 120 s, close 10 s, close_wait 60 s, last_ack 30 s.
  • UDP: nf_conntrack_udp_timeout 30 s, rising to nf_conntrack_udp_timeout_stream 120 s once a stream is detected (UDP has no states, so conntrack synthesizes “is this a stream” from seeing replies).
  • ICMP: 30 s. GRE: 30 s (180 s for a stream). Generic (unknown L4): 600 s.

These are the levers operators reach for: dropping tcp_timeout_established from 5 days to, say, an hour dramatically cuts table occupancy on a busy box, at the cost of breaking genuinely long-idle connections that resume after the timeout.

Helpers (Application-Layer Gateways)

Some protocols negotiate secondary connections on ports chosen at runtime — File Transfer Protocol (FTP) in active mode opens a data connection on a port announced inside the control stream; Session Initiation Protocol (SIP) negotiates RTP media ports the same way. A stateful firewall that only knew the control connection would block the data connection as NEW/unexpected. Conntrack helpers (Application-Layer Gateways, ALGs) parse the control stream, extract the negotiated address/port, and register an expectation — a pending tuple that, when matched by an incoming packet, creates a conntrack entry flagged IPS_EXPECTED, which resolve_normal_ct then labels RELATED. That is the whole mechanism behind “ct state related accept lets FTP data through.” Helpers run at priority +300 (NF_IP_PRI_CONNTRACK_HELPER), after the main lookup but before confirm. Helpers are also a historic security risk (a malicious payload can spoof an expectation, e.g. CVE-2017-style helper abuses), which is why the recommended modern practice is to attach helpers explicitly via a CT target / ct helper rule rather than relying on automatic port-based assignment.

Uncertain

The exact default of automatic helper assignment (the nf_conntrack_helper module parameter / sysctl that historically gated whether helpers auto-attach by port) is not pinned to a primary source in this note. Reason: the per-helper auto-assign default has shifted across kernel versions and was not re-verified against the 6.12 nf_conntrack_helper.c source here. To resolve: check the nf_conntrack_helper parameter default and the nf_conntrack-sysctl docs at v6.12. uncertain

Inspecting Conntrack

The table is exposed two ways. The /proc/net/nf_conntrack file (when CONFIG_NF_CONNTRACK_PROCFS is set) dumps one line per entry; far more usefully, the conntrack userspace tool (from conntrack-tools, over the ctnetlink interface) gives structured access:

# conntrack -L
tcp  6 431999 ESTABLISHED src=10.0.0.1 dst=1.1.1.1 sport=5000 dport=443 \
        src=1.1.1.1 dst=10.0.0.1 sport=443 dport=5000 [ASSURED] mark=0 use=1

Reading the line: protocol tcp (L4 number 6), the remaining-timeout 431999 seconds (≈5 days, an established TCP flow), the state ESTABLISHED, then the two tuples — the first src=/dst=/sport=/dport= set is the original direction, the second is the reply direction (note it is the inverse). [ASSURED] means the flow has the IPS_ASSURED status bit set (it has seen reply traffic) and is therefore skipped by early_drop_list — verified in nf_conntrack_core.c, early_drop_list does if (test_bit(IPS_ASSURED_BIT, &tmp->status) ... ) continue;, so only non-assured entries are evictable under pressure; [UNREPLIED] would mark a NEW-only flow eligible for eviction.

Uncertain

The exact textual format of the conntrack -L output line above (field order, the bracketed [ASSURED]/[UNREPLIED] tokens, the mark=/use= trailers) is reproduced from familiarity with conntrack-tools, not from fetching the tool’s manual or source during this task. The semantics ([ASSURED] ⇔ IPS_ASSURED, two-tuple original/reply layout) are verified against the 6.12 kernel headers and early_drop_list. Reason: conntrack-tools is a separate userspace project not in the kernel tree. To resolve: confirm the precise output format against the conntrack(8) man page / conntrack-tools source. uncertain Other invocations: conntrack -L -p tcp --dport 443 filters, conntrack -E streams live events, conntrack -D ... deletes entries, and conntrack -C (or reading nf_conntrack_count) gives the current count for capacity monitoring.

Failure Modes

Table full → packet drops. The single most common conntrack production failure. Symptom: intermittent connection failures under load and nf_conntrack: table full, dropping packet in dmesg. Diagnosis: compare nf_conntrack_count to nf_conntrack_max. Fix: raise nf_conntrack_max (and nf_conntrack_buckets proportionally — leaving buckets small makes hash chains long and lookups slow), and/or shorten the established-TCP timeout. On a Kubernetes node this manifests as flaky pod networking; it is why CNIs and node tuning guides bump these sysctls.

Asymmetric routing breaks state. Conntrack assumes it sees both directions of a flow. If the reply path bypasses the box (asymmetric routing, ECMP without flow affinity), the reply tuple is never matched, the flow stays NEW/UNREPLIED, and a ct state invalid drop rule (or a NAT that needs the reply) silently breaks the connection. This is a frequent cause of “works in one direction only” bugs in multi-homed setups.

The per-packet cost at scale. Conntrack does a hash lookup, refcount, and timeout-refresh on every packet of every flow, plus an insert on the first packet — under local_bh_disable()/RCU. At millions of flows this is real CPU and cache pressure; the table itself consumes memory (each nf_conn is hundreds of bytes). This is exactly why high-performance paths bypass it: XDP-based load balancers (Katran) and Cilium’s eBPF datapath implement their own flow tables to avoid nf_conntrack, and the kernel’s flowtable offload (IPS_OFFLOAD) lets established flows skip the full conntrack/rule traversal after the first few packets.

INVALID from out-of-window TCP. Strict TCP window tracking can mark legitimately reordered or retransmitted segments INVALID, causing drops. The nf_conntrack_tcp_be_liberal sysctl loosens this (only out-of-window RSTs are INVALID), a common fix for “some TCP connections randomly stall behind the firewall.”

Alternatives and When to Choose Them

  • No conntrack (stateless rules). You can firewall statelessly (match on addresses/ports per packet) and pay zero conntrack cost — but you lose established,related and must hand-write return-traffic rules, and you cannot do NAT (which requires conntrack to store the mapping). Choose stateless only for pure high-rate drop/allow with no NAT, ideally in XDP.
  • eBPF flow tables (Cilium, Katran). For service-mesh / L4-LB scale, an eBPF map-based flow table in XDP or tc-BPF replaces nf_conntrack with a purpose-built, cheaper structure. Choose this when conntrack is the bottleneck and you control the datapath.
  • Flowtable offload. Keep conntrack but offload established flows to the nf_flowtable fast path (optionally to hardware, IPS_HW_OFFLOAD) so they bypass full rule evaluation. Best of both worlds for a forwarding box with steady long-lived flows.

See Also

Graph View

Backlinks