vault

Network Namespaces

56 min read

Network Namespaces

A network namespace (universally abbreviated netns) is a Linux kernel facility that gives a group of processes a completely private copy of the network stack — its own network interfaces, IPv4 and IPv6 protocol state, routing tables, neighbour (ARP) tables, firewall rules, port-number space, /proc/net and /sys/class/net trees, the per-namespace settings under /proc/sys/net, and even the UNIX-domain abstract socket namespace (network_namespaces(7)). Inside the kernel a netns is one kmem_cache-allocated struct net object (defined in include/net/net_namespace.h); creating a namespace allocates a fresh struct net, and the networking code is written so that nearly every table it once kept as a kernel-wide global is instead reached through a struct net pointer. A new network namespace starts almost empty — it has a fresh, down loopback device and nothing else — so it is isolated by construction: a process inside it cannot see the host’s interfaces, cannot reach the host’s routes, and binds ports in its own private port space. This is the single kernel primitive that makes container network isolation possible; everything a CNI plugin assembles for a Pod is built on top of one netns per Pod.

Network namespaces are one of the eight namespace types Linux supports (alongside mount, PID, user, UTS, IPC, cgroup, and time namespaces) — see namespaces(7). This note is about the networking one. It is not the Kubernetes Namespace API object (a multi-tenancy/RBAC partition of an API server) — that lives in Namespace and is an entirely unrelated concept that happens to share the word.

This note is pinned to Linux 6.12, a maintained long-term-support (LTS) release (mainline is now in the 7.x series); every structure, function and line of code quoted below was read from the v6.12 tree during this write-up, and anything that changed after 6.12 is dated explicitly. Manual-page quotations come from man-pages 6.18 as published on man7.org.

The general namespace machinery — how clone(2), unshare(2) and setns(2) work, what /proc/<pid>/ns/ contains, how the ucounts budget is charged, and the pivotal relationship between a namespace and its owning user namespace — is documented once in User Namespaces and Mount Namespaces, and this note assumes it rather than repeating it. What follows is what is specific to the network stack: what precisely is duplicated, how a socket reaches its copy, how packets cross the boundary, and what container runtimes actually build on top.

Mental Model — /proc/net Is a Per-Namespace View

The cleanest way to think about a netns is: there is no global network state; there is only “the network state reachable from this struct net.” Every table you might once have imagined as a single kernel-wide object — the interface list, the Forwarding Information Base (FIB) routing trie, the neighbour cache, the connection-tracking table, the netfilter rule chains, the hash of bound Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) ports — is a field inside struct net. There is one struct net per namespace. The host’s is init_net, declared as a single global struct net init_net; at net/core/net_namespace.c:48. Every other namespace is a kmem_cache_zalloc()’d sibling of equal standing, allocated from a dedicated slab cache created at boot as kmem_cache_create("net_namespace", sizeof(struct net), ...) (net_ns_init(), line 1168).

flowchart TB
  subgraph KERN["Kernel — one struct net per namespace, all peers"]
    direction LR
    subgraph INIT["init_net — the host (global symbol)"]
      I_LO["lo (up, ifindex 1)"]
      I_ETH["eth0 — the physical NIC"]
      I_FIB["FIB / routing tables"]
      I_NF["netfilter + conntrack"]
      I_PORTS["bound-port hashes"]
      I_SYS["/proc/sys/net/* values"]
    end
    subgraph NS1["struct net #1 — container A"]
      A_LO["lo (DOWN at birth, ifindex 1)"]
      A_VE["eth0 = veth peer"]
      A_FIB["own FIB — empty at birth"]
      A_NF["own netfilter — empty"]
      A_PORTS["own port space"]
      A_SYS["own sysctl values"]
    end
    subgraph NS2["struct net #2 — container B"]
      B_LO["lo (ifindex 1 here too)"]
      B_VE["eth0 = veth peer"]
      B_FIB["own FIB"]
    end
  end
  P1["task in container A<br/>task_struct.nsproxy.net_ns"] -.->|points at| NS1
  P2["task in container B"] -.->|points at| NS2
  P0["task on the host"] -.->|points at| INIT

The kernel holds one struct net per network namespace, all peers of the single global init_net. What it shows: a process reaches its network stack indirectly — task_struct → nsproxy → net_ns selects which struct net, and therefore which interface list, FIB, netfilter ruleset, sysctl values and port space it sees. The insight to take: isolation here is not a permission check layered over shared state; it is separate copies of the data structures. Container A cannot name container B’s eth0 because that interface is registered in a different struct net — there is no lookup path that would find it. Note also that lo is interface index 1 in every namespace: the kernel asserts this with BUG_ON(dev->ifindex != LOOPBACK_IFINDEX) in loopback_net_init() (drivers/net/loopback.c), so an interface index is only meaningful paired with a namespace.

What Is Actually Namespaced — Reading struct net

The authoritative list of what a netns isolates is the layout of struct net itself: if a subsystem’s state is a field in that struct, it is per-namespace; if it is a file-scope global elsewhere, it is not. In v6.12 the struct embeds a long series of per-subsystem sub-structures directly by value (lines 124–190), each one a subsystem saying “my state is namespaced”:

classDiagram
  class struct_net {
    +ns_common ns  «inode number seen at /proc/PID/ns/net»
    +user_namespace* user_ns  «owning user ns»
    +ucounts* ucounts  «max_net_namespaces budget»
    +list_head dev_base_head  «every netdev in this ns»
    +hlist_head* dev_name_head, dev_index_head
    +net_device* loopback_dev  «this ns own lo»
    +u32 hash_mix  «per-ns key salt»
    +proc_dir_entry* proc_net  «/proc/net for this ns»
    +ctl_table_set sysctls  «/proc/sys/net for this ns»
    +sock* rtnl, genl_sock  «rtnetlink socket»
    +refcount_t passive  «weak ref»
  }
  class netns_core { somaxconn, optmem_max, txrehash, rps_default_mask }
  class netns_ipv4 { fib_table_hash, udp_table, tcp_death_row, ip_local_ports, tcp_congestion_control, devconf_all, ~200 sysctls }
  class netns_ipv6 { fib6 tables, devconf, sysctls }
  class netns_nf { netfilter hook arrays }
  class netns_ct { conntrack table + sysctls }
  class netns_nftables { nftables rule state }
  class netns_xfrm { IPsec policy + SAs }
  class netns_unix { AF_UNIX abstract-name table }
  class netns_packet { AF_PACKET socket list }
  class netns_bpf { attached BPF programs }
  class netns_xdp { AF_XDP state }
  class netns_mpls { MPLS routes }
  class netns_can { CAN bus state }
  class netns_mctp { MCTP state }
  class netns_smc { SMC state }
  struct_net *-- netns_core
  struct_net *-- netns_ipv4
  struct_net *-- netns_ipv6
  struct_net *-- netns_nf
  struct_net *-- netns_ct
  struct_net *-- netns_nftables
  struct_net *-- netns_xfrm
  struct_net *-- netns_unix
  struct_net *-- netns_packet
  struct_net *-- netns_bpf
  struct_net *-- netns_xdp
  struct_net *-- netns_mpls
  struct_net *-- netns_can
  struct_net *-- netns_mctp
  struct_net *-- netns_smc

The composition of struct net in Linux 6.12, drawn from the declaration in include/net/net_namespace.h. What it shows: a netns is not an abstract “isolation policy” — it is a concrete C struct that contains, by value, one copy of each networking subsystem’s state. The insight to take: you can answer “is X namespaced?” by looking for X in this picture. IPsec policy is (netns_xfrm), so is the conntrack table (netns_ct), so is the UNIX-domain abstract socket namespace (netns_unix — which is why abstract sockets are scoped to a netns and not to a mount namespace). Several members are compiled out entirely when their CONFIG_ option is off, so a stripped kernel literally has a smaller namespace.

The user-visible list

network_namespaces(7) enumerates the same thing from the outside: “network devices, IPv4 and IPv6 protocol stacks, IP routing tables, firewall rules, the /proc/net directory …, the /sys/class/net directory, various files under /proc/sys/net, port numbers (sockets), and so on. In addition, network namespaces isolate the UNIX domain abstract socket namespace.” Worth spelling out concretely, because each entry has a consequence people trip over:

ResourceWhere it lives in struct netConsequence in practice
Network interfacesdev_base_head, dev_name_head, dev_index_head, dev_by_indexAn interface exists in exactly one namespace. ip link in a container cannot even name the host’s eth0. Interface indices restart per namespace, so ifindex 3 is ambiguous without saying which netns.
Loopbackloopback_devEvery namespace gets its own lo, always at ifindex 1, and it is down at birth.
Routing tables (FIB)ipv4.fib_table_hash, IPv6 equivalentsA container has an empty routing table until something installs routes. Policy-routing rules (ip rule) are per-namespace too (rules_ops).
Neighbour/ARP tablesinside the per-namespace device stateARP entries learned in one namespace are invisible in another.
netfilter / nftables / conntracknf, nft, ctiptables -L inside a container shows an empty ruleset even when the host has hundreds of rules. Conntrack entries are per-namespace, and so are the conntrack limits — nf_conntrack_max in a container does not consume the host’s budget.
Port spaceipv4.udp_table, plus per-namespace hash salting of the shared TCP tables (below)A hundred containers can each bind 0.0.0.0:80.
sysctlssysctls, plus the ~200 fields in netns_ipv4net.ipv4.ip_forward, tcp_congestion_control, ip_local_port_range (default 32768 60999) and ip_unprivileged_port_start are all per-namespace; the last is documented as “This is a per-namespace sysctl. It defines the first unprivileged port in the network namespace” (ip-sysctl.rst), which is exactly how a container can be allowed to bind port 80 without CAP_NET_BIND_SERVICE.
/proc/net, /sys/class/netproc_net, proc_net_statThese are views onto the current namespace, which is why ip netns exec has to remount /sys (see below).
AF_UNIX abstract namesunxTwo containers can each bind the abstract name @/tmp/.X11-unix/X0; the host cannot reach either.
AF_PACKET socketspacketA raw packet-capture socket only sees the interfaces of its own namespace — tcpdump inside a container captures only container traffic.

The port-space nuance most explanations get wrong

It is tempting to say “each namespace has its own bind hash table.” For UDP that is optionally true and for TCP it is not true — and the real mechanism is more interesting. The TCP established and bind hash tables are single kernel-wide structures; the namespace is folded into the hash key instead. Each struct net gets a random 32-bit salt at creation, get_random_bytes(&net->hash_mix, sizeof(u32)) in preinit_net(), and the hash functions mix it in:

/* include/net/inet_hashtables.h, v6.12 */
static inline u32 inet_bhashfn(const struct net *net, const __u16 lport,
			       const u32 bhash_size)
{
	return (lport + net_hash_mix(net)) & (bhash_size - 1);
}
 
/* net/ipv4/inet_hashtables.c, v6.12 */
u32 inet_ehashfn(const struct net *net, const __be32 laddr, const __u16 lport,
		 const __be32 faddr, const __be16 fport)
{
	net_get_random_once(&inet_ehash_secret, sizeof(inet_ehash_secret));
	return __inet_ehashfn(laddr, lport, faddr, fport,
			      inet_ehash_secret + net_hash_mix(net));
}

Reading it symbol by symbol: lport is the local port, net_hash_mix(net) returns that namespace’s random hash_mix, and the sum is masked down to a bucket index. So container A’s port 80 and container B’s port 80 hash to different buckets, and even in the rare bucket collision the socket comparison includes a net_eq() test. The insight: port isolation is achieved by making the namespace part of the key, not by duplicating the table — which is why binding a port in a namespace costs nothing extra, and why a pathological number of namespaces does not multiply the kernel’s hash-table memory.

UDP does allow real per-namespace tables, gated on a sysctl read from the creating namespace: udp_set_table() reads old_net->ipv4.sysctl_udp_child_hash_entries and, if non-zero, allocates a private struct udp_table for the child; otherwise net->ipv4.udp_table = &udp_table, the global one (net/ipv4/udp.c). This knob is a genuine scalability lever on hosts running thousands of UDP-heavy containers.

Uncertain

Verify: the exact semantics and stability of net.ipv4.udp_child_hash_entries. Reason: the field is read directly from net/ipv4/udp.c at v6.12, but it is not documented in Documentation/networking/ip-sysctl.rst at that tag (grepped; no match), so there is no primary prose description of its intended use, default, or supported range. To resolve: find the merging commit and its changelog on lore.kernel.org, or a later kernel where ip-sysctl.rst documents it. uncertain

How a Socket Finds Its Namespace

This is the mechanism that makes everything above work, and it is worth tracing precisely because the answer is not “the kernel looks up the current process’s namespace on every operation.”

A socket captures its namespace once, at socket(2) time, and carries a pointer to it for life. sock_create() is a one-liner that reads the caller’s namespace out of the task struct (net/socket.c):

int sock_create(int family, int type, int protocol, struct socket **res)
{
	return __sock_create(current->nsproxy->net_ns, family, type, protocol, res, 0);
}

That struct net * is threaded down into sk_alloc(net, family, priority, prot, kern), which stores it in the socket’s common header (net/core/sock.c):

sk->sk_net_refcnt = kern ? 0 : 1;
if (likely(sk->sk_net_refcnt)) {
	get_net_track(net, &sk->ns_tracker, priority);   /* real refcount */
	sock_inuse_add(net, 1);
} else {
	__netns_tracker_alloc(net, &sk->ns_tracker, false, priority);  /* tracking only */
}
sock_net_set(sk, net);        /* write_pnet(&sk->sk_net, net) */

Three things are packed into those seven lines:

  1. sock_net_set() writes the pointer. Thereafter every operation on that socket calls sock_net(sk) — literally read_pnet(&sk->sk_net) (include/net/sock.h) — to reach the routing tables, the netfilter hooks, the sysctl values. When CONFIG_NET_NS is off, read_pnet() compiles down to return &init_net;, which is how the whole feature costs nothing on a kernel built without it.
  2. A user socket pins its namespace; a kernel socket does not. sk_net_refcnt is 1 for sockets created on behalf of userspace and 0 for in-kernel sockets (the kern argument). A user socket takes a genuine reference with get_net_track(), so the namespace cannot be torn down while the socket lives. A kernel socket — the per-namespace rtnl netlink socket, for instance — deliberately takes no reference, because if it did, every namespace would hold a reference to itself and could never be freed.
  3. The namespace is fixed at creation. Nothing later re-reads current->nsproxy->net_ns for that socket.
flowchart TD
  A["socket(AF_INET, SOCK_STREAM, 0)"] --> B["sock_create()"]
  B --> C["reads current-&gt;nsproxy-&gt;net_ns<br/><b>ONCE</b>"]
  C --> D["__sock_create(net, ...)"]
  D --> E["sk_alloc(net, ...)"]
  E --> F{"kern flag?"}
  F -->|"kern = 0 (userspace)"| G["sk_net_refcnt = 1<br/>get_net_track() — pins the netns"]
  F -->|"kern = 1 (in-kernel)"| H["sk_net_refcnt = 0<br/>tracker only — does NOT pin"]
  G --> I["sock_net_set(sk, net)<br/>write_pnet(&amp;sk-&gt;sk_net, net)"]
  H --> I
  I --> J["every later bind/connect/send:<br/>sock_net(sk) -&gt; read_pnet()"]
  J --> K["FIB lookup, netfilter hooks,<br/>port hash, sysctl values —<br/>all reached through THAT struct net"]
  L["later: setns(fd, CLONE_NEWNET)"] -.->|"changes nsproxy-&gt;net_ns<br/><b>but not sk-&gt;sk_net</b>"| M["existing sockets keep<br/>the OLD namespace"]

How a socket is bound to a network namespace, and what setns() does and does not change. What it shows: the namespace is read from current->nsproxy->net_ns exactly once, at socket() time, written into the socket, and consulted from the socket forever after. The insight to take: setns(CLONE_NEWNET) changes which namespace new sockets will be created in — it does not migrate sockets you already hold. This is not a bug; it is the property that makes the classic privilege-separation pattern work: open a listening socket in the host namespace, then setns() (or unshare) into an empty namespace with no interfaces at all, and keep serving on the already-open fd while being unable to make any new outbound connection. LWN described exactly this design in 2014: “the file descriptor for that connection could be handled by a child process that is placed in a new network namespace … the lack of suitable network devices in the namespace would make it impossible for the child or worker process to make additional network connections” (Edge, LWN 2014).

The same “captured at open” rule applies to /proc/net files. seq_open_net() in fs/proc/proc_net.c resolves the namespace from the inode (net = get_proc_net(inode)), stores it in the file’s struct seq_net_private, and takes a reference on it; every later read calls seq_file_net(seq) to get that same pointer back (include/linux/seq_file_net.h). So an fd on /proc/net/tcp that you hold across a setns() keeps showing the old namespace’s sockets, and it pins that namespace alive for as long as the fd is open. If you want the new view, reopen the file.

Creating a Namespace: copy_net_ns() and the Pernet Registry

Creating a netns is clone(CLONE_NEWNET) or unshare(CLONE_NEWNET); the flag has existed since Linux 2.6.24 and, per LWN’s 2014 retrospective, “it took something approaching a year before they were ready for prime time.” The permission gate lives in the generic namespace code, not the networking code: unshare_nsproxy_namespaces() in kernel/nsproxy.c returns -EPERM unless ns_capable(user_ns, CAP_SYS_ADMIN) holds, and copy_namespaces() applies the same test on the clone() path. That is why rootless containers must create a user namespace first — inside a fresh user namespace the creator holds a full capability set, and CAP_SYS_ADMIN there is enough to unshare a netns. The rule and its consequences are worked through in User Namespaces; this note picks up at the networking half.

The networking half is copy_net_ns() (net/core/net_namespace.c:466):

struct net *copy_net_ns(unsigned long flags,
			struct user_namespace *user_ns, struct net *old_net)
{
	if (!(flags & CLONE_NEWNET))
		return get_net(old_net);          /* (1) not asked for: share the caller's */
 
	ucounts = inc_net_namespaces(user_ns);    /* (2) charge the ucounts budget */
	if (!ucounts)
		return ERR_PTR(-ENOSPC);
 
	net = net_alloc();                        /* (3) kmem_cache_zalloc from net_cachep */
	...
	preinit_net(net, user_ns);                /* (4) refcounts, hash_mix, owning user ns */
	net->ucounts = ucounts;
	get_user_ns(user_ns);
 
	rv = down_read_killable(&pernet_ops_rwsem);
	rv = setup_net(net);                      /* (5) run every subsystem's .init */
	up_read(&pernet_ops_rwsem);
	...
}

Step by step:

  1. The no-op path matters. Every clone() and fork() in the system goes through copy_net_ns(). When CLONE_NEWNET is absent it is a single refcount bump — the network namespace is inherited, not copied.
  2. inc_net_namespaces() is inc_ucount(ns, current_euid(), UCOUNT_NET_NAMESPACES), the per-user budget exposed as /proc/sys/user/max_net_namespaces (namespaces(7) documents it as “a per-user limit on the number of network namespaces that may be created in the user namespace”). Exceeding it yields ENOSPC, not ENOMEM — a fork bomb of unshare -n fails cleanly instead of exhausting kernel memory. The same ucounts chain guards mount and user namespaces; see User Namespaces.
  3. net_alloc() takes a zeroed object from the dedicated net_namespace slab cache. Note what it does not do: it does not copy anything from the parent. Unlike a mount namespace, which is born as a copy of its parent’s mount tree, a network namespace is born empty. This is the single biggest behavioural difference between the two, and it is why a fresh mount namespace still lets you read files while a fresh network namespace cannot ping anything.
  4. preinit_net() initializes the two reference counts (ns.count and passive), draws the per-namespace hash salt with get_random_bytes(&net->hash_mix, sizeof(u32)), and records net->user_ns = user_ns — the owning user namespace, which is what later capability checks are evaluated against.
  5. setup_net() is where a namespace acquires its contents. It walks the global pernet_list and calls ops_init(ops, net) for every registered struct pernet_operations. Each networking subsystem registers one of these at boot with register_pernet_subsys() / register_pernet_device(); its .init callback builds that subsystem’s per-namespace state. loopback_net_ops is one of them, which is exactly when the new lo springs into existence. If any .init fails, setup_net() unwinds by walking the list backwards calling .pre_exit, .exit and .free on everything that succeeded — a proper transactional rollback.
sequenceDiagram
    autonumber
    participant U as userspace
    participant NS as kernel/nsproxy.c
    participant NN as net/core/net_namespace.c
    participant PL as pernet_list
    participant LO as loopback_net_ops
    participant NF as netfilter/conntrack ops

    U->>NS: unshare(CLONE_NEWNET)
    NS->>NS: ns_capable(user_ns, CAP_SYS_ADMIN)?
    Note over NS: EPERM if not — rootless<br/>containers unshare CLONE_NEWUSER first
    NS->>NN: copy_net_ns(flags, user_ns, old_net)
    NN->>NN: inc_net_namespaces() — charge ucounts
    Note over NN: ENOSPC if over<br/>/proc/sys/user/max_net_namespaces
    NN->>NN: net_alloc() — zeroed struct net from slab
    NN->>NN: preinit_net() — refcounts, hash_mix, user_ns
    NN->>PL: setup_net(): for each pernet_operations
    PL->>LO: .init = loopback_net_init(net)
    LO-->>PL: alloc_netdev("lo"), register_netdev()<br/>BUG_ON(ifindex != 1); state DOWN
    PL->>NF: .init — empty hook arrays, empty conntrack table
    PL->>PL: every other registered subsystem:<br/>ipv4, ipv6, xfrm, nft, bpf, packet, unix, ...
    PL-->>NN: all .init succeeded
    NN->>NN: list_add_tail_rcu(&net->list, &net_namespace_list)
    NN-->>NS: struct net *
    NS-->>U: return 0 — caller now in the new namespace

The full creation path for a network namespace, from the syscall to a usable (if empty) stack. What it shows: a namespace is not “allocated” so much as constructed by a registry — setup_net() invites every networking subsystem in the kernel to build its own private copy of its state for the new namespace. The insight to take: this registry design explains both the emptiness and the cost. Emptiness, because each .init builds a fresh, default structure rather than copying the parent’s. Cost, because creating one namespace runs every registered subsystem’s .init callback under pernet_ops_rwsem, and — as the teardown section shows — destroying one runs the matching .exit callbacks on a single global, single-threaded workqueue.

Lifetime: What Keeps a Namespace Alive, and What Teardown Costs

A network namespace does not die when its last process exits. It dies when its last reference goes away, and processes are only one of several kinds of reference. struct net carries two counts: ns.count, the active count that keeps the namespace functional, and passive, a weaker count that only keeps the allocation alive so lingering pointers do not dangle. When ns.count reaches zero, __put_net() runs:

void __put_net(struct net *net)
{
	ref_tracker_dir_exit(&net->refcnt_tracker);
	/* Cleanup the network namespace in process context */
	if (llist_add(&net->cleanup_list, &cleanup_list))
		queue_work(netns_wq, &net_cleanup_work);
}

That is the whole hand-off: push the dying namespace onto a lock-free list and kick a work item. The critical detail is what netns_wq is, set up once at boot in net_ns_init():

netns_wq = create_singlethread_workqueue("netns");
if (!netns_wq)
	panic("Could not create netns workq");

Every network namespace in the system is destroyed by one single-threaded workqueue running one work item, cleanup_net(). There is no per-CPU parallelism and no per-container isolation of the work. cleanup_net() grabs the whole pending list at once with llist_del_all() — deliberately, so that bursts get batched — and then, for the batch, runs: every subsystem’s .pre_exit, a synchronize_rcu_expedited(), then rtnl_lock() + every .exit_batch_rtnl + unregister_netdevice_many() + rtnl_unlock(), then every .exit, then every .free, then a full rcu_barrier(), and only then net_free() on each namespace.

That sequence is the mechanical explanation for a symptom operators see constantly on dense container hosts: namespace teardown is asynchronous, globally serialized, and gated on two RCU grace periods plus the rtnl_mutex. A burst of pod deletions does not tear down namespaces in parallel; it lengthens one queue. Because rtnl_lock() is the same global mutex every ip command and every netlink link operation needs, a long teardown batch also stalls unrelated interface configuration.

stateDiagram-v2
    direction TB
    [*] --> Alive : copy_net_ns() succeeds<br/>ns.count = 1, passive = 1

    state Alive {
        [*] --> Referenced
        Referenced --> Referenced : a task's nsproxy-&gt;net_ns
        Referenced --> Referenced : an open fd on /proc/PID/ns/net
        Referenced --> Referenced : a <b>bind mount</b> of that nsfs file<br/>(this is the ip netns trick)
        Referenced --> Referenced : a user socket (sk_net_refcnt = 1)
        Referenced --> Referenced : an open /proc/net/* seq_file
    }

    Alive --> Dying : last reference dropped<br/>__put_net(): ns.count -&gt; 0
    Dying --> Queued : llist_add(&amp;net-&gt;cleanup_list)<br/>queue_work(netns_wq)

    state Queued {
        [*] --> WaitingForWorker
        note right of WaitingForWorker
            netns_wq is a SINGLE-THREADED,
            system-wide workqueue. Every
            dying namespace queues here.
        end note
    }

    Queued --> Cleanup : cleanup_net() runs<br/>llist_del_all() batches the queue

    state Cleanup {
        [*] --> PreExit : every ops-&gt;pre_exit
        PreExit --> RcuWait1 : synchronize_rcu_expedited()
        RcuWait1 --> RtnlPhase : rtnl_lock()<br/>exit_batch_rtnl + unregister_netdevice_many()
        RtnlPhase --> Exit : every ops-&gt;exit, then ops-&gt;free
        Exit --> RcuWait2 : rcu_barrier()
    }

    Cleanup --> Freed : net_free() — slab object returned<br/>dec_net_namespaces(), put_user_ns()
    Freed --> [*]

    note left of Cleanup
        Physical devices are pushed back to
        init_net here (renamed dev&lt;ifindex&gt;
        if the name is taken). Virtual devices
        (veth, macvlan, bridge) are DESTROYED.
    end note

The lifetime of a struct net, with the reference kinds that hold it in the Alive state and the phases teardown must pass through. What it shows: five distinct kinds of reference keep a namespace alive, only one of which is “a process is running in it” — and once the last one goes, the namespace enters a globally serialized, RCU-gated destruction pipeline. The insight to take: two operational facts fall straight out of this picture. First, “the container exited but the namespace is still there” is normal, not a leak: an open fd or a bind mount is enough. Second, high pod churn is slow to clean up by construction — the single-threaded netns_wq plus two RCU waits plus the rtnl_mutex means namespace destruction cannot be scaled by adding CPUs.

What happens to the interfaces

network_namespaces(7) states the rule tersely: “When a network namespace is freed …, its physical network devices are moved back to the initial network namespace (not to the namespace of the parent of the process).” The code that implements it, default_device_exit_net() in net/core/dev.c, is more discriminating than the man page suggests, and the details matter:

for_each_netdev_safe(net, dev, aux) {
	if (dev->netns_local)                    /* (a) loopback: skip, it dies with the ns */
		continue;
	if (dev->rtnl_link_ops && !dev->rtnl_link_ops->netns_refund)
		continue;                        /* (b) veth/macvlan/bridge: destroyed, not returned */
	snprintf(fb_name, IFNAMSIZ, "dev%d", dev->ifindex);   /* (c) fallback name */
	if (netdev_name_in_use(&init_net, fb_name))
		snprintf(fb_name, IFNAMSIZ, "dev%%d");
	err = dev_change_net_namespace(dev, &init_net, fb_name);
}
  • (a) The loopback device sets dev->netns_local = true in gen_lo_setup() (drivers/net/loopback.c). This flag means “this device may never change namespace” — it is also why ip link set lo netns foo fails with EINVAL rather than doing something surprising.
  • (b) Devices created through an rtnl_link_ops link type — veth, macvlan, ipvlan, bridge, VLAN, netkit — are not rescued; the loop skips them and the batch cleanup then calls their dellink. This matches the man page’s “when a namespace is freed, the veth(4) devices that it contains are destroyed.” The exception is the netns_refund flag, documented in include/net/rtnetlink.h as “Physical device, move to init_net on netns exit”, for drivers that expose a link type but sit on real hardware.
  • (c) A device that is pushed home gets renamed if its name is already taken in init_net: first to dev<ifindex>, then to the dev%d pattern. So a physical NIC that went into a container as eth0 can come back to the host as dev7. This is one of the more baffling symptoms in the wild and it is a two-line consequence of name-collision handling.

The corresponding footgun is documented in ip-netns(8): delete a namespace that still has a running process and a moved-in physical device, and “eth0 will appear in the default netns only after SOME_PROCESS_IN_BACKGROUND will exit or will be killed.” The manual’s own prescription is to kill first: ip netns pids net0 | xargs kill, then ip netns del net0.

Entering a Namespace, and the /var/run/netns Bind-Mount Trick

setns(2) and the double capability check

setns(fd, CLONE_NEWNET) reassociates the calling thread with the namespace referred to by fd. The networking-specific part is netns_install() (net/core/net_namespace.c:1454):

static int netns_install(struct nsset *nsset, struct ns_common *ns)
{
	struct nsproxy *nsproxy = nsset->nsproxy;
	struct net *net = to_net_ns(ns);
 
	if (!ns_capable(net->user_ns, CAP_SYS_ADMIN) ||
	    !ns_capable(nsset->cred->user_ns, CAP_SYS_ADMIN))
		return -EPERM;
 
	put_net(nsproxy->net_ns);
	nsproxy->net_ns = get_net(net);
	return 0;
}

There are two capability checks, and conflating them is a classic reasoning error. The first asks: do you have CAP_SYS_ADMIN in the user namespace that owns the target network namespace (net->user_ns, set at creation)? The second asks: do you have CAP_SYS_ADMIN in your own user namespace? You need both. The consequence is that an unprivileged user who creates a user namespace and a netns inside it can freely re-enter their own netns, but cannot setns() into the host’s — because they hold no capability in init_user_ns. The owning-user-namespace relationship this depends on is developed in full in User Namespaces; the netns simply records the owner in net->user_ns at preinit_net() time and defers to it forever after.

The fd comes from one of two places: open("/proc/<pid>/ns/net"), which names the namespace of a running process, or open("/var/run/netns/<name>"), which names a named namespace. nsenter --net=... and a container runtime’s “join the pod’s network” are both this call.

The trick: how a namespace survives with no processes in it

The kernel has no concept of a namespace name. Names are purely an iproute2 userspace convention, and the mechanism behind them is genuinely non-obvious. Reading netns_add() in ip/ipnetns.c, ip netns add foo does this:

  1. mkdir /var/run/netns if needed (NETNS_RUN_DIR, defined in include/namespace.h).
  2. Take an exclusive flock() on that directory, then make it a shared mount subtree: mount("", NETNS_RUN_DIR, "none", MS_SHARED | MS_REC, NULL), bind-mounting it onto itself first if it is not already a mount point.
  3. open(netns_path, O_RDONLY|O_CREAT|O_EXCL, 0) — create an empty regular file /var/run/netns/foo, then immediately close it. This file has no content and never will; it exists only to be a mount point.
  4. unshare(CLONE_NEWNET) — the ip process itself moves into a brand-new namespace.
  5. mount("/proc/self/ns/net", "/var/run/netns/foo", "none", MS_BIND, NULL) — bind-mount the namespace’s nsfs entry over the empty file.
  6. netns_restore() — ip switches back to its original namespace and exits.

Step 5 is the whole trick. /proc/self/ns/net is a magic symlink into nsfs, a tiny internal filesystem whose inodes are namespaces; opening one takes a reference. A bind mount is a persistent reference of exactly that kind. So after ip exits in step 6 — leaving zero processes in the new namespace — the namespace’s ns.count is still non-zero because the mount holds it, and cleanup_net() is never queued. ip-netns(8) says it plainly: “Holding that file descriptor open keeps the network namespace alive.”

This is why ip netns add foo gives you a namespace you can configure over many separate commands, while unshare -n gives you one that evaporates the instant the shell exits. It is also why ip netns del foo is documented as an unmount: “If NAME is present in /var/run/netns it is umounted and the mount point is removed.”

Step 2 — making /var/run/netns a shared subtree — is a subtlety with a comment in the source explaining exactly why: “Make it possible for network namespace mounts to propagate between mount namespaces. This makes it likely that unmounting a network namespace file in one namespace will unmount the network namespace file in all namespaces allowing the network namespace to be freed sooner.” Without it, a copy of the bind mount trapped in some other mount namespace would keep the netns alive after you deleted it. The flock() around it is scar tissue from a real bug: parallel ip netns add invocations at boot recursively re-created the mount point until the system locked up (Debian #949235, referenced by URL in the source comment).

sequenceDiagram
    autonumber
    participant IP as "ip netns add foo"
    participant FS as "/var/run/netns (shared mount)"
    participant K as kernel
    participant NSFS as nsfs

    IP->>FS: flock(LOCK_EX) then make it MS_SHARED|MS_REC
    IP->>FS: open("foo", O_CREAT|O_EXCL) then close — an EMPTY file
    IP->>K: unshare(CLONE_NEWNET)
    K->>NSFS: allocate struct net; ns.count = 1<br/>ns_alloc_inum() -> nsfs inode
    K-->>IP: ip is now inside the new netns
    IP->>K: mount("/proc/self/ns/net", "/var/run/netns/foo", MS_BIND)
    K->>NSFS: the mount takes a reference on the nsfs inode
    Note over NSFS: ns.count is now held by the MOUNT,<br/>not by any process
    IP->>K: netns_restore() — setns() back to the original netns
    IP-->>IP: exit
    Note over K,NSFS: ZERO processes in netns "foo",<br/>yet it is fully alive and configurable
    participant U as "later: ip netns exec foo ..."
    U->>FS: open("/var/run/netns/foo")
    U->>K: setns(fd, CLONE_NEWNET)

How ip netns add produces a process-less but living network namespace. What it shows: the named-namespace feature is not a kernel facility at all — it is an empty file plus a bind mount of /proc/self/ns/net, holding a reference that outlives the process that created the namespace. The insight to take: namespace lifetime is reference lifetime, and a mount is a first-class reference. This is also the debugging recipe: if a namespace refuses to die, something still holds a reference — enumerate them with lsns -t net, ls -l /proc/*/ns/net, and grep nsfs /proc/*/mountinfo.

Why ip netns exec also touches mounts

ip netns exec is more than setns() + exec(). Reading netns_switch() in lib/namespace.c, after the setns() it does:

unshare(CLONE_NEWNS);                                   /* private mount namespace */
mount("", "/", "none", MS_SLAVE | MS_REC, NULL);        /* don't leak back to the parent */
umount2("/sys", MNT_DETACH);                            /* drop the old sysfs instance */
mount(name, "/sys", "sysfs", mountflags, NULL);         /* mount a FRESH sysfs */
bind_etc(name);                                         /* /etc/netns/NAME/* over /etc/* */

The /sys remount is the part people miss. sysfs records the network namespace it was mounted against, so if you merely setns() into another netns, /sys/class/net keeps showing the old namespace’s interfaces even though ip link (which uses netlink) shows the new one’s. Tools that read /sys — ethtool in some paths, ifconfig on some systems, monitoring agents, language runtimes enumerating interfaces — would silently report the wrong namespace. So ip netns exec unshares a mount namespace first, marks / as MS_SLAVE so its mounts do not propagate back to the host, and remounts sysfs. The bind_etc() step then bind-mounts anything under /etc/netns/NAME/ over the corresponding /etc/ path — the documented convention that lets a VPN namespace have its own resolv.conf (ip-netns(8)).

Uncertain

Verify: whether ip netns exec’s /sys remount is still strictly necessary on current kernels, or whether sysfs has since learned to follow the caller’s network namespace. Reason: the requirement is inferred from the iproute2 source’s behaviour (read from the main branch during this write-up) plus the sysfs per-netns tagging design; no primary kernel document consulted here states the rule explicitly. To resolve: read fs/sysfs/ and net/core/net-sysfs.c namespace-tagging code, or find the commit that introduced the remount in iproute2. uncertain

Connecting Namespaces: the Topology Every Container Runtime Builds

A namespace born empty is useless until something plumbs connectivity into it. There are four mechanisms in wide use — veth pairs, macvlan, ipvlan, and (newest) netkit — and they differ in where in the stack the connection is made. The overwhelmingly common one, the shape that Docker’s default network, the CNI bridge plugin, and most Kubernetes clusters build, is a veth pair per container with the host end enslaved to a software bridge.

flowchart TB
  subgraph HOST["init_net — the host network namespace"]
    direction TB
    ETH0["eth0<br/>physical NIC<br/>192.0.2.10/24"]
    NFT["netfilter: MASQUERADE<br/>-s 172.17.0.0/16 -o eth0"]
    BR["docker0 / cni0<br/><b>Linux bridge</b><br/>172.17.0.1/16<br/><i>acts as the containers' gateway</i>"]
    VHA["vethA1B2C3<br/>(host end, no IP)<br/>enslaved: master docker0"]
    VHB["vethD4E5F6<br/>(host end, no IP)<br/>enslaved: master docker0"]
    ROUTE["host FIB:<br/>172.17.0.0/16 dev docker0"]
    BR --- VHA
    BR --- VHB
    BR -.-> ROUTE
    ROUTE -.-> NFT
    NFT --> ETH0
  end

  subgraph NSA["struct net #1 — container A"]
    CEA["eth0<br/>172.17.0.2/16<br/><i>the veth PEER</i>"]
    LOA["lo (must be brought up!)"]
    RTA["FIB:<br/>default via 172.17.0.1"]
    CEA --- RTA
  end

  subgraph NSB["struct net #2 — container B"]
    CEB["eth0<br/>172.17.0.3/16"]
    LOB["lo"]
    RTB["FIB:<br/>default via 172.17.0.1"]
    CEB --- RTB
  end

  VHA <== "veth pair — one cable,<br/>two ends in two namespaces" ==> CEA
  VHB <== "veth pair" ==> CEB

  ETH0 --> WAN(("the network"))

The canonical container network topology: one veth pair per container, host ends enslaved to a bridge, bridge IP as the containers’ default gateway, source NAT on the way out. What it shows: the veth pair is the only thing that crosses the namespace boundary; everything else — bridging, routing, NAT — happens entirely inside init_net on the host end. The container’s eth0 is literally one end of a two-ended virtual cable whose other end sits on a switch. The insight to take: three separate facts explain almost every container-networking bug. (1) The host-side veth has no IP address — it is a bridge port, an L2 device; do not go looking for one. (2) The container’s default gateway is the bridge’s address, so 172.17.0.1 answers from the host’s struct net, not from any container. (3) Container-to-container traffic on the same bridge never leaves L2 and never touches the host’s routing table or NAT rules, which is why iptables rules on the FORWARD chain behave differently for same-node and cross-node traffic.

What a packet actually does when it crosses

The veth driver’s transmit function is where a namespace boundary is crossed, and it is short enough to read whole. veth_xmit() (drivers/net/veth.c) looks up the peer and hands the buffer over:

rcv = rcu_dereference(priv->peer);          /* the other end of the pair */
...
if (likely(veth_forward_skb(rcv, skb, rq, use_napi) == NET_RX_SUCCESS))

and veth_forward_skb() is three lines:

static int veth_forward_skb(struct net_device *dev, struct sk_buff *skb,
			    struct veth_rq *rq, bool xdp)
{
	return __dev_forward_skb(dev, skb) ?: xdp ? veth_xdp_rx(rq, skb) : __netif_rx(skb);
}

__dev_forward_skb() bottoms out in ____dev_forward_skb() (include/linux/netdevice.h), and this is the line that defines what “crossing a namespace” means:

skb_scrub_packet(skb, !net_eq(dev_net(dev), dev_net(skb->dev)));
skb->priority = 0;

The second argument, conventionally called xnet, is literally “are the two devices in different network namespaces?” When it is true, skb_scrub_packet() (net/core/skbuff.c) does more than when it is false:

void skb_scrub_packet(struct sk_buff *skb, bool xnet)
{
	skb->pkt_type = PACKET_HOST;
	skb->skb_iif = 0;
	skb->ignore_df = 0;
	skb_dst_drop(skb);        /* forget the cached route */
	skb_ext_reset(skb);
	nf_reset_ct(skb);         /* forget the conntrack association */
	nf_reset_trace(skb);
	...
	if (!xnet)
		return;
	ipvs_reset(skb);
	skb->mark = 0;            /* <-- cross-namespace only */
	skb_clear_tstamp(skb);    /* <-- cross-namespace only */
}

skb->mark is cleared when a packet crosses into another network namespace. This is not documented anywhere a user is likely to look, and it breaks a design people reach for constantly: setting an fwmark inside a container (with iptables -j MARK or SO_MARK) and expecting a host-side ip rule fwmark ... lookup ... to see it. It will not. Likewise the conntrack association is reset at the boundary, which is why a connection is tracked twice — once in the container’s namespace and once in the host’s — and why conntrack -L on the host does not show the container’s view.

sequenceDiagram
    autonumber
    participant App as "app in container A"
    participant CT as "container netns: TCP/IP"
    participant CE as "container eth0<br/>(veth end)"
    participant VD as veth driver
    participant HE as "host vethXXXX<br/>(peer end)"
    participant BL as "per-CPU backlog queue"
    participant SI as NET_RX_SOFTIRQ
    participant BR as "host netns: bridge + FIB"

    App->>CT: write(fd, buf, n)
    CT->>CT: tcp_sendmsg -&gt; ip_queue_xmit<br/>FIB lookup in struct net #1
    CT->>CE: dev_queue_xmit()
    CE->>VD: veth_xmit()
    VD->>VD: rcu_dereference(priv-&gt;peer) — find the far end
    VD->>VD: __dev_forward_skb(peer, skb)
    Note over VD: skb_scrub_packet(skb, xnet = true)<br/>drops dst, resets conntrack,<br/><b>clears skb-&gt;mark</b>, clears timestamp
    VD->>HE: skb-&gt;dev = host veth; eth_type_trans()
    VD->>BL: __netif_rx() -&gt; enqueue_to_backlog(smp_processor_id())
    Note over BL: still the SENDING CPU,<br/>still in the sender's softirq context
    BL->>SI: raise NET_RX_SOFTIRQ
    SI->>BR: process_backlog -&gt; __netif_receive_skb()<br/>now executing in <b>init_net</b>
    BR->>BR: bridge rx handler, then FIB/netfilter in struct net (host)

One packet leaving a container, traced from write() to the host’s bridge. What it shows: the namespace transition is a single function call — veth_xmit() swaps skb->dev to the peer device and re-injects the buffer at the receive side of the stack — with a metadata scrub in between. The insight to take: there is no copy and no context switch, but there is a full second trip through the receive path: the packet is enqueued on a per-CPU backlog queue and re-processed under NET_RX_SOFTIRQ, which is exactly the cost that veth-elimination projects target. Note also that the work happens on the sending CPU by default (enqueue_to_backlog(skb, smp_processor_id(), ...), net/core/dev.c), so a busy container’s egress consumes softirq time on the core its own thread is running on.

Two performance footnotes that follow directly from the source:

  • Generic Receive Offload (GRO) is off by default on veth. veth_newlink() calls veth_disable_gro() on both ends, with the comment “keep GRO disabled by default to be consistent with the established veth behavior.” The NAPI machinery in the driver is only instantiated when an eXpress Data Path (XDP) program is attached or GRO is explicitly enabled. Turning GRO on with ethtool -K vethXXX gro on measurably helps bulk TCP throughput into containers, at the cost of latency — it is one of the few genuinely free tuning knobs here.
  • The peer lookup is an RCU dereference, not a lookup by name or index. priv->peer is a direct struct net_device *. ethtool -S vethA exposes the far end as the peer_ifindex statistic, which is the documented way to find which interface is the other half of a pair (veth(4)).

The Other Three Connection Mechanisms, and When Each Is Right

macvlan — one NIC, many MAC addresses

A macvlan device is a child of a real (“lower”) device that gets its own MAC address. There is no bridge and no veth pair: frames from the child are handed straight to the lower device, and inbound frames are demultiplexed to a child by destination MAC. Modes are declared in the user API (include/uapi/linux/if_link.h): private (“don’t talk to other macvlans”), vepa (“talk to other ports through ext bridge”), bridge (“talk to bridge ports directly”), passthru (“take over the underlying device”) and source (MAC allow-list).

The famous macvlan gotcha — a macvlan child cannot talk to its own host — is a direct consequence of macvlan_queue_xmit() (drivers/net/macvlan.c). In bridge mode it looks the destination MAC up in macvlan_hash_lookup(port, eth->h_dest), a hash containing only the macvlan children. The lower device’s own MAC is not in that hash, so a frame addressed to the host falls through to xmit_world, where skb->dev = vlan->lowerdev sends it out the physical wire. The switch upstream has no reason to reflect it back to the port it arrived on, so the host never sees it. The standard workaround is to give the host its own macvlan child on the same lower device and route through that.

ipvlan — one NIC, one MAC, many IPs

ipvlan solves the case where macvlan cannot be used at all: a switch port configured for one MAC, a NIC whose MAC table would overflow, or a hostile namespace you do not want changing L2 state. Its documentation states the design in one line: “This is conceptually very similar to the macvlan driver with one major exception of using L3 for mux-ing/demux-ing among slaves. This property makes the master device share the L2 with its slave devices” (Documentation/networking/ipvlan.rst). All children share the parent’s MAC; demultiplexing is by IP address.

The three modes matter:

  • l2 — TX is processed on the child’s own stack instance and switched to the master to send. Children can send and receive multicast/broadcast.
  • l3 (the default) — TX is processed up to L3 on the child’s stack, then handed to the master’s stack instance for L2 processing and routing. Children “will not receive nor can send multicast/broadcast traffic,” which breaks anything relying on Address Resolution Protocol (ARP), Dynamic Host Configuration Protocol (DHCP) or IPv6 Neighbour Discovery inside the namespace.
  • l3s — like l3 “except that iptables (conn-tracking) works in this mode and hence it is L3-symmetric … This will have slightly less performance but that shouldn’t matter since you are choosing this mode over plain-L3 mode to make conn-tracking work.”

The documentation’s own decision rule is worth quoting because it is the clearest statement of when to pick ipvlan over macvlan: use it when “(a) the Linux host that is connected to the external switch/router has policy configured that allows only one mac per port, (b) [the] number of virtual devices created on a master exceed the mac capacity and puts the NIC in promiscuous mode and degraded performance is a concern, (c) if the slave device is to be put into the hostile/untrusted network namespace where L2 on the slave could be changed/misused.”

netkit — a BPF-programmable veth replacement

The newest option, netkit, was added in Linux 6.7 (verified by existence-checking drivers/net/netkit.c across tags: HTTP 404 at v6.5 and v6.6, HTTP 200 at v6.7 and v6.8). Its MODULE_DESCRIPTION is “BPF-programmable network device” and its authors are Daniel Borkmann and Nikolay Aleksandrov (drivers/net/netkit.c). Like veth it is a pair, but instead of unconditionally re-injecting into the peer’s receive path it runs a chain of attached BPF programs first and acts on their verdict:

netkit_prep_forward(skb, !net_eq(dev_net(dev), dev_net(peer)));  /* same xnet scrub as veth */
...
entry = rcu_dereference(nk->active);
if (entry)
	ret = netkit_run(entry, skb, ret);
switch (ret) {
case NETKIT_PASS:     ... __netif_rx(skb); break;   /* like veth */
case NETKIT_REDIRECT: ... skb_do_redirect(skb); break;  /* skip the peer's stack entirely */
case NETKIT_DROP:     ... kfree_skb(skb); break;
}

NETKIT_REDIRECT is the point: a BPF program in the sending namespace can push the packet directly at its final device without the receiving namespace’s stack ever running. Cilium, whose engineers wrote the driver, describes the result as removing the boundary cost entirely: netkit devices “provide connectivity for Pods with the goal to improve throughput and latency for applications as if they would have resided directly in the host namespace, meaning, it reduces the datapath overhead for network namespaces down to zero. The netkit driver in the kernel has been specifically designed for Cilium’s needs and replaces the old-style veth device type” (Cilium tuning guide, read 2026-09-04, documenting Cilium 1.20.1). Cilium requires kernel ≥ 6.8 for it, one release later than the merge, and flags it as a beta feature; it also warns that you cannot swap veth for netkit in place on a running cluster, because “the CNI plugin cannot simply replace veth with netkit after Pod creation.”

flowchart TB
  subgraph V["veth pair + bridge"]
    direction TB
    V1["container eth0"] -->|"veth_xmit"| V2["host veth"]
    V2 --> V3["bridge -&gt; host FIB -&gt; NIC"]
  end
  subgraph M["macvlan"]
    direction TB
    M1["container macvlan0<br/><b>own MAC</b>"] -->|"macvlan_queue_xmit"| M2["lower dev (eth0)"]
    M2 --> M3["the wire — host stack BYPASSED"]
  end
  subgraph I["ipvlan (l3)"]
    direction TB
    I1["container ipvl0<br/><b>parent's MAC</b>, own IP"] --> I2["L3 handoff to the<br/>MASTER's stack instance"]
    I2 --> I3["host routing -&gt; NIC"]
  end
  subgraph N["netkit"]
    direction TB
    N1["container netkit peer"] -->|"netkit_xmit + BPF verdict"| N2{"PASS / REDIRECT / DROP"}
    N2 -->|REDIRECT| N3["straight to the NIC —<br/>peer stack skipped"]
    N2 -->|PASS| N4["host veth-like receive"]
  end

Where each mechanism makes the namespace crossing. What it shows: the four options differ in how much of the host’s network stack a container’s packet must traverse — everything (veth+bridge), almost nothing (macvlan), the host’s L3 only (ipvlan l3), or whatever a BPF program decides (netkit). The insight to take: the choice is a trade between host-stack features and host-stack cost. veth+bridge is the only shape where the full host toolbox — netfilter, tc, conntrack, NAT — applies to container traffic, which is why every general-purpose runtime defaults to it. macvlan is fastest and gives the container a first-class L2 identity, but it bypasses the host stack, so host-side firewalling of container traffic does not work and the container cannot reach its own host.

veth + bridgemacvlanipvlan (l3/l3s)netkit
Kernel object per container2 netdevs + 1 bridge port1 netdev1 netdev2 netdevs
MAC addresses on the wirebridge/NAT hides themone per containerone, shared with parentas configured
Container ↔ its own hostworks (bridge is the gateway)does not workworks in l3 via host routingworks
Host netfilter sees container trafficyesnol3s onlyas programmed
Broadcast/multicast in the containeryesyesno in l3/l3smode-dependent
Needs a switch tolerant of many MACsnoyesnono
Minimum kernelancientancient4.2-era6.7 (Cilium wants 6.8)
Typical userDocker default bridge, CNI bridge/flannel/CalicoNIC-passthrough-style CNI, telecom workloadssingle-MAC switch ports, dense hostsCilium ≥ 1.16 with bpf.datapathMode=netkit

Depth on the first two rows lives in the sibling notes: veth Pairs for the pair semantics and Linux Bridges and Software Switching for the bridge’s forwarding database and STP behaviour. Container Networking with veth and Bridges walks the combined setup end to end.

Worked Example — Two Namespaces, Run on a Real Machine

Everything below was executed while writing this note, on Fedora 44 running kernel 7.1.8 (the note’s source pin is the 6.12 LTS tree; the runtime observations come from 7.1.8, and where the two could differ it is said so). All of it runs unprivileged, because unshare -r creates a user namespace first and CAP_SYS_ADMIN inside that namespace is enough to unshare a netns.

Step 1 — what a brand-new namespace actually contains

$ unshare -rn ip link show
1: lo: <LOOPBACK> mtu 65536 qdisc noop state DOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

One interface. Not “a copy of the host’s interfaces with restrictions” — one interface, and it is DOWN. The consequence is immediate and catches people constantly:

$ unshare -rn ping -c1 -W1 127.0.0.1
ping: connect: Network is unreachable
 
$ unshare -rn sh -c 'ip link set lo up; ping -c1 -W1 127.0.0.1'
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.023/0.023/0.023/0.000 ms

Network is unreachable, not “Destination host unreachable” — the kernel has no route to 127.0.0.1 because the route that normally exists (local 127.0.0.0/8 dev lo) is installed when lo comes up, and in a fresh namespace it never has. A minimal container image whose init does not ip link set lo up is a container where localhost does not work, and the error message points at the network rather than at the loopback device.

The rest of the stack is equally empty:

$ unshare -rn sh -c 'ip route show; echo "[routes above]"; nft list ruleset; echo "[nft ruleset above]"; cat /proc/net/dev'
[routes above]
[nft ruleset above]
Inter-|   Receive                                                |  Transmit
 face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop fifo colls carrier compressed
    lo:     168       2    0    0    0     0          0         0      168       2    0    0    0     0       0          0

Zero routes, zero firewall rules, one interface in /proc/net/dev — even though the host this ran on has five bridges, six veths, a Tailscale tunnel and a full nftables ruleset. That is the isolation, visible in three lines of output.

Step 2 — two namespaces joined by a veth pair

# Run the whole thing inside one unprivileged user+net+mount namespace so that
# `ip netns` has somewhere writable to put its bind mounts.
unshare -rnm bash <<'EOF'
mount -t tmpfs none /run/netns          # (1) private /run/netns, no root needed
ip link set lo up
ip netns add peer                       # (2) creates the empty file + bind mount
 
ip link add v0 type veth peer name v1 netns peer   # (3) one cable, two namespaces
ip addr add 10.9.9.1/24 dev v0 && ip link set v0 up
ip -n peer addr add 10.9.9.2/24 dev v1
ip -n peer link set v1 up && ip -n peer link set lo up
 
ping -c1 -W1 10.9.9.2                   # (4) the crossing works
ethtool -k v0 | grep '^generic-receive-offload'
EOF
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.070/0.070/0.070/0.000 ms
generic-receive-offload: off

Line by line: (1) ip netns needs /run/netns to be writable and mountable, which inside a user namespace means giving it a tmpfs of our own — this is exactly what rootless container tooling does. (2) ip netns add peer runs the empty-file-plus-bind-mount dance from the previous section; from here on the namespace exists with no process in it. (3) ip link add v0 type veth peer name v1 netns peer creates both ends in one netlink message and drops the far end straight into peer — no separate move step, and therefore no window in which the peer is visible in the wrong namespace. (4) 70 µs round-trip across the boundary.

The last line is the measured confirmation of a source-level fact from the previous section: Generic Receive Offload is off by default on veth (veth_disable_gro() in veth_newlink()), so bulk TCP into a container is not benefiting from receive coalescing unless something turned it on.

Step 3 — the port-space proof

unshare -rn sh -c 'ip link set lo up; python3 -m http.server 80 --bind 127.0.0.1' &
# ... and again, and again. Each one binds :80 in its own struct net with no conflict,
# while the host's own :80 (if any) is untouched.

Binding port 80 unprivileged inside the namespace works for a second reason worth knowing: net.ipv4.ip_unprivileged_port_start is itself per-namespace, documented as “a per-namespace sysctl. It defines the first unprivileged port in the network namespace” (ip-sysctl.rst), so a container can be given the low ports without CAP_NET_BIND_SERVICE and without touching the host’s policy.

What Crossing a Namespace Actually Costs

Folk wisdom says veth is expensive. The measurements below — taken on the machine described above, a ping-pong TCP benchmark with TCP_NODELAY on both ends, client and server pinned to different cores, 40,000 iterations for the small sizes and 6,000 for 64 KiB — say something more precise and more useful.

Message size(A) TCP over lo, one namespace(B) TCP over a veth pair, two namespacesB ÷ A
64 B4.45 µs round-trip4.14 µs0.93×
1 KiB3.71 µs4.12 µs1.11×
64 KiB13.71 µs12.58 µs0.92×

Read that carefully, because it is the opposite of what the folklore predicts: crossing a network-namespace boundary through a veth pair costs roughly nothing relative to a loopback connection inside a single namespace. The run-to-run spread on these numbers is of the same order as the differences, so the honest reading is “indistinguishable,” not “veth is faster.” Enabling GRO on both ends (ethtool -K v0 gro on) moved the 64 KiB figure from 12.58 µs to 12.73 µs — i.e. nowhere, for this latency-bound workload; GRO helps bulk streaming throughput, not ping-pong latency.

The mechanism explains why. From the packet trace above, veth_xmit() does not copy the buffer, does not switch address spaces, and does not context-switch — it swaps skb->dev to the peer, scrubs a handful of metadata fields, and re-injects on the receive path. Loopback does almost exactly the same thing: loopback_xmit() also hands the skb straight back into __netif_rx(). The two paths are cousins.

So where does the real cost of container networking come from? Not the namespace crossing, but everything bolted around it:

  • The bridge and its forwarding decision — one more device traversal and an FDB lookup per packet (see Linux Bridges and Software Switching).
  • netfilter and conntrack, twice. Because skb_scrub_packet() calls nf_reset_ct() at the boundary, a connection is tracked once inside the container’s struct net and again in the host’s. On a busy node that is two conntrack entries, two rule-chain traversals, and two table’s worth of memory per flow.
  • NAT. The default Docker/CNI bridge topology masquerades egress, adding a translation and a conntrack dependency to every outbound flow.
  • The second softirq trip. The re-injection is real work even if it is cheap: the packet is enqueued on a per-CPU backlog and reprocessed under NET_RX_SOFTIRQ on the sending CPU.

That is the correct target list, and it is exactly what the optimization projects attack. Cilium’s eBPF host-routing “fully bypass[es] iptables and the upper host stack, and … achieve[s] a faster network namespace switch compared to regular veth device operation” — note that the claim is about bypassing the host stack, not about making the veth hop itself cheaper (Cilium tuning guide, read 2026-09-04).

Namespace creation is likewise cheap but not free. Measuring 200 sequential unshare invocations with and without CLONE_NEWNET:

$ # 200 x (fork + unshare + exec /bin/true + exit)
with  CLONE_NEWNET: 0.96 ms each
without CLONE_NEWNET: 0.54 ms each
=> ~0.42 ms attributable to creating and destroying one network namespace

Roughly 0.4 ms per namespace, which is the cost of setup_net() walking the whole pernet_list plus the eventual cleanup_net() batch. For a container start that is noise. For a workload that churns thousands of short-lived namespaces it is not, and — per the lifetime section — the teardown half of that cost is serialized through one single-threaded workqueue and gated on two RCU grace periods and the rtnl_mutex, so it does not parallelize.

Uncertain

Verify: whether the veth-versus-loopback parity above generalizes. Reason: these are single-machine, single-flow, ping-pong measurements on one CPU family and one kernel (7.1.8), with GRO/GSO defaults and no bridge, netfilter, or NAT in the path — deliberately isolating only the namespace crossing. Multi-flow, multi-core, bridged, NAT’ed and MTU-constrained paths behave differently, and veth’s lack of GRO does penalize bulk throughput. To resolve: reproduce with iperf3 or netperf across a bridged topology with the host’s real ruleset loaded before generalizing any of it. uncertain

What Container Runtimes Actually Build

Docker’s default bridge, observed live

The host this note was written on runs Docker with several user-defined bridge networks. The topology described earlier is not a diagram of an idealized system — it is literally what ip reports:

$ ip -br addr show br-df1af58002b7
br-df1af58002b7  UP   172.21.0.1/16 fe80::b446:9dff:fe06:9d1f/64
 
$ bridge link show
196: veth424793b@enp191s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br-df1af58002b7 state forwarding
198: veth5a7bc86@enp191s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br-df1af58002b7 state forwarding
1395: veth101fe33@enp191s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br-df1af58002b7 state forwarding
 
$ ip -d -o link show veth424793b
196: veth424793b@if2: ... master br-df1af58002b7 state UP ... link-netnsid 0 ... veth bridge_slave state forwarding ...

Three things in that output repay attention.

  1. The bridge holds the gateway address (172.21.0.1/16), and the veths hold none. ip -br addr show veth424793b returns no address at all. The host-side veth is a bare L2 bridge port; every container on that network uses the bridge’s address as its default gateway. If you go looking for an IP on the host end of a container’s veth, you will not find one, and that is correct.
  2. veth424793b@if2 — the @if2 suffix is the peer’s interface index, and index 2 is inside the container’s namespace, not the host’s. An interface index only means something paired with a namespace, which is why ip prints link-netnsid 0 alongside: a per-namespace-relative identifier assigned by the kernel so userspace can correlate the two ends. ethtool -S veth424793b | grep peer_ifindex reports the same number.
  3. bridge link show renders the peer as veth424793b@enp191s0 while ip link renders it as veth424793b@if2 — the same device, two tools guessing differently at what the peer index refers to in the local namespace. When the printed peer name looks nonsensical, this is why; trust ethtool -S’s peer_ifindex plus ip netns identify.

Docker keeps its container namespaces alive using the same bind-mount trick as ip netns, just in a different directory — /run/docker/netns/ rather than /var/run/netns/. lsns -t net on this host shows it directly:

$ lsns -t net
        NS TYPE NPROCS     PID USER   NETNSID NSFS                           COMMAND
4026531833 net     331    3289 linman unassigned /run/docker/netns/default   ...
4026533610 net       2 1874178 linman          2 /run/docker/netns/6132c34f7f7e  /sbin/docker-init ...

This is also why ip netns list is empty on a machine full of containers: ip netns only enumerates /var/run/netns/. lsns -t net is the tool that sees all of them, and the NSFS column tells you which bind mount is holding each one alive.

The CNI contract

Kubernetes does not build any of this itself; it delegates to a CNI plugin, and the contract is explicit that a network namespace is the unit of work. The specification defines the container’s identity as its “isolation domain”: CNI_NETNS is “a reference to the container’s ‘isolation domain’. If using network namespaces, then a path to the network namespace (e.g. /run/netns/[nsname])” (CNI SPEC.md). The runtime creates an empty netns, pins it at a path, and hands that path to the plugin; the plugin’s ADD operation must “create the interface defined by CNI_IFNAME inside the container at CNI_NETNS.”

The reference bridge plugin implements it in an order that is worth knowing because it is the reverse of what most people would write. It does not create the veth pair on the host and then move one end in. It enters the container’s namespace and creates the pair from there, with the peer placed directly into the host namespace by file descriptor (pkg/ip/link_linux.go):

// makeVethPair is called from within the container's network namespace
veth := &netlink.Veth{
	LinkAttrs:     linkAttrs,          // Name = "eth0" (CNI_IFNAME), MTU
	PeerName:      peerName,           // random "vethXXXXXXXX"
	PeerNamespace: netlink.NsFd(int(hostNS.Fd())),   // <- peer lands on the HOST
}

and then, back on the host side (plugins/main/bridge/bridge.go):

// need to lookup hostVeth again as its index has changed during ns move
hostVeth, err := netlinksafe.LinkByName(hostIface.Name)
...
// connect host veth end to the bridge
if err := netlink.LinkSetMaster(hostVeth, br); err != nil { ... }

Two lessons are embedded there. First, doing it from the inside means the container-side interface is named eth0 at creation, never briefly visible under another name in the host namespace — a small but real race and information-leak avoided. Second, the comment its index has changed during ns move restates the rule from earlier in this note in operational form: an interface index is namespace-relative, so any handle you captured before a move is stale afterwards and must be re-resolved by name.

sequenceDiagram
    autonumber
    participant K as kubelet
    participant CRI as "containerd / CRI-O"
    participant PAUSE as "pause container process"
    participant PL as "CNI plugin (bridge)"
    participant HOST as "host netns (init_net)"
    participant POD as "Pod netns"

    K->>CRI: RunPodSandbox
    CRI->>PAUSE: clone(CLONE_NEWNET, ...) — create the sandbox
    PAUSE-->>CRI: pid
    CRI->>CRI: bind-mount /proc/&lt;pid&gt;/ns/net -&gt; /run/netns/&lt;id&gt;<br/>(the namespace now survives independently)
    CRI->>PL: exec plugin, CNI_COMMAND=ADD<br/>CNI_NETNS=/run/netns/&lt;id&gt;, CNI_IFNAME=eth0<br/>config on stdin
    PL->>POD: setns() into CNI_NETNS
    PL->>POD: LinkAdd(veth{Name:"eth0", PeerNamespace: NsFd(hostNS)})
    Note over POD,HOST: BOTH ends created from inside;<br/>peer lands directly in init_net
    PL->>HOST: back on the host: LinkByName(hostVeth)<br/><b>index changed by the move — re-resolve</b>
    PL->>HOST: LinkSetMaster(hostVeth, br) — enslave to the bridge
    PL->>POD: IPAM: assign address, add default route via the bridge IP
    PL-->>CRI: JSON result: interfaces[], ips[], routes[], sandbox=/run/netns/&lt;id&gt;
    CRI->>K: sandbox ready
    Note over PAUSE,POD: every app container in the Pod is then started<br/>with setns() into the SAME netns — one struct net,<br/>hence one IP and localhost between containers

How a Kubernetes Pod’s network namespace is created, pinned, and plumbed. What it shows: the runtime owns the namespace and the CNI plugin owns its contents, with the pinned path (CNI_NETNS) as the only handle passed between them. The insight to take: this is why “the Pod is the network identity, not the container.” Every container in a Pod is setns()-ed into one struct net, so they share one IP address, one port space, one routing table and one lo — which is exactly why two containers in a Pod cannot both bind port 8080, and exactly why they can reach each other on 127.0.0.1. The pause container exists to be the process that holds that namespace’s reference open while app containers restart around it.

Failure Modes and Common Misunderstandings

  • “localhost works in a new namespace.” It does not, and the error you get (Network is unreachable) points at routing rather than at the loopback device, which sends people down the wrong path. Fix: ip link set lo up. Verified above on a live kernel.
  • “ip netns list shows all the namespaces.” It shows only named ones — the bind mounts under /var/run/netns/. Container runtimes pin theirs elsewhere (/run/docker/netns/) or not at all. Use lsns -t net, whose NSFS column names the holding mount, or compare readlink /proc/<pid>/ns/net inode numbers.
  • “Moving an interface keeps its configuration.” It does not. Addresses, routes and neighbour entries lived in the source struct net’s tables; the device arrives at the destination bare. Re-address on the far side.
  • “A device pushed back to init_net comes back with its name.” Only if the name is free. default_device_exit_net() renames a colliding device to dev<ifindex>, then to a dev%d pattern — so a NIC that went in as eth0 can come home as dev7.
  • fwmark set inside a container is invisible on the host. skb_scrub_packet(skb, xnet=true) sets skb->mark = 0 on every cross-namespace hop. Any design that marks in the container and matches with ip rule fwmark on the host is silently broken. Mark on the host-side veth instead (tc ingress, or an iptables rule matching the host veth’s name).
  • Conntrack is per-namespace, so a flow is tracked twice. conntrack -L on the host does not show what the container sees, and nf_conntrack_max in a container does not consume the host’s budget. Both facts follow from netns_ct ct being a field of struct net.
  • The namespace outlives the container, and that is usually not a leak. An open fd, a /proc/net/* seq_file, a user socket, or a bind mount is each enough to hold it. A named namespace with no processes persists forever until ip netns del — genuinely leakable by tooling that forgets to clean up. The ucounts cap (/proc/sys/user/max_net_namespaces, 511917 on this host) bounds the damage with ENOSPC rather than memory exhaustion.
  • Deleting a namespace with a live process and a moved-in physical NIC strands the NIC. ip-netns(8) documents that eth0 “will appear in the default netns only after SOME_PROCESS_IN_BACKGROUND will exit or will be killed.” Kill first: ip netns pids foo | xargs kill, then ip netns del foo.
  • setns() does not migrate your existing sockets. They keep the namespace captured at socket() time. This is a feature (it is what makes the privilege-separation pattern work) but it surprises people who expect setns() to be a mode switch for the whole process.
  • /sys lies after a bare setns(). sysfs is tagged with the namespace it was mounted against, so /sys/class/net keeps showing the old namespace while ip link shows the new one. ip netns exec avoids this by unsharing a mount namespace and remounting sysfs; anything doing its own setns() must do the same or read only netlink.
  • Teardown latency under churn. One single-threaded workqueue, batched, with synchronize_rcu_expedited() and rcu_barrier() inside and rtnl_lock() held across the device-unregistration phase. High pod churn therefore also stalls unrelated ip link operations on the same host.

Alternatives and Boundaries

A network namespace isolates the stack; it does not limit bandwidth, CPU or packet rate. Pair it with tc qdiscs on the host-side veth for shaping and with cgroup accounting for attribution — a container that saturates a NIC does so from inside a perfectly isolated namespace.

It is also not a security boundary on its own. A process holding CAP_SYS_ADMIN in the owning user namespace can setns() back out; the kernel is shared, so a networking bug is a cross-namespace bug; and — the case that has bitten real deployments — sharing a namespace deliberately re-couples things you assumed were separate. CVE-2020-15257 is the canonical example: containerd’s shim exposed its control API on an abstract Unix-domain socket, and because [[The Abstract Socket Namespace|the abstract socket namespace is a field of struct net]], any container sharing the host’s network namespace (docker run --net=host, hostNetwork: true) could reach it. The advisory’s own words: “Access controls for the shim’s API socket verified that the connecting process had an effective UID of 0, but did not otherwise restrict access to the abstract Unix domain socket. This would allow malicious containers running in the same network namespace as the shim … to cause new processes to be run with elevated privileges” (GHSA-36xw-fx78-c5r4 / CVE-2020-15257). The lesson is not “abstract sockets are bad” but “the network namespace is a capability boundary for more than IP, and --net=host gives that away.”

For genuinely stronger isolation the answer is a second kernel: a microVM (Firecracker, Kata Containers — see microVMs vs Containers vs Full VMs) or a full virtual machine, paying for it with a virtual NIC and a second TCP/IP stack in the path. Within the single-kernel model, the options are the four connection mechanisms compared above, and the honest summary is: use veth + bridge unless you have a specific reason not to, because it is the only shape where the host’s entire toolbox — netfilter, tc, conntrack, NAT, tcpdump — applies to container traffic.

Two adjacent primitives are worth naming so they are not confused with netns. VRF (Virtual Routing and Forwarding, Documentation/networking/vrf.rst) gives you multiple routing tables inside one namespace, bound to interfaces — lighter than a namespace and appropriate when you want route separation without process separation. AF_XDP / XDP operates below the namespace boundary entirely, at the driver, and its per-namespace state (netns_xdp) exists precisely so that an XDP socket cannot reach across.

See Also

Graph View

Backlinks