vault

struct socket and struct sock

16 min read

struct socket and struct sock

The Linux kernel represents a single open network endpoint with two paired C structures, not one. struct socket (defined in include/linux/net.h) is the thin BSD-layer object — the thing a file descriptor points at, holding little more than the socket’s type, state, the protocol operation table ops, a back-pointer to the struct file, and the wait queue. The real work lives in struct sock (in include/net/sock.h), reachable as the sk field — the network-layer protocol control block that holds the receive and transmit queues of sk_buffs, the buffer-size limits, the connection state, the memory accounting, and the callbacks (sk_data_ready, sk_write_space) that wake sleeping readers and writers. The split exists so the generic VFS-and-syscall machinery talks to struct socket without knowing anything about TCP, while every protocol — TCP, UDP, raw IP, AF_UNIX, AF_PACKET — embeds struct sock as the first member of its own larger structure (inet_sock, tcp_sock, …) and casts up to it. Source references below are pinned to Linux 6.12 LTS (released 2024-11-17), the current long-term branch.

This note is the structural anatomy of a kernel socket. The userspace-facing side — socket(), bind(), the proto_ops dispatch — is owned by The Socket Layer; how the queues are sized and charged against limits is Socket Buffers and Memory Accounting; how sk_data_ready actually wakes an epoll waiter is Socket Wait Queues and Wakeups. This note explains the shapes of the structures themselves and how they nest.

Mental Model — Two Layers, One Endpoint

Think of an open socket as a stack of three nested boxes. The outermost box is the file descriptor — an int in userspace that indexes into the process’s file table and lands on a struct file whose private_data (via the socket inode) reaches a struct socket. The middle box, struct socket, is protocol-agnostic: it knows the socket is SOCK_STREAM, that it is SS_CONNECTED, and which proto_ops table to dispatch syscalls through — but it does not know what TCP is. The innermost box, struct sock (always written sk in the code), is protocol-specific state: the actual receive queue, the sequence numbers, the congestion window. Crucially the innermost box is itself the first member of an even larger protocol box: struct sock is the first field of struct inet_sock, which is the first field of struct inet_connection_sock, which is the first field of struct tcp_sock. Because C guarantees the first member shares the structure’s starting address, a pointer to any of these can be safely cast to a struct sock *, and tcp_sk(sk) simply casts back down.

flowchart TD
  FD["fd (int in userspace)"] --> FILE["struct file<br/>f_op = socket_file_ops"]
  FILE -->|"SOCK_INODE / container_of"| SOCKET["struct socket<br/>state, type, flags<br/>ops (proto_ops)<br/>file, wq (wait queue)<br/>sk ───────────────┐"]
  SOCKET -->|"sock->sk"| SOCK["struct sock (sk)<br/>sk_receive_queue / sk_write_queue<br/>sk_state, sk_rcvbuf / sk_sndbuf<br/>sk_data_ready() / sk_write_space()<br/>sk_socket ──── back-pointer"]
  SOCK -.->|"sk->sk_socket"| SOCKET
  SOCK -->|"embedded as 1st member"| INET["struct inet_sock<br/>(struct sock sk; first)<br/>inet_saddr, inet_sport, tos…"]
  INET -->|"1st member"| ICSK["struct inet_connection_sock<br/>(struct inet_sock; first)<br/>icsk_ca_ops, icsk_retransmit_timer…"]
  ICSK -->|"1st member"| TCP["struct tcp_sock<br/>(inet_connection_sock; first)<br/>snd_cwnd, rcv_nxt, snd_una…"]

The nesting of socket structures in Linux 6.12. What it shows: a file descriptor resolves to a struct socket (BSD layer), which points at a struct sock (sk, network layer) via sock->sk, and sk points back via sk_socket. The struct sock is then the first member of progressively larger protocol structures, so up-casting (tcp_sk(), inet_sk()) is a free pointer reinterpretation. The insight to take: there is no “TCP socket struct” separate from struct sock — TCP wraps the generic sock by embedding it at offset 0, and the kernel walks between layers by pointer arithmetic that costs nothing at runtime.

struct socket — the BSD Layer

The full definition in include/linux/net.h (v6.12) is remarkably small:

struct socket {
	socket_state		state;   /* SS_CONNECTED, SS_UNCONNECTED, … */
	short			type;    /* SOCK_STREAM, SOCK_DGRAM, … */
	unsigned long		flags;   /* SOCK_NOSPACE, SOCK_PASSCRED, … */
	struct file		*file;   /* back-pointer to the file (for gc) */
	struct sock		*sk;     /* the protocol-agnostic networking sock */
	const struct proto_ops	*ops;   /* protocol-specific socket operations */
	struct socket_wq	wq;      /* wait queue for poll/epoll/async */
};

Walking the fields. state is a socket_state enum — SS_FREE (not allocated), SS_UNCONNECTED, SS_CONNECTING, SS_CONNECTED, SS_DISCONNECTING (per include/uapi/linux/net.h). This is the BSD notion of state and is deliberately coarse; it is not the TCP state machine. The fine-grained TCP states (LISTEN, SYN-SENT, ESTABLISHED, TIME-WAIT, …) live in sk->sk_state inside struct sock, not here — a common point of confusion (see The TCP State Machine). type is the socket type passed to socket(): SOCK_STREAM = 1, SOCK_DGRAM = 2, SOCK_RAW = 3, SOCK_SEQPACKET = 5, etc. (enum sock_type in net.h).

flags carries bits such as SOCK_NOSPACE (the transmit buffer is full — set when a write would block, cleared when space frees up) and SOCK_PASSCRED. file is the back-pointer to the owning struct file, needed because the socket layer participates in the AF_UNIX file-descriptor garbage collector (passing an fd over a Unix socket can create reference cycles). ops is a pointer to a const struct proto_ops — the dispatch table of function pointers (bind, connect, accept, sendmsg, recvmsg, setsockopt, poll, …) that implements the socket’s behaviour for its address family; for an IPv4 stream socket this is inet_stream_ops. The comment in the source notes ops “might change with IPV6_ADDRFORM or MPTCP” — i.e. it is not strictly immutable. Finally wq is the struct socket_wq, whose first member is a wait_queue_head_t — the list of tasks sleeping in poll/select/epoll/blocking recv, woken by the callbacks on sk. The proto_ops table is the heart of The Socket Layer and Protocol Families and Address Families; this note treats it as a black box.

The asymmetry is the point: struct socket is ~7 fields, while struct sock is well over a hundred. Everything generic and VFS-facing is in socket; everything protocol-specific and hot is in sock.

struct sock — the Network-Layer Control Block

struct sock (the sk) is the workhorse. In 6.12 it opens with an embedded struct sock_common __sk_common and then dozens of fields, organised into cacheline groups — a deliberate 6.x-era layout optimisation. The structure is bracketed by __cacheline_group_begin(...) / __cacheline_group_end(...) markers (sock_write_rx, sock_read_rx, sock_read_rxtx, sock_write_rxtx, sock_write_tx, sock_read_tx) that pack fields touched on the same path (receive-write, receive-read, transmit-write, …) into the same cache lines, so that a busy receive does not dirty cache lines a concurrent transmit reads. The grouping is asserted at boot by CACHELINE_ASSERT_GROUP_MEMBER checks in sock.c.

sock_common — the shared head

The first member, struct sock_common __sk_common, is shared with the minimal socket variants (inet_timewait_sock for connections in TIME-WAIT, request_sock for half-open connections) so that the connection-lookup hash tables can hold any of them. A large block of #defines immediately after the struct opening aliases sk_common fields up into the sk_ namespace — e.g. #define sk_state __sk_common.skc_state, #define sk_family __sk_common.skc_family, #define sk_prot __sk_common.skc_prot, #define sk_refcnt __sk_common.skc_refcnt. So when code writes sk->sk_state it is really reading sk->__sk_common.skc_state. The common head holds: the address/port 4-tuple used for demultiplexing (skc_daddr, skc_rcv_saddr, skc_dport, skc_num, and the IPv6 skc_v6_daddr/skc_v6_rcv_saddr), skc_family (AF_INET/AF_INET6/…), skc_state (the protocol state — for TCP this is the TCP state machine value), skc_prot (a pointer to the struct proto — the protocol’s method table: tcp_prot, udp_prot, …), skc_net (the network namespace), the skc_refcnt, and the hash-table linkage nodes. The layout is hand-tuned so that on 64-bit builds with IPv6 enabled, offsetof(struct sock, sk_refcnt) == 128, padded by a union (skc_flags/skc_listener/skc_tw_dr) — a comment in sock_common documents this explicitly.

Receive and transmit queues

The two most important fields conceptually are the packet queues. struct sk_buff_head sk_receive_queue is the list of received-but-not-yet-read sk_buffs — when TCP/UDP finishes processing an inbound packet destined for this socket, it links the skb here and calls sk_data_ready. struct sk_buff_head sk_write_queue is the transmit side — data the application has handed to send() but that has not yet been fully transmitted/acknowledged. A struct sk_buff_head (from include/linux/skbuff.h) is just a doubly-linked list head with a length counter and a spinlock:

struct sk_buff_head {
	/* next/prev must be first to match sk_buff */
	struct sk_buff	*next;
	struct sk_buff	*prev;
	__u32		qlen;   /* number of skbs queued */
	spinlock_t	lock;   /* protects the list */
};

The next/prev are deliberately first so a struct sk_buff (whose first two fields are also next/prev) can be enqueued by the same generic list helpers — the queue head and the queue elements share a layout. For TCP, the retransmit queue is special: rather than sk_write_queue alone, the unacknowledged segments live in a red-black tree, and struct sock overlays this with a union { struct sk_buff *sk_send_head; struct rb_root tcp_rtx_queue; } — UDP uses sk_send_head, TCP uses tcp_rtx_queue. There is also struct sk_buff_head sk_error_queue (for MSG_ERRQUEUE error reporting, e.g. ICMP errors and TX timestamps) and the per-socket backlog queue (sk_backlog) — a small fast-path stash used when the socket lock is held by another context, drained when the lock is released (see Socket Buffers and Memory Accounting for why the backlog exists).

Buffer limits and memory accounting

int sk_rcvbuf and int sk_sndbuf are the receive- and send-buffer limits in bytes — the ceilings that SO_RCVBUF/SO_SNDBUF set and that bound how much queued data the socket may hold. sock_init_data() seeds them from the sysctls net.core.rmem_default / net.core.wmem_default (the code reads sysctl_rmem_default / sysctl_wmem_default). Against those limits the kernel tracks live usage: atomic_t sk_rmem_alloc (aliased onto sk_backlog.rmem_alloc) counts bytes of skb data charged to the receive side, refcount_t sk_wmem_alloc and int sk_wmem_queued track the transmit side, int sk_forward_alloc is a per-socket pre-charged memory reservation, and atomic_t sk_omem_alloc counts “option” memory (e.g. filters). The whole accounting machinery — why sk_rcvbuf is doubled on the way in, how sk_forward_alloc amortises the global tcp_memory_allocated charge — is the subject of Socket Buffers and Memory Accounting and is not re-derived here.

Protocol state, locks, and timers

sk_state (via sock_common) is the protocol connection state. socket_lock_t sk_lock is the socket lock — a hybrid of a spinlock plus an “owned” flag that lets a process hold the socket across a sleep (lock_sock/release_sock) while still excluding softirq-context bottom halves; this is the central serialisation primitive of the socket. rwlock_t sk_callback_lock protects the callback pointers and the sk_wq association. struct timer_list sk_timer is the protocol’s primary timer (for TCP, the retransmit/keepalive timer driver). int sk_err / int sk_err_soft hold the pending error returned by SO_ERROR. u8 sk_shutdown records SHUT_RD/SHUT_WR half-close state. u32 sk_ack_backlog / sk_max_ack_backlog track the listen accept-queue occupancy and limit (the backlog argument to listen()).

The callback functions — how a socket signals userspace

A handful of function pointers are how the protocol code, running deep in softirq context, signals the socket layer that something changed. The most important:

  • void (*sk_data_ready)(struct sock *sk) — called when new data has been queued to sk_receive_queue; the default is sock_def_readable, which wakes any task sleeping in the socket’s wait queue with an EPOLLIN event.
  • void (*sk_write_space)(struct sock *sk) — called when transmit buffer space frees up (an ACK retired queued data); the default sock_def_write_space wakes writers/epoll with EPOLLOUT, but only once “significant” space is available (a Dave-Miller comment in the source guards the wake so writers are not woken for one byte of room).
  • void (*sk_state_change)(struct sock *sk) — called on connection-state transitions (e.g. a connect() completing), default sock_def_wakeup.
  • void (*sk_error_report)(struct sock *sk) — called when an error is set, wakes EPOLLERR waiters.
  • int (*sk_backlog_rcv)(struct sock *sk, struct sk_buff *skb) — the per-protocol handler that processes a backlogged skb when the socket lock is released.
  • void (*sk_destruct)(struct sock *sk) — the destructor run when the final reference drops.

sock_init_data() installs all the sock_def_* defaults. A subsystem can override them: this is precisely how a higher layer hooks the socket. For example, when a TCP listening socket accepts a child, or when a userspace consumer like the Go netpoller or an in-kernel consumer wants its own wakeup, it swaps sk_data_ready for a custom function under sk_callback_lock. The mechanics of these wakeups — the wait queue, wake_up_interruptible_sync_poll, the epoll interaction — are Socket Wait Queues and Wakeups.

The back-pointer struct socket *sk_socket closes the loop: given a sk, code reaches the BSD-layer socket (and through it the file). It is NULL for sockets with no userspace file (e.g. kernel sockets, or an inet_timewait_sock).

How inet_sock and tcp_sock Extend sock

The protocol-specific extension is by embedding at offset 0, not inheritance or pointers. struct inet_sock (in include/net/inet_sock.h) begins:

struct inet_sock {
	/* sk and pinet6 has to be the first two members of inet_sock */
	struct sock		sk;          /* MUST be first */
#if IS_ENABLED(CONFIG_IPV6)
	struct ipv6_pinfo	*pinet6;
#endif
	unsigned long		inet_flags;
	__be32			inet_saddr;  /* source address */
	__s16			uc_ttl;
	__be16			inet_sport;  /* source port */
	struct ip_options_rcu __rcu *inet_opt;
	atomic_t		inet_id;
	__u8			tos;         /* IP type-of-service */
	__u8			min_ttl;
	__u8			mc_ttl;      /* multicast TTL */
	/* … pmtudisc, mc_index, mc_list, cork … */
};

Because struct sock sk is the first member, (struct sock *)inet_sk == &inet_sk->sk is the same address, and the helper inet_sk(sk) is just container_of/cast back to struct inet_sock *. The inet_sock adds the IPv4-specific fields: source address/port (inet_saddr, inet_sport), the IP options, the type-of-service byte (tos), multicast TTL, path-MTU-discovery mode (pmtudisc), and the egress cork state (cork) used by IP_CORK/MSG_MORE. The destination address and port are not duplicated here — they live in the sock_common head (inet_daddr/inet_dport are #defined onto sk.__sk_common.skc_daddr/skc_dport) so the connection-lookup path can read them from sock_common uniformly.

TCP nests one level deeper. struct tcp_sock (in include/linux/tcp.h) begins with struct inet_connection_sock inet_conn as its first member, and inet_connection_sock begins with struct inet_sock. So the chain is tcp_sock → inet_connection_sock → inet_sock → sock → sock_common, all sharing offset 0. The inet_connection_sock layer adds connection-oriented machinery: the congestion-control operations (icsk_ca_ops), the retransmit timer, the accept queue, the address-family ops. The tcp_sock layer adds the TCP protocol variables — and in 6.12 these too are cacheline-grouped (tcp_sock_read_tx, tcp_sock_read_txrx, tcp_sock_read_rx, …): the send congestion window snd_cwnd, the receive next-expected sequence rcv_nxt, the send-unacknowledged snd_una, the maximum-segment-size cache mss_cache, the receive window, reordering metrics, and the nonagle flags that TCP_NODELAY/TCP_CORK toggle (see Socket Options and setsockopt). The helper tcp_sk(sk) casts a struct sock * to struct tcp_sock *. UDP analogously has struct udp_sock (embedding inet_sock).

This embedding-at-offset-0 pattern is the kernel’s idiom for “single inheritance” in C: a base struct as the first member, up-casts for free, down-casts via cast helpers, and one allocation holds the whole chain. When socket() creates an AF_INET stream socket, the kernel allocates a struct tcp_sock-sized object from tcp_prot.slab, and the same memory is the sock, the inet_sock, and the tcp_sock simultaneously.

Allocation and Wiring — sock_init_data

The two structures are created and linked during socket creation. sk_alloc() allocates the protocol-sized object (e.g. tcp_sock) and sock_init_data(struct socket *sock, struct sock *sk) wires it up. In 6.12 sock_init_data (in net/core/sock.c) performs the cross-linking: it sets sk->sk_type = sock->type, points the wait queue both ways with RCU_INIT_POINTER(sk->sk_wq, &sock->wq) and sock->sk = sk, seeds sk_rcvbuf/sk_sndbuf from the default sysctls, sets sk->sk_state = TCP_CLOSE, and installs the default callbacks (sk->sk_data_ready = sock_def_readable; sk->sk_write_space = sock_def_write_space; sk->sk_state_change = sock_def_wakeup; sk->sk_error_report = sock_def_error_report; sk->sk_destruct = sock_def_destruct;). It also sets sk_rcvlowat = 1, infinite send/receive timeouts (MAX_SCHEDULE_TIMEOUT), and finally refcount_set(&sk->sk_refcnt, 1) after an smp_wmb() barrier. After sock_init_data returns, sock->sk and sk->sk_socket point at each other, and the endpoint is live. The proto-specific ->init (e.g. tcp_v4_init_sock) then initialises the TCP-only fields.

Common Misunderstandings

“struct socket holds the TCP state.” No — socket->state is the BSD coarse state (SS_CONNECTED etc.). The TCP state machine value (TCP_ESTABLISHED, TCP_TIME_WAIT, …) lives in sk->sk_state inside struct sock. Conflating the two leads to reading the wrong field; tools like ss report sk_state, not socket->state.

“sk is a pointer to a separate TCP struct.” sk is a struct sock *, but the object it points at is actually a struct tcp_sock (for TCP) — the struct sock is just its first member. tcp_sk(sk) does not chase a pointer; it reinterprets the same address. Allocating a “bare” struct sock for a TCP connection would be a bug — there would be no room for the TCP fields.

“Every socket has a struct socket.” Not necessarily. Kernel-internal sockets (created with sock_create_kern) and the minimal inet_timewait_sock/request_sock connection-table entries have a struct sock (or a sock_common-compatible head) but a NULL or absent sk_socket — there is no userspace file. The socket↔sk pairing is the userspace-visible case, not a universal invariant.

“The callbacks are fixed.” sk_data_ready and friends are function pointers installed by sock_init_data to defaults, but routinely overridden — by epoll’s integration, by AF_PACKET, by kernel TLS, by tunnel encapsulation sockets. Overriding sk_data_ready (under sk_callback_lock) is the canonical way to intercept “data arrived” for a socket.

“SO_RCVBUF of N gives me N bytes of queue.” The value stored in sk_rcvbuf is doubled on the way in (__sock_set_rcvbuf does val * 2) to account for sk_buff and bookkeeping overhead; getsockopt then reports the doubled value. This is struct sock’s accounting model, detailed in Socket Buffers and Memory Accounting.

Why the Two-Layer Split Exists

The separation is not historical accident — it is what lets one body of generic code serve every protocol family. The VFS, the read/write/poll syscall machinery, and the file-descriptor lifecycle deal only in struct socket and its proto_ops, oblivious to TCP versus UDP versus AF_UNIX. The networking core and each protocol implementation deal in struct sock and struct proto, oblivious to file descriptors. The thin sk_socket/sk cross-pointers are the only coupling. A second motivation is the minimal sock variants: a connection in TIME-WAIT does not need a full tcp_sock (most of those fields are meaningless once the connection is closing), so the kernel keeps it as a tiny inet_timewait_sock that shares only sock_common — and because the demux hash tables key on sock_common fields, they can hold full socks and time-wait socks side by side. Splitting the universal lookup head (sock_common) from the full control block (sock) from the BSD wrapper (socket) is what makes that memory saving possible without special-casing the lookup.

See Also

Graph View

Backlinks