sysfs and the Kernel Object Hierarchy

sysfs is the pseudo-filesystem mounted at /sys that exposes the kernel’s unified device model — its devices, drivers, buses, and classes, and the links between them — to userspace as a directory tree. Per its own documentation, “sysfs is a RAM-based filesystem initially based on ramfs … [that] provides a means to export kernel data structures, their attributes, and the linkages between them to userspace,” and it is “tied inherently to the kobject infrastructure” (sysfs.rst, v6.12). The organizing primitive is the kobject (kernel object): a small embeddable struct carrying a name, a parent pointer, a reference count, and a type. The mapping is mechanical and total — every registered kobject becomes a directory, and each of its attributes becomes a file inside that directory. Underneath, sysfs does not implement its own VFS plumbing; it is built on kernfs, a generic in-memory pseudo-filesystem library that also backs cgroupfs. The convention that makes /sys usable — one value per file — is what distinguishes it from its older, undisciplined sibling procfs and the proc Filesystem. This note pins to the 6.12 LTS kernel (released 2024-11-17).

Mental Model

Think of sysfs as a filesystem projection of an in-kernel object graph. The kernel already maintains a tree of kobjects (a device’s kobject has a parent kobject, which has a parent, up to a subsystem root). sysfs walks that graph and renders each node as a directory and each node’s exported fields as files. The kobject was introduced by Patrick Mochel in the 2.5/2.6 development era as the backbone of the new driver model — the copyright headers in include/linux/kobject.h (2002–2003, Mochel) and include/linux/sysfs.h (2001–2002, Mochel) date the design. The struct kobject first appeared in the 2.5.45 development kernel and, per LWN’s “The Zen of Kobjects” (2003), “was initially meant as a simple way of unifying kernel code which manages reference counted objects” before “mission creep” made it central to the device model and sysfs (LWN 51437). sysfs exists because the kernel needed a structured, ABI-stable way to expose this model — explicitly as a corrective to procfs’s free-form sprawl.

flowchart TB
  subgraph KERN["In-kernel object graph"]
    KSET["kset<br/>(a set of kobjects,<br/>e.g. /sys/bus/pci)"]
    K1["kobject<br/>(a device)"]
    KT["kobj_type / ktype<br/>(release + sysfs_ops + default_groups)"]
    A1["attribute<br/>(name, mode)"]
    KSET --> K1
    K1 -->|"->ktype"| KT
    KT -->|"default_groups"| A1
  end
  K1 -->|"->sd (kernfs_node)"| KN["kernfs_node<br/>(the in-memory tree node)"]
  subgraph KERNFS["kernfs (generic pseudo-fs library)"]
    KN
  end
  KN -->|"mounted as /sys"| FS["sysfs"]
  FS -->|"VFS: read()/write()"| OPS["sysfs_ops.show / .store"]
  OPS -->|"dispatch by ktype"| SHOW["attribute show()/store()"]
  FS -.->|"kobject_uevent over netlink"| UDEV["udevd (userspace)"]

The kobject-to-sysfs projection. What it shows: a kobject belongs to a kset, has a ktype (which supplies the sysfs_ops and default attribute groups), and holds a pointer sd to a kernfs_node — the actual in-memory tree node that kernfs (a generic library, also used by cgroupfs) manages; sysfs is the mount of that tree, and file I/O is dispatched through sysfs_ops.show/.store to per-attribute callbacks. The insight: sysfs itself is thin — the real data structures are the kobject graph and the kernfs tree, and /sys is just a window onto them, with kobject_uevent notifying userspace’s device manager udevd of changes over a netlink socket.

The kobject: Names, Parents, Reference Counts, and Types

Everything in /sys ultimately rests on struct kobject, from include/linux/kobject.h:

struct kobject {
	const char		*name;
	struct list_head	entry;
	struct kobject		*parent;
	struct kset		*kset;
	const struct kobj_type	*ktype;
	struct kernfs_node	*sd;   /* sysfs directory entry */
	struct kref		kref;
	unsigned int state_initialized:1;
	unsigned int state_in_sysfs:1;
	/* ... more state bits ... */
};

Walk the load-bearing fields. name is the directory name in sysfs. parent points at the kobject whose directory will contain this one — this is what builds the /sys tree, because “that directory is created as a subdirectory of the kobject’s parent, expressing internal object hierarchies to userspace” (sysfs.rst). kset is the set this kobject belongs to (more below). ktype points at the kobject’s type descriptor, which supplies the operations and the default attributes. sd (“sysfs dentry,” historically) is the pointer into kernfs — the actual tree node. kref is the embedded reference counter.

The kobject is almost never used standalone; it is embedded in a larger structure. The canonical pattern, from the kobject documentation, is that struct device (and dozens of others) contain a struct kobject member, and code recovers the container from a kobject pointer via container_of() (kobject.rst):

struct my_object *m = container_of(kp, struct my_object, kobj);

container_of(ptr, type, member) does pointer arithmetic: given the address of the kobj field and the knowledge that it sits at a known offset inside struct my_object, it returns the address of the enclosing object. This is the kernel’s idiom for “object-oriented” composition without C++ — the kobject provides reference counting and sysfs presence; the embedding struct provides the actual data. The docs are emphatic: “No structure should EVER have more than one kobject embedded within it” (kobject.rst), because the single embedded kref is the lifetime of the whole object.

Reference counting governs lifetime. kobject_init() sets the count to one; kobject_get() increments and kobject_put() decrements; when the count hits zero, the kobject’s ktype->release() method runs to free the enclosing structure. Crucially, after kobject_add() you must never kfree() the object directly — only kobject_put(), because “other portions of the kernel can get a reference on any kobject that is registered in the system” (e.g. a userspace process holding a sysfs file open), so “a structure protected by a kobject cannot be freed before its reference count goes to zero” (kobject.rst). Every kobject must provide a release() method; the kernel warns if one is missing, and providing an empty one to silence the warning is explicitly called out as wrong.

ktype and sysfs_ops: How a File Read Reaches a Callback

The kobject’s type is struct kobj_type (a “ktype”), from include/linux/kobject.h:

struct kobj_type {
	void (*release)(struct kobject *kobj);
	const struct sysfs_ops *sysfs_ops;
	const struct attribute_group **default_groups;
	/* ... namespace / ownership hooks ... */
};

release is the destructor just discussed. default_groups is a NULL-terminated array of attribute groups that are automatically created as files when the kobject is added. sysfs_ops is the indirection that connects a read()/write() on a sysfs file to the right callback:

struct sysfs_ops {
	ssize_t (*show)(struct kobject *, struct attribute *, char *);
	ssize_t (*store)(struct kobject *, struct attribute *, const char *, size_t);
};

When userspace reads a sysfs file, sysfs “calls the appropriate method for the type. The method then translates the generic struct kobject and struct attribute pointers to the appropriate pointer types, and calls the associated methods” (sysfs.rst). The driver model’s dev_attr_show is the textbook example — it container_of()s the generic struct attribute up to a struct device_attribute and the generic struct kobject up to a struct device, then calls the attribute’s own show:

#define to_dev_attr(_attr) container_of(_attr, struct device_attribute, attr)
 
static ssize_t dev_attr_show(struct kobject *kobj, struct attribute *attr, char *buf)
{
	struct device_attribute *dev_attr = to_dev_attr(attr);
	struct device *dev = kobj_to_dev(kobj);
	ssize_t ret = -EIO;
	if (dev_attr->show)
		ret = dev_attr->show(dev, dev_attr, buf);
	if (ret >= (ssize_t)PAGE_SIZE)
		printk("dev_attr_show: %pS returned bad count\n", dev_attr->show);
	return ret;
}

This two-level dispatch — generic sysfs_ops per ktype, specific show/store per attribute — is what lets one filesystem (sysfs) serve heterogeneous object types (devices, drivers, buses, modules) without knowing anything about any of them.

Attributes: One Value Per File

An attribute is a file. The base type, from include/linux/sysfs.h, is deliberately tiny:

struct attribute {
	const char	*name;   /* the filename */
	umode_t		mode;    /* permission bits */
};

A bare attribute carries no read/write logic; subsystems wrap it. The driver model’s struct device_attribute embeds an attribute plus a show/store pair:

struct device_attribute {
	struct attribute attr;
	ssize_t (*show)(struct device *dev, struct device_attribute *attr, char *buf);
	ssize_t (*store)(struct device *dev, struct device_attribute *attr,
			 const char *buf, size_t count);
};

The cardinal convention of sysfs is stated bluntly in the docs: “Attributes should be ASCII text files, preferably with only one value per file. … Mixing types, expressing multiple lines of data, and doing fancy formatting of data is heavily frowned upon. Doing these things may get you publicly humiliated and your code rewritten without notice” (sysfs.rst). This one-value-per-file rule is the entire reason /sys is scriptable: cat /sys/class/net/eth0/speed returns a number, not a paragraph you must parse. It is the precise discipline procfs lacks.

show() and store(): The Single-Call, Page-Buffer Contract

sysfs’s read/write contract is unusual and must be understood to write a correct attribute. “sysfs allocates a buffer of size (PAGE_SIZE) and passes it to the method. sysfs will call the method exactly once for each read or write” (sysfs.rst). Consequences:

• On read, show() must fill the entire value into the buffer in one shot; userspace partial reads and seeks are served from sysfs’s copy. Seeking back to zero re-arms show() (it is called again). The buffer is always PAGE_SIZE (4096 on x86).
• On write, the whole buffer is passed to store() in the first write; sysfs appends a terminating NUL so helpers like sysfs_streq() are safe. Userspace should read-modify-write the whole value.
• New show() implementations “should only use sysfs_emit() or sysfs_emit_at()” to format output — these are bounds-checked wrappers that prevent the PAGE_SIZE overruns that plagued hand-rolled snprintf code. A minimal example: return sysfs_emit(buf, "%s\n", dev->name); (sysfs.rst).

Declaring Attributes: The __ATTR Macros and Groups

Hand-filling an attribute struct is verbose, so sysfs provides macros (sysfs.rst): __ATTR_RO(name) assumes a name_show and mode 0444 (read-only); __ATTR_WO(name) a name_store and mode 0200 (root write only); __ATTR_RW(name) both, mode 0644; __ATTR_NULL is the end-of-list sentinel. The driver-model wrappers DEVICE_ATTR_RO/RW/WO, BUS_ATTR_*, and DRIVER_ATTR_* build on these. Rather than registering attributes one at a time, the modern idiom bundles them into a struct attribute_group (an attrs array plus an optional name that creates a subdirectory and an optional is_visible callback to hide attributes dynamically), and points the ktype’s default_groups at it so all the files appear and disappear atomically with the kobject (sysfs.h). For the simplest case, kobject_create_and_add("name", parent) plus sysfs_create_group() makes a directory with kobj_attribute files without defining a custom ktype at all (kobject.rst).

kset: Grouping kobjects into Subsystems

A kset is “a group of kobjects” that “want to be grouped together and operated on in the same manner” — it is “the basic container type for collections of kobjects” (kobject.rst). Structurally a kset embeds its own kobject, so a kset is itself a directory in sysfs, and the kobjects belonging to it appear beneath it:

struct kset {
	struct list_head list;        /* all kobjects in this set */
	spinlock_t list_lock;
	struct kobject kobj;          /* the kset's own directory */
	const struct kset_uevent_ops *uevent_ops;
};

/sys/bus/pci is a kset; each PCI device’s kobject is a member. The uevent_ops field is the kset’s hook into the uevent machinery — when a member kobject changes, the kset can filter or annotate the resulting event (e.g. add a SUBSYSTEM= variable). This is why uevents carry consistent subsystem metadata: the kset, not the individual kobject, owns the policy.

kernfs: The In-Memory Tree Underneath

sysfs does not implement directory locking, inode management, or the dentry plumbing itself. It is built on kernfs, described in include/linux/kernfs.h as a “pseudo filesystem decoupled from vfs locking.” kernfs was factored out of sysfs (historically sysfs’s internals) into a reusable library so that other pseudo-filesystems could share it — most importantly cgroupfs (the control-group filesystem) is also a kernfs client. Each node in the tree is a struct kernfs_node:

struct kernfs_node {
	atomic_t		count;
	atomic_t		active;
	struct kernfs_node	*parent;
	const char		*name;
	struct rb_node		rb;        /* siblings in a red-black tree */
	unsigned short		flags;     /* KERNFS_DIR / KERNFS_FILE / KERNFS_LINK */
	umode_t			mode;
	union {
		struct kernfs_elem_dir		dir;
		struct kernfs_elem_symlink	symlink;
		struct kernfs_elem_attr		attr;
	};
	u64			id;
	void			*priv;     /* back-pointer, e.g. to the kobject */
	struct rcu_head		rcu;
};

Each kernfs_node is a directory, a regular file (attribute), or a symlink, discriminated by flags (KERNFS_DIR, KERNFS_FILE, KERNFS_LINK). Siblings are kept in a red-black tree (rb) for fast name lookup. The priv field is the back-pointer sysfs uses to store “a pointer to the kobject that implements a directory” (sysfs.rst) — that is the kobject->sd ↔ kernfs_node->priv linkage that ties the two halves together. kernfs holds the whole tree in RAM (there is no backing device), manages its own locking with a hashed lock table sized by CPU count to reduce contention (kernfs.h), and exposes kernfs_ops callbacks (seq_show, read, write) that sysfs and cgroupfs fill in. In short: kernfs is the generic “in-memory hierarchy of nodes with per-file read/write callbacks” engine; sysfs and cgroupfs are two policies layered on it.

The Top-Level Layout of /sys

The first-level directories of /sys encode the relationships among kernel structures (sysfs.rst):

• devices/ — “a filesystem representation of the device tree. It maps directly to the internal kernel device tree, which is a hierarchy of struct device.” This is the canonical location; everything else is symlinks into it.
• bus/ — one directory per bus type (pci, usb, i2c, …); each contains devices/ (symlinks to the devices on that bus, pointing back into devices/) and drivers/ (one directory per loaded driver for that bus).
• class/ — devices grouped by functional type (net/, block/, tty/, …) regardless of which bus they sit on; symlinks back into devices/.
• block/ — symlinks to block devices.
• dev/ — char/ and block/ subdirectories of <major>:<minor> symlinks, “a quick way to lookup the sysfs interface for a device from the result of a stat(2).”
• module/ — parameters and state for loaded modules (/sys/module/<name>/parameters/<param> is how you read a module parameter at runtime).
• fs/ — per-filesystem attribute hierarchies (e.g. /sys/fs/cgroup, FUSE’s directory).
• firmware/, kernel/, power/, hypervisor/ — firmware tables, kernel runtime knobs, power-management state, and hypervisor data; the kernel exposes the kernel_kobj, power_kobj, firmware_kobj, hypervisor_kobj, and mm_kobj globals (kobject.h) as the chaining points for these subtrees.

The same physical device thus appears multiple times: once under devices/ (the real node) and as symlinks under bus/, class/, block/, and dev/. The docs stress this is an ABI: “the sysfs directory structure and the attributes in each directory define an ABI between the kernel and user space,” and all new attributes “must be documented in Documentation/ABI” (sysfs.rst).

uevents and udev: Pushing Changes to Userspace

sysfs is a pull interface — userspace reads files. But hotplug needs push: when a USB stick is inserted, userspace must be told a new device appeared. That is the job of uevents. When a kobject is added, removed, or changed, the kernel calls kobject_uevent(kobj, action) with an action from the kobject_action enum (KOBJ_ADD, KOBJ_REMOVE, KOBJ_CHANGE, KOBJ_MOVE, KOBJ_ONLINE, KOBJ_OFFLINE, KOBJ_BIND, KOBJ_UNBIND — kobject.h). The implementation in lib/kobject_uevent.c builds an environment of KEY=value strings — add_uevent_var(env, "ACTION=%s", ...), "DEVPATH=%s" (the device’s sysfs path), "SUBSYSTEM=%s" — and broadcasts it as a netlink message over the NETLINK_KOBJECT_UEVENT socket via netlink_broadcast(). Userspace’s device manager, udevd (systemd-udevd), listens on that netlink socket, receives each uevent, applies its rule database (/etc/udev/rules.d/, /usr/lib/udev/rules.d/), and acts — creating /dev nodes, setting permissions, loading firmware, running scripts, assigning persistent network names. So the full loop is: kernel changes the kobject graph → sysfs reflects it as files → kobject_uevent notifies udev over netlink → udev consumes the event and reads back the details from /sys. You can synthesize events for testing by writing to a kobject’s uevent attribute (kobject_synth_uevent), which is how udevadm trigger re-runs rules.

Uncertain

Verify: that udevadm trigger works by writing add/change to each device’s /sys/.../uevent attribute (driving kobject_synth_uevent). Reason: the kernel side (kobject_synth_uevent exists in kobject.h) is verified, but the exact userspace udevadm mechanism was not read from udev/systemd source during this task. To resolve: read udevadm-trigger(8) and the systemd-udevd source. uncertain

Failure Modes and Misunderstandings

• Putting more than one value in a file. The single most common sysfs mistake; it breaks the scripting contract and gets rejected in review. If you need multiple values, you need multiple files (or, rarely, an array of homogeneous values). The rule is honored imperfectly in practice: a 2010 survey of 9,254 sysfs files found roughly 10% (897 files) violating the one-value guideline with multi-word or structured content — Bluetooth records with several fields per line, PCI statistics with hex dumps — and inconsistent unit and boolean conventions across the tree (LWN 378884). The convention is policy, not enforcement.
• Exceeding PAGE_SIZE in show(). Because the buffer is exactly one page and the method is called once, output longer than ~4096 bytes is truncated. sysfs_emit() guards against this; hand-rolled sprintf does not.
• kfree() after kobject_add(). Frees an object other code may still reference via an open sysfs file → use-after-free. Always kobject_put().
• Missing or empty release(). The kernel warns, and an empty release leaks the embedding structure. Every ktype needs a real release that container_of()s and kfree()s.
• Assuming the object is “present” because the directory exists. sysfs reference-counts the kobject, which may outlive the physical device (a removed disk whose sysfs node is still open). A show() may need to check liveness.
• Confusing sysfs with procfs or debugfs. New per-device knobs go in sysfs (stable ABI, one value per file); throwaway debug interfaces go in debugfs (no ABI promise); legacy free-form data lives in procfs.

Alternatives and When to Choose Them

• procfs (/proc) — the older pseudo-filesystem; process info and the /proc/sys sysctl tree. sysfs was created precisely to take the structured device-model data out of /proc’s undisciplined text files. Both are pseudo-filesystems with on-read generation, but procfs is not built on kernfs and has no one-value-per-file rule.
• debugfs (/sys/kernel/debug) — for debugging interfaces with no stability guarantee; the right home for ad-hoc files that must not become ABI.
• cgroupfs (/sys/fs/cgroup) — a sibling kernfs client, not an alternative: it shares the same in-memory tree machinery but exposes control-group hierarchies rather than the device model. Understanding kernfs explains why cgroupfs feels structurally like sysfs.
• netlink (directly) — for high-rate, push-style kernel↔userspace messaging without a filesystem face; uevents already use a netlink family, but general configuration (routing, etc.) uses netlink directly rather than sysfs.

Production Notes

/sys is the substrate of modern Linux device management. udevd turns its uevents into /dev nodes and persistent network-interface names; lsblk, lspci, and nvme list read device topology from /sys/class/block, /sys/bus/pci, etc.; container runtimes and orchestrators read /sys for device and (via the kernfs-shared cgroupfs) resource information; power-management and thermal daemons write tunables under /sys/devices/system/cpu/.../cpufreq/ and /sys/class/thermal/. Reading a CPU’s current frequency (cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq) or a network link’s speed (cat /sys/class/net/eth0/speed) is the everyday face of the show/store machinery described above. The ABI-stability promise matters operationally: because tools parse exact sysfs paths and single-value formats, kernel developers cannot rename or restructure existing attributes without breaking userspace — which is why the documentation insists every new attribute be registered in Documentation/ABI before merge.

See Also

• procfs and the proc Filesystem — the older, undisciplined pseudo-filesystem sysfs was built to supplant for device-model data
• The Virtual File System Layer — the VFS layer through which sysfs/kernfs reads and writes are dispatched
• Mount Namespaces — how /sys (and the kernfs-based cgroupfs) is presented per-container
• MOC: Linux Filesystems and VFS MOC