vault

fanotify and Filesystem Monitoring

16 min read

fanotify and Filesystem Monitoring

fanotify (“filesystem access notify”) is Linux’s powerful, privileged file-monitoring API — the heavyweight counterpart to inotify File System Notification. Where inotify watches individual inodes and only tells you about changes, fanotify can watch an entire mount or an entire filesystem with a single mark, can deliver an open file descriptor (or a portable file handle) straight to the affected object, can report which process caused the event, and — uniquely — can intercept and deny an operation before it happens via permission events, which is what antivirus scanners and access-control daemons are built on (per fanotify(7)). Both fanotify and inotify ride on the same in-kernel fsnotify framework, so they share the same detection hooks, but fanotify exposes far richer machinery at the cost of requiring CAP_SYS_ADMIN for its potent modes. This note pins behavior to 6.12 LTS as the baseline and is explicit about the cluster of features that arrived after 6.12 and are present only in 6.18 LTS — most notably the pre-content/HSM event FAN_PRE_ACCESS (kernel 6.14) and mount-attach/detach events (kernel 6.15), verified by diffing include/uapi/linux/fanotify.h at the v6.12 and v6.18 tags (2026-06-04).

Mental Model

Think of fanotify as inotify with three superpowers and one tax. The superpowers: (1) you can attach a mark to a whole mount or a whole filesystem instead of one inode at a time, eliminating inotify’s per-directory watch explosion; (2) each event can carry a real, openable reference to the object — either an fd the kernel opened for you, or a stable file handle you resolve with open_by_handle_at(2); (3) some events are synchronous and blocking — the kernel pauses the process that triggered the access and waits for your allow/deny verdict. The tax: the interesting modes need CAP_SYS_ADMIN, so fanotify is a tool for daemons and system services, not unprivileged apps.

The pivotal concept is the notification group class, chosen once at fanotify_init() and immutable thereafter. There are three, and they form a priority ladder for permission decisions: FAN_CLASS_PRE_CONTENT (highest) → FAN_CLASS_CONTENT → FAN_CLASS_NOTIF (lowest, the default, notification-only). When an operation triggers permission events in several groups, the kernel consults them in that order (per fanotify_init(2)). The class also gates which events a group may request: only the two content classes may register permission events.

flowchart TB
  subgraph U["Userspace daemon (CAP_SYS_ADMIN)"]
    INIT["fanotify_init(FAN_CLASS_CONTENT,...)<br/>-> group fd"]
    MARK["fanotify_mark(fd, FAN_MARK_MOUNT,<br/>FAN_OPEN_PERM, '/mnt')"]
    LOOP["read(fd) -> metadata + fd/handle"]
    RESP["write(fd, {fd, FAN_ALLOW or FAN_DENY})"]
  end
  subgraph K["Kernel"]
    OP["process opens /mnt/file"]
    HOOK["fsnotify permission hook"]
    BLOCK["BLOCK opener<br/>until response"]
  end
  INIT --> MARK
  OP --> HOOK
  HOOK -->|"permission event"| LOOP
  LOOP --> RESP
  RESP --> BLOCK
  BLOCK -->|"FAN_ALLOW: proceed<br/>FAN_DENY: EPERM"| OP

The permission-event control loop. What it shows: a FAN_OPEN_PERM mark on a mount causes every open under /mnt to block in the kernel while the fanotify daemon reads the event, inspects the delivered fd, and writes back FAN_ALLOW or FAN_DENY; deny turns the opener’s open() into EPERM. The insight to take: unlike inotify (which only ever reports the past), fanotify permission events sit in the critical path of the operation — this is exactly why antivirus can stop a malicious file from being opened, and exactly why a buggy or slow fanotify daemon can hang every process on the system.

Relationship to fsnotify and inotify

fanotify is a backend of the kernel’s shared fsnotify framework (fs/notify/), the same framework that backs inotify File System Notification. The framework supplies the VFS hook points and the abstract notion of a mark — a subscription that can be attached to an inode, a mount, or a superblock. inotify only ever uses inode marks; fanotify uses all three, which is the structural reason it can do mount-wide and filesystem-wide watching while inotify cannot. (The full fsnotify/FS_*-bit explanation lives in the inotify File System Notification note; here it is enough to know both APIs share the same detection plumbing and therefore the same blind spots — mmap writes, remote NFS changes, and pseudo-filesystems are invisible to both.)

The Two Calls

fanotify, like inotify, is built from two system calls plus read/write/close.

fanotify_init(unsigned int flags, unsigned int event_f_flags) creates a notification group and returns its fd. flags carries exactly one class (FAN_CLASS_NOTIF / FAN_CLASS_CONTENT / FAN_CLASS_PRE_CONTENT) OR’d with reporting and resource modifiers; event_f_flags is the open mode (O_RDONLY/O_WRONLY/O_RDWR, plus O_LARGEFILE, O_CLOEXEC, etc.) that the kernel uses when it opens the fd it hands you for each event. The reporting modifiers are the headline features and each is version-gated (see the spine below): FAN_REPORT_FID, FAN_REPORT_DIR_FID, FAN_REPORT_NAME (and the convenience FAN_REPORT_DFID_NAME), FAN_REPORT_TARGET_FID, FAN_REPORT_PIDFD, and FAN_REPORT_TID. Resource modifiers FAN_UNLIMITED_QUEUE and FAN_UNLIMITED_MARKS lift the per-user caps (both require CAP_SYS_ADMIN).

fanotify_mark(int fanotify_fd, unsigned int flags, uint64_t mask, int dirfd, const char *pathname) adds, removes, or flushes a mark. flags combines an operation (FAN_MARK_ADD / FAN_MARK_REMOVE / FAN_MARK_FLUSH) with a mark type (FAN_MARK_INODE default / FAN_MARK_MOUNT / FAN_MARK_FILESYSTEM) and optional modifiers. mask is the set of FAN_* events. This is where fanotify’s reach is selected:

  • FAN_MARK_INODE (default, value 0) — watch one file or directory, like inotify.
  • FAN_MARK_MOUNT (value 0x10) — watch every object accessed through a given mount point. One mark covers the whole subtree, no recursion, no per-directory cost. The catch: directory-entry events (FAN_CREATE/FAN_DELETE/FAN_MOVED_*) are not allowed on mount marks (you get EINVAL), because a mount mark cannot carry the name/handle info those events need.
  • FAN_MARK_FILESYSTEM (value 0x100, since Linux 4.20) — watch every object on the whole superblock, across all its mount points. This is the broadest scope and the one that truly replaces “recursively watch a tree.”

Modifiers include FAN_MARK_DONT_FOLLOW (mark a symlink itself), FAN_MARK_ONLYDIR (fail with ENOTDIR if not a directory), and the ignore-mask family (FAN_MARK_IGNORED_MASK, the newer FAN_MARK_IGNORE, and *_SURV_MODIFY to make an ignore survive modify events).

The Event Record and Getting at the File

Reading the group fd yields a stream of struct fanotify_event_metadata records. From the UAPI header (v6.12):

#define FANOTIFY_METADATA_VERSION 3
struct fanotify_event_metadata {
    __u32 event_len;     /* total length of this record incl. info records */
    __u8  vers;          /* must equal FANOTIFY_METADATA_VERSION */
    __u8  reserved;
    __u16 metadata_len;  /* length of this fixed struct */
    __aligned_u64 mask;  /* the FAN_* event bits */
    __s32 fd;            /* open fd to the object, or FAN_NOFD (-1) */
    __s32 pid;           /* PID, or TID if FAN_REPORT_TID, or 0 */
};

Field by field: event_len lets you stride to the next record (events are not fixed-length once info records are attached); vers must be checked against FANOTIFY_METADATA_VERSION (currently 3) before you trust the layout; mask is the union of FAN_* bits that fired; pid identifies the actor; and fd is fanotify’s signature feature. Unless the group was created with a FAN_REPORT_FID-family flag, the kernel opens the affected object for you and puts a ready-to-use file descriptor in fd — you can read() it, fstat() it, or scan it for malware, then you must close() it (forgetting to is the classic fanotify fd leak). If no object is associated, fd is FAN_NOFD (−1).

FID mode: events without an fd-per-object

Opening an fd for every event is expensive and, for create/delete/move events, semantically wrong (the object may not exist, or there may be no single object). The fix is FAN_REPORT_FID (Linux 5.1) and friends. In FID mode the kernel does not open an fd; instead it appends an info record after the fixed metadata containing a __kernel_fsid_t plus an opaque struct file_handle:

struct fanotify_event_info_fid {
    struct fanotify_event_info_header hdr;  /* info_type, len */
    __kernel_fsid_t fsid;
    unsigned char handle[];   /* a struct file_handle for open_by_handle_at(2) */
};

You resolve that handle to a real fd on demand with open_by_handle_at(2), so a daemon that only needs the identity of objects (a dedup engine, an indexer) never pays for opening files it does not actually read. The related flags refine what identity you get (info-type constants from the UAPI header):

  • FAN_REPORT_FID — a handle to the object itself (FAN_EVENT_INFO_TYPE_FID).
  • FAN_REPORT_DIR_FID — a handle to the parent directory (FAN_EVENT_INFO_TYPE_DFID), needed for directory-entry events.
  • FAN_REPORT_NAME — adds the entry name after the directory handle (FAN_EVENT_INFO_TYPE_DFID_NAME); requires FAN_REPORT_DIR_FID, hence the convenience macro FAN_REPORT_DFID_NAME = FAN_REPORT_DIR_FID | FAN_REPORT_NAME (Linux 5.9). This is what finally lets fanotify report “file foo was created in directory X” with a stable identity and a name — the combination inotify could never give with stable identity.
  • FAN_REPORT_TARGET_FID (Linux 5.17) — the handle of the target of a dirent change, so you can tell the created/deleted child apart from its parent.

FID mode is also what unlocks the create/delete/move/rename events (FAN_CREATE, FAN_DELETE, FAN_MOVED_FROM/TO, FAN_RENAME — the latter added in 5.17), which fanotify simply could not report before 5.1.

Permission Events: Allow / Deny

The feature inotify has nothing comparable to. A group in FAN_CLASS_CONTENT or FAN_CLASS_PRE_CONTENT may register permission events: FAN_OPEN_PERM (open is being attempted), FAN_ACCESS_PERM (a read is being attempted), and FAN_OPEN_EXEC_PERM (an execve/open-for-exec is being attempted). When such an event fires, the triggering process blocks in the kernel and the fanotify daemon must respond by write()ing a struct fanotify_response { __s32 fd; __u32 response; } back to the group fd, with response set to FAN_ALLOW or FAN_DENY. Deny makes the original syscall fail with EPERM. Optional bits: FAN_AUDIT records the decision in the audit log, and FAN_INFO (Linux 6.3) attaches extra context. This blocking, in-band design is why antivirus products (e.g. ClamAV’s on-access scanning) and mandatory-access daemons use fanotify and not inotify — they need to stop the access, not merely learn about it afterward.

Uncertain

Verify: that FAN_OPEN_EXEC_PERM and FAN_AUDIT/FAN_INFO behave as described on 6.12 specifically (vs only being documented in the latest man pages). Reason: the FAN_*_PERM and FAN_AUDIT constants are present in the v6.12 UAPI header (confirmed), but FAN_INFO/FAN_RESPONSE_INFO_AUDIT_RULE semantics were refined in 6.x point releases per the man page. To resolve: cross-check the v6.12 fanotify_user.c response-handling path and the fanotify_init(2) “VERSIONS” section against the running kernel. uncertain

The 6.12 → 6.18 Version Spine

This is the load-bearing accuracy content. The two facts that decide whether a feature is usable are (a) is the flag in the UAPI header, and (b) does the kernel implement it. The split below comes from directly diffing include/uapi/linux/fanotify.h at the v6.12 and v6.18 tags (2026-06-04) and cross-checking intermediate tags.

Present in BOTH 6.12 LTS and 6.18 LTS (already mature by 6.12):

  • FID reporting: FAN_REPORT_FID (5.1), FAN_REPORT_DIR_FID (5.9), FAN_REPORT_DFID_NAME (5.9), FAN_REPORT_TARGET_FID (5.17).
  • Directory-entry events FAN_CREATE/FAN_DELETE/FAN_MOVED_* (5.1) and FAN_RENAME (5.17).
  • Filesystem-wide marks FAN_MARK_FILESYSTEM (4.20 per fanotify_mark(2)).
  • FAN_MARK_EVICTABLE (5.19) and FAN_MARK_IGNORE (6.0).
  • FAN_REPORT_PIDFD (5.15) and FAN_FS_ERROR reporting.
  • Permission events and FAN_ENABLE_AUDIT.

Added AFTER 6.12 — present only in 6.18 LTS (absent from the v6.12 header):

  • Pre-content / HSM event FAN_PRE_ACCESS (value 0x00100000) and the range info record struct fanotify_event_info_range — kernel 6.14. Confirmed: absent from v6.13 UAPI, present from v6.14.
  • Mount events FAN_MNT_ATTACH / FAN_MNT_DETACH, the reporting flag FAN_REPORT_MNT, the mark type FAN_MARK_MNTNS, and struct fanotify_event_info_mnt — kernel 6.15 (present in v6.15 UAPI).
  • FAN_DENY_ERRNO — letting a deny response return an errno other than EPERM (e.g. EIO, EBUSY) — kernel 6.13.
  • FAN_REPORT_FD_ERROR and FAN_EVENT_INFO_TYPE_RANGE/_MNT info types — same 6.14/6.15 window.

Uncertain

Verify: the precise merge-window kernel version for each “after-6.12” feature (I pinned FAN_PRE_ACCESS→6.14, mount events→6.15, FAN_DENY_ERRNO→6.13 from UAPI-header presence across tags, not from changelog confirmation of the merge commit). Reason: a flag can appear in a -rc header slightly before/after the final release tag, so header-presence is strong but not commit-level proof. To resolve: git log --oneline the relevant lines of include/uapi/linux/fanotify.h to find the introducing commit and its first release tag. The 6.12-absent / 6.18-present split itself is verified by direct diff. uncertain

Pre-content events and the FAN_PRE_MODIFY trap

The HSM (Hierarchical Storage Management) story is the marquee post-6.12 addition, and it carries a verification trap. Pre-content events let a userspace daemon fill a file’s content on demand, before the read completes — the use case is a cloud-storage gateway that keeps stub files locally and streams the real bytes from a remote tier only when something actually reads them (LWN, “Fanotify and HSM”; LWN, “fanotify: add pre-content hooks”). Regular FAN_ACCESS_PERM was insufficient because it fires at open time and forces downloading the whole file; pre-content events fire on the actual byte-range access and carry an offset/count range (struct fanotify_event_info_range) so the daemon populates only the bytes needed. The daemon’s group must use FAN_CLASS_PRE_CONTENT, the highest priority class, and the filesystem must advertise the kernel-internal FS_ALLOW_HSM flag (initially xfs, ext4, bcachefs, gfs2). Meta runs this in production; their cover-letter figure: a kernel tree that is 1.6 GB fully populated occupies 125 MB as stubs (Phoronix).

The trap: the patch series and the LWN write-ups discuss both FAN_PRE_ACCESS and FAN_PRE_MODIFY. But the v6.18 UAPI header defines only FAN_PRE_ACCESS — there is no FAN_PRE_MODIFY userspace constant (verified: it is absent from both the v6.14 and v6.18 headers, 2026-06-04). So as of 6.18 LTS the userspace-visible pre-content event is FAN_PRE_ACCESS alone. A second, important clarification: the class FAN_CLASS_PRE_CONTENT (0x00000008) is old — it is already in the v6.12 header and governs permission-event priority; what 6.14 introduced is the pre-content event and its range info, not the class. So the accurate statement is “the pre-content event FAN_PRE_ACCESS is new in 6.14, though the priority class predates it,” not “pre-content is new in 6.14.”

This was verified directly: grepping FAN_PRE_MODIFY in include/uapi/linux/fanotify.h at every tag from v6.14 through v6.18 returns zero matches, while FAN_PRE_ACCESS is present at each (checked 2026-06-04). So FAN_PRE_ACCESS is the only userspace-visible pre-content event through 6.18 LTS; any FAN_PRE_MODIFY you see referenced is design discussion, not a shipped userspace constant.

Resource Limits

fanotify’s caps live under /proc/sys/fs/fanotify/ and mirror inotify’s structure. From fs/notify/fanotify/fanotify_user.c (v6.12): max_queued_events defaults to FANOTIFY_DEFAULT_MAX_EVENTS = 16384; max_user_groups defaults to FANOTIFY_DEFAULT_MAX_GROUPS = 128; and max_user_marks is not the old 8192 — that is the legacy per-group limit and now serves as the floor of a per-user clamp. The default per-user mark limit is computed as FANOTIFY_OLD_DEFAULT_MAX_MARKS (8192) * FANOTIFY_DEFAULT_MAX_GROUPS (128) = 1,048,576 and then clamped into [8192, 1048576] against memory cost (each inode mark pins the marked inode, costing roughly 2 * sizeof(struct inode)). So, exactly as with inotify’s max_user_watches, the widely-quoted “8192 marks” figure is the floor, and the real default on a normal machine is ~1 million. FAN_UNLIMITED_QUEUE and FAN_UNLIMITED_MARKS (both CAP_SYS_ADMIN) bypass these entirely — antivirus and HSM daemons set them so a busy system cannot make them drop events.

Failure Modes

  • A slow or crashing permission-event daemon hangs the system. Because permission events block the triggering process in-kernel until you respond, a fanotify daemon that stalls, deadlocks, or dies without draining its queue can freeze every process that touches the watched mount. This is the single scariest fanotify operational risk and why such daemons are written with extreme care (bounded response time, watchdogs).
  • Self-deadlock. If the permission-event handler itself opens the watched file, it triggers another permission event and can deadlock against itself; the pre-content work had to introduce special handling (a “no-perm” fd) precisely to break this (LWN). Always access watched files through descriptors that won’t re-trigger your own group.
  • EINVAL from mount marks + dirent events. Asking for FAN_CREATE/FAN_DELETE/FAN_MOVED_* on a FAN_MARK_MOUNT fails — those events require inode or filesystem marks plus a FID-reporting group.
  • ENODEV on filesystems with no fsid. Mount/filesystem marks need a non-zero filesystem ID; some filesystems (notably FUSE in older configs) report a zero fsid and reject these marks.
  • fd leaks. In non-FID mode every event delivers an fd you must close(); a loop that forgets exhausts the process’s descriptor table fast.
  • Shared fsnotify blind spots. Like inotify File System Notification, fanotify cannot see mmap-write/msync/munmap, cannot see remote changes on NFS, and cannot watch /proc, /sys, /dev/pts.
  • Queue overflow. Without FAN_UNLIMITED_QUEUE, a slow reader overflows and gets FAN_Q_OVERFLOW; events are lost and a rescan is the only recovery.

Alternatives and When to Choose Them

  • inotify File System Notification — choose it when you do not need fanotify’s power and you cannot be privileged. inotify needs no CAP_SYS_ADMIN and is perfect for unprivileged “watch this directory” UI use. Reach for fanotify when you need mount/filesystem-wide scope without per-directory watches, an fd/handle to the object, the actor PID, or the ability to deny.
  • FAN_MARK_FILESYSTEM vs recursive inotify — the decisive trade. One filesystem-wide fanotify mark replaces hundreds of thousands of inotify watches (and dodges inotify’s max_user_watches ENOSPC wall), but it requires privilege and sees events from the entire filesystem (you filter in userspace). For a privileged daemon watching a whole volume, this is the correct design; for a sandboxed app watching one folder, inotify is simpler.
  • eBPF / LSM hooks — for some security/observability use cases you can hook the VFS via eBPF instead. More flexible and programmable, but a much larger engineering surface than a fanotify permission daemon; choose fanotify when its allow/deny model fits.
  • io_uring and the File Path / Asynchronous IO Models in Linux — orthogonal; these are about issuing I/O asynchronously, not being notified of others’ file activity.

Production Notes

The flagship production consumers are on-access antivirus (ClamAV’s clamonacc, and commercial endpoint scanners) using FAN_OPEN_PERM/FAN_ACCESS_PERM to block infected files, and HSM/tiered-storage gateways. CTERA’s cloud-gateway product motivated the entire pre-content-events effort, having previously had to implement HSM over FUSE with its attendant problems (LWN). Meta runs fanotify-based HSM in production for tiered storage and drove the 6.14 FAN_PRE_ACCESS work alongside Amir Goldstein and Josef Bacik (Phoronix). System indexers and audit tooling use FAN_MARK_FILESYSTEM + FAN_REPORT_DFID_NAME to track every change on a volume with stable file identity — the workload that recursive inotify handles badly. The operational golden rule across all of these: a fanotify permission daemon is in the critical path of every guarded access, so it must respond within a tight, bounded time and must never block on anything that could itself touch the watched filesystem.

See Also

Graph View