Finders and Loaders

When you write import foo, CPython does not reach for the filesystem directly. It runs a two-phase protocol: a finder answers where does foo live and who knows how to load it?, returning a module spec; a loader then creates and executes the module object. Since PEP 451 (Python 3.4) the spec — an importlib.machinery.ModuleSpec — is the contract that decouples the two halves, carrying everything the loader needs (PEP 451; import reference). There are two kinds of finder — meta path finders registered on sys.meta_path, and path entry finders discovered through sys.path_hooks — and one loader protocol built from create_module() and exec_module(). This note traces that protocol end to end; its sibling sys.modules and the Module Cache covers the cache that short-circuits it before any finder is consulted.

Mental Model

The cleanest way to think about import is as a pipeline with a cache gate at the front and two cooperating roles behind it. The cache gate is sys.modules (the sibling note). If the name misses there, control reaches the finding phase: CPython walks sys.meta_path asking each meta path finder “do you handle this name?” Each finder returns either a fully-populated ModuleSpec or None. The first non-None spec wins. Then the loading phase takes that spec, builds the module object, registers it in sys.modules, and executes the module body into the module’s namespace.

flowchart TD
    A["import foo<br/>(__import__ → _gcd_import)"] --> B{"foo in sys.modules<br/>and fully initialized?"}
    B -- yes --> Z["return cached module<br/>(see sys.modules note)"]
    B -- no --> C["_find_spec(name, path, target)"]
    C --> D["iterate sys.meta_path:<br/>BuiltinImporter →<br/>FrozenImporter →<br/>PathFinder"]
    D --> E{"any finder returns<br/>a ModuleSpec?"}
    E -- no --> X["raise ModuleNotFoundError"]
    E -- yes --> F["PathFinder only:<br/>walk path entries,<br/>sys.path_hooks →<br/>PathEntryFinder<br/>(cached in<br/>sys.path_importer_cache)"]
    F --> G["spec returned"]
    E -- "builtin/frozen" --> G
    G --> H["_load_unlocked(spec):<br/>module_from_spec →<br/>create_module"]
    H --> I["register in sys.modules<br/>BEFORE executing"]
    I --> J["loader.exec_module(module)"]
    J --> Z2["fully initialized module"]

Figure: the finder/loader pipeline. The insight to extract is that finding and loading are separate, ordered phases joined by a single ModuleSpec object: meta path finders only locate and hand back a spec; the loader inside that spec does the work. The PathFinder branch has a sub-pipeline of its own (path entry finders), which is why there are “two finder types.” The registration-before-execution step (I) is the hinge that makes circular imports possible — that mechanism is owned by sys.modules and the Module Cache.

The Two Finder Types

A finder is an object that, given a module name, decides whether it can locate that module and — if so — returns a spec describing how to load it. CPython has two distinct finder protocols, and conflating them is the single most common source of confusion when writing a custom importer.

Meta path finders (sys.meta_path)

A meta path finder is an object on the list sys.meta_path that implements find_spec(name, path, target) (import reference). When a module is not in sys.modules, “Python next searches sys.meta_path, which contains a list of meta path finder objects. These finders are queried in order to see if they know how to handle the named module” (per the import reference). The method takes three arguments: the fully qualified name (foo.bar.baz), the import path (None for a top-level module, or the parent package’s __path__ for a submodule), and an optional target module — “the import system passes in a target module only during reload” (per the import reference). If the finder can handle the module it returns a ModuleSpec; otherwise it returns None. “If sys.meta_path processing reaches the end of its list without returning a spec, then a ModuleNotFoundError is raised.”

The meta path is traversed once per name segment. Importing foo.bar.baz calls mpf.find_spec("foo", None, None) on each finder, then after foo loads, mpf.find_spec("foo.bar", foo.__path__, None), then mpf.find_spec("foo.bar.baz", foo.bar.__path__, None) (per the import reference). This is why the second argument matters: top-level finders that “only support top level imports … will always return None when anything other than None is passed as the second argument.”

The actual driver is _find_spec in Lib/importlib/_bootstrap.py (3.14.5):

def _find_spec(name, path, target=None):
    meta_path = sys.meta_path
    if meta_path is None:
        raise ImportError("sys.meta_path is None, Python is likely shutting down")
    # gh-130094: Copy sys.meta_path so that we have a consistent view of the
    # list while iterating over it.
    meta_path = list(meta_path)
    ...
    is_reload = name in sys.modules
    for finder in meta_path:
        with _ImportLockContext():
            try:
                find_spec = finder.find_spec
            except AttributeError:
                continue
            else:
                spec = find_spec(name, path, target)
        if spec is not None:
            ...
            return spec
    else:
        return None

Line by line: sys.meta_path is None is the interpreter-shutdown guard. The meta_path = list(meta_path) copy is a 3.14-era fix (gh-130094) — it snapshots the list so a finder that mutates sys.meta_path mid-iteration cannot corrupt the loop, which matters under the free-threaded build where another thread could mutate it concurrently. is_reload = name in sys.modules records whether this is a reload, which changes how a parent-import side effect is reconciled. The loop fetches each finder’s find_spec (skipping any object that lacks the attribute) and returns the first non-None spec; the for…else returns None if every finder declined.

The default sys.meta_path has exactly three entries — confirmed by running python3 -S -c "import sys; print(sys.meta_path)" on CPython 3.14.5, which yields [BuiltinImporter, FrozenImporter, PathFinder]. (Without -S, the site module may prepend a setuptools shim; that is not a CPython default.) They are: one finder for built-in modules, one for frozen modules, and the path based finder for everything on an import path (per the import reference).

Path entry finders (sys.path_hooks)

The third meta path finder, PathFinder (the path based finder), “doesn’t know how to import anything. Instead, it traverses the individual path entries, associating each of them with a path entry finder that knows how to handle that particular kind of path” (per the import reference). A path entry is one element of sys.path (or of a package’s __path__): a directory, a zip file, or any string a hook understands.

For each path entry, PathFinder needs a path entry finder. Finding one is expensive (it may involve stat() calls), so the result is cached in sys.path_importer_cache, a dict mapping path-entry string → finder (per the import reference; “despite the name, this cache actually stores finder objects”). On a cache miss, PathFinder “iterates over every callable in sys.path_hooks. Each … hook … is called with a single argument, the path entry … This callable may either return a path entry finder … or it may raise ImportError.” An ImportError means “this hook can’t handle this entry; try the next.” If no hook produces a finder, PathFinder “will store None in sys.path_importer_cache … and return None.”

The default sys.path_hooks (verified live on 3.14.5) is [zipimport.zipimporter, FileFinder.path_hook(...)] — first try to treat the entry as a zip archive, then fall back to a directory-based FileFinder. The caching logic is PathFinder._path_importer_cache in Lib/importlib/_bootstrap_external.py:

@classmethod
def _path_importer_cache(cls, path):
    if path == '':
        try:
            path = _os.getcwd()
        except (FileNotFoundError, PermissionError):
            return None      # cwd vanished — do NOT cache the failure
    try:
        finder = sys.path_importer_cache[path]
    except KeyError:
        finder = cls._path_hooks(path)       # run the hooks
        sys.path_importer_cache[path] = finder
    return finder

The empty-string entry ('') denotes the current working directory and is special-cased: it is resolved fresh via os.getcwd() on every lookup and never cached under the empty key, because the cwd can change between imports (per the import reference). Otherwise the cache is consulted; a miss runs _path_hooks (which loops over sys.path_hooks) and stores the result — including None, the negative cache that says “no finder here.”

A path entry finder implements find_spec(name, target) — note: two arguments, not three; there is no path argument because the finder is already bound to a specific path entry (per the import reference). It “returns a fully populated spec for the module. This spec will always have ‘loader’ set (with one exception)” — the exception being namespace-package portions, where the finder instead sets submodule_search_locations (see Packages and Namespace Packages).

The ModuleSpec — PEP 451

Both finder protocols return a ModuleSpec, introduced by PEP 451 (Python 3.4) to fix a concrete architectural gap: “there’s an API void between finders and loaders that causes undue complexity.” Before PEP 451, a finder’s find_module() returned a loader, and the loader’s load_module() had to re-derive everything — the module’s name, file location, whether it was a package — and set all the module’s import attributes itself, “requirements … common to all loaders and mostly … implemented in exactly the same way.” The spec consolidates that state into one object passed from finder to loader, so loaders no longer reimplement the boilerplate.

The constructor (from _bootstrap.py, 3.14.5) is ModuleSpec(name, loader, *, origin=None, loader_state=None, is_package=None). Its attributes (per PEP 451 and the importlib docs) map directly onto the module’s dunder attributes:

Spec attribute	Module attribute	Meaning
name	__name__	fully-qualified module name
loader	__loader__	the loader that will execute it
origin	__file__ (if located)	where it came from ("built-in", a file path, …)
parent (property)	__package__	containing package name
submodule_search_locations	__path__	search dirs if it is a package; None otherwise
cached (property)	__cached__	location of cached bytecode
has_location (property)	—	whether origin is a real, loadable location

The function _init_module_attrs(spec, module) in _bootstrap.py is what copies these onto the module object; loaders themselves “should not set any import-related module attributes” (per PEP 451). Every module thus gets a __spec__ attribute holding its spec — the canonical, machine-readable record of how it was imported.

The Loader Protocol — create_module and exec_module

A loader is the object in spec.loader. The modern protocol has two methods. create_module(spec) “takes one argument, the module spec, and returns the new module object to use during loading … If the method returns None, the import machinery will create the new module itself” (per the import reference; added in Python 3.4). Most loaders return None and let CPython build a plain module. exec_module(module) does the real work: “the import machinery calls importlib.abc.Loader.exec_module() … with a single argument, the module object to execute. Any value returned … is ignored.” For a Python module, exec_module runs the compiled code in module.__dict__.

The orchestration lives in module_from_spec and _load_unlocked in _bootstrap.py (3.14.5):

def module_from_spec(spec):
    module = None
    if hasattr(spec.loader, 'create_module'):
        module = spec.loader.create_module(spec)      # loader may build it
    elif hasattr(spec.loader, 'exec_module'):
        raise ImportError('loaders that define exec_module() '
                          'must also define create_module()')
    if module is None:
        module = _new_module(spec.name)               # else default module
    _init_module_attrs(spec, module)                  # copy spec → dunders
    return module

The elif enforces a rule that has hardened over releases: a loader defining exec_module must also define create_module. This was a DeprecationWarning in 3.5 and became a hard ImportError in 3.6 (per the import reference). module_from_spec either uses the loader’s custom module object or makes a default one, then stamps the spec’s attributes onto it via _init_module_attrs.

_load_unlocked then registers the module and executes it — and the ordering here is the crux of circular imports, owned by the sibling note:

def _load_unlocked(spec):
    ...
    module = module_from_spec(spec)
    spec._initializing = True
    try:
        sys.modules[spec.name] = module        # register BEFORE executing
        ...
        spec.loader.exec_module(module)        # run the body
        ...
    finally:
        spec._initializing = False
    return module

The module is placed in sys.modules before exec_module runs, with spec._initializing flagging it as half-built. Why that order enables circular imports and how it fails is the domain of sys.modules and the Module Cache and Circular Import Mechanics — this note only notes that the loader is invoked here.

The legacy load_module path

Before PEP 451, the single method load_module(fullname) did everything: create the module, register it, execute it, set its attributes. CPython 3.14 still honors it for backward compatibility — “the import machinery will use the load_module() method of loaders if it exists and the loader does not also implement exec_module()” — but it is deprecated, not removed, and “loaders should implement exec_module() instead” (per the import reference). When _load_unlocked finds a loader without exec_module, it emits an ImportWarning and falls back to _load_backward_compatible, which calls spec.loader.load_module(spec.name).

The deprecation timeline is worth pinning precisely, because two different sunsets are routinely conflated:

• load_module (loader method): ImportWarning on use since Python 3.10; still present and functional in 3.14 (per the import reference).
• find_module / find_loader (finder methods): these are the finder-side legacy methods, and they were removed entirely in Python 3.12 (per the import reference, “Changed in version 3.12: find_module() and find_loader() have been removed”). A finder written for 3.11 or earlier that defines only find_module will silently never be called in 3.12+.

So in 3.14: a finder must use find_spec; a loader may still use load_module but should use exec_module/create_module.

The Default Finders in Detail

BuiltinImporter (in _bootstrap.py) handles modules compiled into the interpreter (sys, builtins, _thread). Its find_spec asks the C layer _imp.is_builtin(fullname) and, if true, returns spec_from_loader(fullname, cls, origin="built-in"). Its create_module calls _imp.create_builtin; its exec_module calls _imp.exec_builtin. Built-in modules are never packages and have no source or code object.

FrozenImporter handles frozen modules — Python modules whose marshalled bytecode is embedded in the executable (notably importlib._bootstrap itself, which must exist before any filesystem import works). See Frozen and Built-in Modules.

PathFinder is the path based finder described above — the only one of the three that consults the filesystem (or zip files), via path entry finders. Its concrete file-based path entry finder is FileFinder, which pairs file suffixes (.py, .pyc, .so) with the matching loaders (SourceFileLoader, SourcelessFileLoader, ExtensionFileLoader).

Writing a Custom Importer

A useful, real pattern: a meta path finder that intercepts a namespace and loads modules from an in-memory source dict — the skeleton behind import hooks for encrypted, networked, or generated code.

import sys, importlib.abc, importlib.util
 
class DictLoader(importlib.abc.Loader):
    def __init__(self, sources): self.sources = sources
    def create_module(self, spec):       # required alongside exec_module
        return None                      # use the default module object
    def exec_module(self, module):
        code = compile(self.sources[module.__name__], module.__name__, "exec")
        exec(code, module.__dict__)      # run the body into the namespace
 
class DictFinder(importlib.abc.MetaPathFinder):
    def __init__(self, sources): self.sources = sources
    def find_spec(self, name, path, target=None):
        if name in self.sources:
            return importlib.util.spec_from_loader(name, DictLoader(self.sources))
        return None                      # decline → next finder tries
 
sys.meta_path.insert(0, DictFinder({"virtual_mod": "VALUE = 42"}))
import virtual_mod                       # runs DictLoader.exec_module
print(virtual_mod.VALUE)                 # → 42

Line by line: DictLoader.create_module returns None (the required companion to exec_module, per the 3.6 rule above), so CPython builds a default module object. exec_module compiles the stored source and execs it into module.__dict__ — exactly what a file loader does, minus the disk read. DictFinder.find_spec returns a spec via spec_from_loader (a helper that fills in origin, is_package, etc.) when it recognizes the name, and None otherwise so the next meta path finder gets its turn. Inserting at position 0 makes it win over BuiltinImporter/FrozenImporter/PathFinder. Running import virtual_mod triggers the full pipeline and the module ends up in sys.modules like any other. (This snippet was executed verbatim under CPython 3.14.5 with -W error::DeprecationWarning: it prints 42 and emits no deprecation warning, confirming importlib.abc.MetaPathFinder/Loader and the find_spec/exec_module signatures are current.)

Failure Modes and Common Misunderstandings

A finder that returns None for everything is silently skipped. _find_spec simply moves to the next finder; there is no error. A custom finder that never matches looks “installed but dead.” Diagnose by checking it is actually in sys.meta_path and that its find_spec signature accepts three arguments.

Defining find_module instead of find_spec on a finder is a no-op in 3.12+. Since find_module/find_loader were removed in 3.12, _find_spec will AttributeError on finder.find_spec and continue past your finder entirely. The fix is to rename to find_spec(self, name, path, target=None).

Defining exec_module without create_module raises ImportError (since 3.6), not a warning. Always pair them; return None from create_module if you want the default module object.

Stale sys.path_importer_cache. If you create a directory at runtime and try to import from it, PathFinder may have already cached None for a parent path entry. Call importlib.invalidate_caches() — which “invalidate[s] the internal caches of finders stored at sys.meta_path” by calling each finder’s invalidate_caches() (per the importlib docs); PathFinder.invalidate_caches in turn drops relative-path sys.path_importer_cache entries and refreshes FileFinder directory mtimes (per _bootstrap_external.py) — before the import.

Alternatives and When to Choose Them

For locating code, a meta path finder is the heavy hammer: it sees every import and can override even built-ins, so it suits global interception (lazy imports, instrumentation, import-from-database). A path entry finder plus a sys.path_hooks hook is the lighter tool: it only activates for path entries it claims (e.g. a custom archive format), leaving normal filesystem imports untouched — the right choice when you are adding a new kind of location rather than overriding by name. For merely transforming already-located source (e.g. a macro preprocessor), subclass SourceFileLoader and override source_to_code, reusing PathFinder for discovery.

Production Notes

The whole mechanism is itself bootstrapped from a frozen copy of importlib._bootstrap (_frozen_importlib), because the import system cannot import itself from disk before the import system exists — hence BuiltinImporter/FrozenImporter come first on sys.meta_path. Tools that rely on this protocol in the wild include pytest’s assertion-rewriting import hook (a meta path finder that recompiles test modules to give rich assert failure messages), six/future module-aliasing finders, and zipimport (a built-in path hook for running code straight out of a .zip/wheel). The 3.14 gh-130094 snapshot of sys.meta_path during iteration is a free-threading-hardening change: under Free-Threaded CPython two threads importing concurrently must not see a torn meta-path list.

See Also

• sys.modules and the Module Cache — the cache gate that runs before any finder; owns the _initializing flag and insert-before-exec ordering
• The Python Import System — the parent overview of __import__, _gcd_import, and _handle_fromlist
• Packages and Namespace Packages — how submodule_search_locations and __path__ drive package resolution
• Frozen and Built-in Modules — what BuiltinImporter and FrozenImporter actually load
• importlib Internals — the pure-Python reimplementation in _bootstrap.py / _bootstrap_external.py
• Circular Import Mechanics — the failure mode the loading phase enables
• Python Internals MOC — §11 The Import System