vault

Attribute Lookup Mechanics

14 min read

Attribute Lookup Mechanics

Every obj.attr in Python triggers a precise, ordered search implemented in C, not a single dictionary lookup. The dot operator calls the type’s tp_getattro slot, which for almost every object is PyObject_GenericGetAttr → _PyObject_GenericGetAttrWithDict in Objects/object.c. That function walks a fixed precedence: (1) the type’s Method Resolution Order is searched for a data descriptor (one defining __set__ or __delete__) — if found it wins outright; (2) the instance’s own attributes (inline values or __dict__) are checked; (3) failing that, a non-data descriptor (only __get__, e.g. a plain function) or a plain class attribute is used; (4) if nothing matched and the class defines __getattr__, that fallback is called. This ordering — data descriptor, instance dict, non-data descriptor, class attribute, __getattr__ — is the single rule that explains how methods bind, why property overrides an instance attribute, and why __slots__ works. Verified against CPython 3.14.5 source and the descriptor HowTo.

The crucial insight is that the instance dict is only the second stop, not the first — a data descriptor on the class beats anything in the instance, while a non-data descriptor (a function) loses to the instance dict. This asymmetry is what lets a method live on the class yet be shadowed by an instance attribute of the same name, while a property cannot be shadowed. Note that step (2) — “look in the instance’s attributes” — is itself a dict probe (or the inline-values fast path that precedes it); this note owns the resolution order, while CPython Dict Internals owns the probe mechanics that step relies on.

Mental Model — A Five-Stop Search Down the Class and Up Again

Think of attribute lookup as a search with a strict priority list. The interpreter first consults the class (and its bases, in MRO order) to see whether the attribute name is governed by a descriptor — an object with __get__/__set__/__delete__ that intercepts access. Whether that descriptor wins immediately, or only as a last resort, depends on whether it is a data descriptor (defines __set__ or __delete__) or a non-data descriptor (defines only __get__). Between those two checks, the instance’s own storage is consulted.

flowchart TD
    start["obj.attr → tp_getattro → _PyObject_GenericGetAttrWithDict"]
    mro["_PyType_Lookup: walk type(obj)'s MRO, find 'attr' on a class<br/>(served from the method cache when warm)"]
    isdata{"Found a descriptor<br/>with __set__/__delete__?<br/>(data descriptor)"}
    inst["Check instance: inline values, else obj.__dict__<br/>(this is the dict probe → [[CPython Dict Internals]])"]
    found_inst{"Present in instance?"}
    isnondata{"Class attr is a<br/>non-data descriptor<br/>(only __get__)?"}
    plain{"Plain class attribute<br/>was found?"}
    getattr["AttributeError → dot operator calls __getattr__ if defined"]

    start --> mro --> isdata
    isdata -- yes --> data_win["call descr.__get__(obj, type) — WINS"]
    isdata -- no --> inst --> found_inst
    found_inst -- yes --> inst_win["return instance value — WINS"]
    found_inst -- no --> isnondata
    isnondata -- yes --> nondata_win["call descr.__get__(obj, type)<br/>e.g. function → bound method"]
    isnondata -- no --> plain
    plain -- yes --> classvar["return the class attribute"]
    plain -- no --> getattr

Figure: the precedence inside _PyObject_GenericGetAttrWithDict. Insight: the MRO is searched first*, but the result is only used immediately if it is a* data descriptor; otherwise the instance is checked before the (non-data descriptor or plain) class attribute is used. __getattr__ is not part of this function at all — it is invoked by the dot operator only after this returns AttributeError.

Mechanical Walk-Through — _PyObject_GenericGetAttrWithDict

The dispatch begins at the C level. obj.attr compiles to a LOAD_ATTR bytecode whose handler calls PyObject_GetAttr, which invokes Py_TYPE(obj)->tp_getattro. For the vast majority of objects that slot is PyObject_GenericGetAttr, a thin wrapper over _PyObject_GenericGetAttrWithDict(obj, name, NULL, 0). Here is the actual control flow from object.c, traced step by step.

Step 0 — find the name on the type. The function calls _PyType_LookupStackRefAndVersion(tp, name, &cref.ref) to search the type’s MRO for name, yielding a borrowed descr (the class-level object bound to that name, or NULL). This single call is the MRO walk plus the method cache (described below). It also reads descr’s tp_descr_get into a local f:

descr = ... // result of _PyType_Lookup over the MRO
f = NULL;
if (descr != NULL) {
    f = Py_TYPE(descr)->tp_descr_get;     // does the descr define __get__?
    if (f != NULL && PyDescr_IsData(descr)) {   // ... and __set__/__delete__?
        res = f(descr, obj, (PyObject *)Py_TYPE(obj));   // DATA DESCRIPTOR WINS
        goto done;
    }
}

PyDescr_IsData(descr) is defined in descrobject.c as exactly Py_TYPE(ob)->tp_descr_set != NULL — i.e. “does the descriptor’s type define __set__ (or __delete__, which also populates tp_descr_set)?” If the class attribute is a data descriptor and it has a __get__, it short-circuits the entire rest of the search: its __get__(obj, type) is called and returned. This is step (1), and it is why a property (a data descriptor) on the class cannot be overridden by an instance attribute of the same name.

Step 2 — the instance’s own storage. If no data descriptor won, the function looks in the instance. In modern CPython this is not unconditionally a dict probe; it tries the inline-values fast path first:

if (dict == NULL) {
    if (tp->tp_flags & Py_TPFLAGS_INLINE_VALUES) {
        if (PyUnicode_CheckExact(name) &&
            _PyObject_TryGetInstanceAttribute(obj, name, &res)) {
            if (res != NULL) goto done;   // found in inline values
        } else {
            dict = (PyObject *)_PyObject_MaterializeManagedDict(obj);
        }
    } else if (tp->tp_flags & Py_TPFLAGS_MANAGED_DICT) {
        dict = (PyObject *)_PyObject_GetManagedDict(obj);
    } else {
        PyObject **dictptr = _PyObject_ComputedDictPointer(obj);
        if (dictptr) dict = *dictptr;
    }
}
if (dict != NULL) {
    int rc = PyDict_GetItemRef(dict, name, &res);    // THE DICT PROBE
    if (res != NULL) goto done;                       // INSTANCE WINS
}

_PyObject_TryGetInstanceAttribute consults the object’s inline values array against the type’s shared keys — no per-instance dict is allocated unless one is needed. If the object has a real __dict__ (a managed dict, or a tp_dictoffset dict), the search becomes a literal PyDict_GetItemRef — the open-addressing probe described in CPython Dict Internals. Either way, step (2) returns the instance’s own value if the name is present there.

Step 3 — non-data descriptor, then plain class attribute. If the instance had nothing, control returns to the descr found in step 0:

if (f != NULL) {                          // descr had __get__ (non-data)
    res = f(descr, obj, (PyObject *)Py_TYPE(obj));   // e.g. function → bound method
    goto done;
}
if (descr != NULL) {                      // plain class attribute, no __get__
    res = ...steal(descr);
    goto done;
}

This is step (3): a non-data descriptor’s __get__ is now invoked (this is exactly how a plain function becomes a bound method — functions are non-data descriptors), or, if descr had no __get__ at all, the class attribute is returned as-is.

Step 4 — failure and __getattr__. If nothing matched, _PyObject_GenericGetAttrWithDict raises AttributeError (unless suppress was requested). Critically, __getattr__ is not called here. The descriptor HowTo is explicit: “there is no __getattr__() hook in the __getattribute__() code… it is the dot operator and the getattr() function that are responsible for invoking __getattr__() whenever __getattribute__() raises an AttributeError.” The mechanism is in typeobject.c: when a class defines __getattr__, its tp_getattro is set to _Py_slot_tp_getattr_hook, which first runs the generic getattr and only on a suppressed AttributeError calls __getattr__. So __getattr__ is a true fallback — it fires only for names the normal search could not find, which is why it is cheap for present attributes and the right hook for lazy/computed attributes.

Why Data vs Non-Data Descriptor Is the Whole Game

The single asymmetry that the precedence enforces is: data descriptors beat the instance dict; non-data descriptors lose to it. Walking the consequences makes the rule concrete (all verified empirically on 3.14.5):

class Data:                       # data descriptor: has __set__
    def __get__(self, o, t): return "descr-get"
    def __set__(self, o, v): pass
class NonData:                    # non-data descriptor: only __get__
    def __get__(self, o, t): return "nondata-get"
 
class T:
    d = Data()
    nd = NonData()
 
t = T()
t.__dict__['d']  = "instance-d"   # try to shadow the data descriptor
t.__dict__['nd'] = "instance-nd"  # try to shadow the non-data descriptor
 
t.d   # -> "descr-get"   : data descriptor WINS over the instance dict (step 1)
t.nd  # -> "instance-nd" : instance dict WINS over the non-data descriptor (step 2 before 3)

This is precisely why:

  • property, classmethod, staticmethod, and slot descriptors (__slots__ members) are data descriptors and cannot be shadowed by an instance attribute — they all define __set__ (even staticmethod/classmethod route through a data-descriptor protocol). A property getter always runs, even if obj.__dict__ happens to contain a key of the same name.
  • Plain functions are non-data descriptors (they define only __get__, which returns a bound method). That is why you can shadow a method by assigning obj.method = something — the instance dict (step 2) is consulted before the function’s __get__ (step 3). The function still lives on the class; the instance attribute simply wins.

The pure-Python equivalent of the whole algorithm appears in the descriptor HowTo as object_getattribute, which mirrors the C precedence exactly: data descriptor → vars(obj) (instance) → non-data descriptor → class variable → AttributeError.

__slots__ Short-Circuits the Instance Dict

A class with __slots__ replaces the per-instance __dict__ with a fixed set of C-level storage slots. The mechanism plugs straight into the precedence above: for each name in __slots__, CPython creates a member_descriptor object on the class. A member_descriptor defines both __get__ and __set__, so by PyDescr_IsData’s test it is a data descriptor — it wins at step (1) of the search. Reading obj.x for a slotted attribute therefore never reaches the instance-dict step; it calls the member descriptor’s __get__, which reads a value directly from a fixed offset in the object’s C structure. And because no tp_dictoffset dict is created for a fully-slotted class, step (2)‘s dict probe is absent entirely (__dict__ raises AttributeError). This is the source of both __slots__’s memory savings (no dict per instance) and its restriction (you cannot set undeclared attributes — there is no dict to hold them). The full treatment, including inheritance subtleties and the __weakref__ slot, is in The slots Optimization.

The Method Cache — Making the MRO Walk Free

Step 0 (_PyType_Lookup) must, in principle, walk the entire MRO and do a dict lookup in each class’s tp_dict (find_name_in_mro in typeobject.c does exactly this — note that each step of the MRO walk is a dict probe into a type’s namespace, reinforcing that attribute lookup is dict-probing all the way down, per CPython Dict Internals). For a deep class hierarchy that would be expensive on every single attribute access, so CPython interposes a type attribute cache (the “method cache”).

The cache is a global open-addressed hash table of 4096 entries (MCACHE_SIZE_EXP = 12, so 1 << 12), defined in pycore_interp_structs.h:

struct type_cache_entry {
    unsigned int version;   // snapshot of type->tp_version_tag
    PyObject *name;         // the attribute name (exact str or None)
    PyObject *value;        // the resolved attribute (borrowed)
};
#define MCACHE_SIZE_EXP 12  // 4096 entries

Each entry is keyed by a hash of (type->tp_version_tag, name):

#define MCACHE_HASH(version, name_hash) \
    (((unsigned int)(version) ^ (unsigned int)(name_hash)) & ((1 << MCACHE_SIZE_EXP) - 1))

On lookup, CPython computes this slot and checks whether the cached entry’s version matches the type’s current tp_version_tag and the name matches. On a hit, it returns the cached value without touching the MRO at all — turning a multi-class dict-walk into a couple of comparisons. This cache is what makes Python’s method dispatch competitive despite its dynamism, and it is the foundation the specializing interpreter’s LOAD_ATTR inline caches build on.

Correctness depends entirely on cache invalidation. Every type carries a tp_version_tag (a uint32_t). Each cache entry remembers the version that was current when it was filled. The instant a type’s namespace changes — a method is reassigned, a base class is altered, __bases__ is reassigned — CPython resets that type’s tp_version_tag to 0 (and propagates to subclasses). A tp_version_tag of 0 is treated as “never matches”, so every stale cache entry for that type is automatically invalidated: the next lookup misses, walks the MRO, and re-fills the entry with a freshly assigned version. This versioning is why the cache can be correct without ever being explicitly cleared on mutation — the version mismatch is the invalidation. (A type whose tp_version_tag is 0 simply isn’t cacheable until assign_version_tag gives it a new nonzero version on the next lookup.)

Failure Modes and Common Misunderstandings

  • “Methods are looked up in the instance first.” No — the MRO is searched first (step 0). Methods are non-data descriptors, so they lose to a same-named instance attribute (step 2 before step 3), but the function object itself is found on the class, not the instance.
  • property won’t go away. Because a property is a data descriptor (step 1), assigning obj.x = v does not create an instance attribute that shadows it — it calls the property’s setter (or raises AttributeError if read-only). A frequent surprise for those who expect the instance dict to win.
  • __getattr__ vs __getattribute__. __getattribute__ runs for every attribute access and is the entry point above; overriding it (carelessly) breaks all attribute access. __getattr__ runs only when the normal search fails. Calling super().__getattribute__(name) or object.__getattribute__(obj, name) directly bypasses __getattr__ entirely, because the hook lives in the dot operator, not in the generic getattr.
  • Class attribute access uses a different path. SomeClass.attr (attribute access on a type, not an instance) does not go through _PyObject_GenericGetAttrWithDict. It uses the type’s own tp_getattro = _Py_type_getattro, which is metatype-aware: it searches the metaclass’s MRO for a data descriptor, then the type’s own MRO, then the metaclass for non-data/plain attributes. This is why a descriptor placed on a metaclass governs access to class attributes. The metaclass machinery is covered in Metaclasses and The type and object Relationship.
  • A stale value after monkeypatching is almost never the cache. The version-tag invalidation is robust; if you reassign Cls.method = newfn, the cache is invalidated and the next call sees newfn. Apparent staleness usually comes from a bound method already captured into a variable, not from the lookup cache.

Alternatives and Customization Hooks

The generic algorithm above is the default, but it is fully overridable. Defining __getattribute__ replaces step 0–3 wholesale (and is how proxy objects intercept everything). Defining __getattr__ adds only the step-4 fallback (the common, safe choice for lazy attributes). Setting tp_getattro directly in a C extension bypasses the Python protocol entirely. For setting attributes, the symmetric path is __setattr__ → tp_setattro → _PyObject_GenericSetAttrWithDict, which mirrors the precedence: a data descriptor’s __set__ wins, otherwise the value goes into the instance’s inline values / __dict__. For deletion, __delete__/__delattr__ play the analogous role. The descriptor protocol that all of this rests on is the subject of Descriptors, and the MRO that step 0 walks is built by the C3 algorithm described in Method Resolution Order and C3 Linearization.

Production Notes

The data-vs-non-data-descriptor precedence is not an obscure internal — it is the load-bearing rule behind the entire object system: @property, @cached_property (a non-data descriptor, deliberately, so it can cache into the instance dict and be served from there on subsequent accesses), @classmethod/@staticmethod, ORM column descriptors (SQLAlchemy, Django models), and __slots__ all derive their behavior from where they sit in this ordering. The method cache, introduced long ago and continuously refined, is one of the reasons Python’s attribute access — despite resolving through a fully dynamic MRO on every access — is fast enough to be invisible in profiles; the specializing interpreter (3.11+) layers per-call-site inline caches on top of it so that a monomorphic obj.attr site can skip even the global cache lookup. Understanding this path is the prerequisite for diagnosing the “why did my descriptor not fire / why can’t I override this attribute” class of bugs that otherwise look like Python being arbitrary.

See Also

Graph View

Backlinks