vault

Object Creation new init and alloc

14 min read

Object Creation: __new__, __init__, and tp_alloc

When you write Point(3, 4), Python does not “call the constructor” in one step the way C++ or Java does. It runs a three-stage pipeline: the call itself is handled by the metaclass through type.__call__ (the C slot tp_call); that dispatcher invokes __new__ (the slot tp_new) to allocate and return a brand-new instance, and then — only if __new__ returned an instance of the class being constructed — invokes __init__ (the slot tp_init) to initialize that already-existing instance in place. Underneath __new__, the raw memory comes from tp_alloc (normally PyType_GenericAlloc). __new__ is the genuine constructor; __init__ is merely an initializer that runs on an object that already exists. The whole sequence is implemented by the C function type_call in Objects/typeobject.c and specified in the data model. This note traces it as of CPython 3.14 (released 7 October 2025).

The single most important consequence: __init__ is not where an object comes into being. By the time __init__ runs, the instance already exists, already has a type, and already has an id(). That is why immutable types must do their work in __new__ — __init__ is too late to change a value that was fixed at allocation.

Mental Model: Three Slots, One Call Expression

The Python-level expression Cls(*args, **kwds) is sugar for calling a type object. Calling any object in CPython routes through that object’s type’s tp_call slot. For a normal class Cls, the type of Cls is its metaclass — by default type — so Cls(...) runs type.__call__(Cls, *args, **kwds), whose C implementation is type_call. That single function orchestrates the other two stages.

flowchart TD
    A["Cls(args, kwds)"] --> B["type.__call__  (tp_call on the metaclass)<br/>C: type_call()"]
    B --> C["Cls.__new__(Cls, args, kwds)  (tp_new)<br/>the CONSTRUCTOR: allocate + return instance"]
    C --> D{"is the result an<br/>instance of Cls?"}
    D -- no --> E["return result as-is<br/>__init__ is SKIPPED"]
    D -- yes --> F["instance.__init__(args, kwds)  (tp_init)<br/>the INITIALIZER: mutate in place, return None"]
    F --> G["return the instance"]
    C -.->|under the hood| H["object.__new__ → tp_alloc<br/>= PyType_GenericAlloc<br/>raw memory + header + refcount=1"]

Figure: the object-creation pipeline. The insight to extract is that there are three distinct, separately-overridable hooks — tp_call on the metaclass, tp_new, and tp_init — and that the is-it-an-instance-of-Cls? branch is the gate that decides whether __init__ runs at all. tp_alloc sits one level below __new__, doing the byte-level allocation.

The mnemonic that survives every edge case: __new__ makes, __init__ sets. __new__ receives the class and returns an object; __init__ receives the object (self) and returns nothing (None). They take the same trailing arguments because both are handed the original call arguments by type_call.

Mechanical Walk-Through: Inside type_call

The orchestration is short enough to read in full. In Objects/typeobject.c (CPython 3.14), type_call first handles the special case of calling type itself with one argument (type(x) returns type(x)’s type), then does the general construction:

if (type->tp_new == NULL) {
    _PyErr_Format(tstate, PyExc_TypeError,
                  "cannot create '%s' instances", type->tp_name);
    return NULL;
}
 
obj = type->tp_new(type, args, kwds);                 /* stage 2: __new__   */
obj = _Py_CheckFunctionResult(tstate, (PyObject*)type, obj, NULL);
if (obj == NULL)
    return NULL;
 
/* If the returned object is not an instance of type,
   it won't be initialized. */
if (!PyObject_TypeCheck(obj, type))                   /* the gate           */
    return obj;
 
type = Py_TYPE(obj);
if (type->tp_init != NULL) {
    int res = type->tp_init(obj, args, kwds);         /* stage 3: __init__  */
    if (res < 0) {
        assert(_PyErr_Occurred(tstate));
        Py_SETREF(obj, NULL);
    }
    else {
        assert(!_PyErr_Occurred(tstate));
    }
}
return obj;

Reading it line by line:

  • type->tp_new(type, args, kwds) — stage two. tp_new is the C slot backing __new__. It is passed the type being instantiated as its first argument (this is why __new__(cls, ...) takes cls) plus the original positional and keyword arguments. It is responsible for producing the instance and returning a new reference to it.
  • _Py_CheckFunctionResult — sanity check: a C-level function must not return NULL without setting an exception, and must not return non-NULL with an exception pending. This catches buggy extension types.
  • if (!PyObject_TypeCheck(obj, type)) return obj; — the gate. PyObject_TypeCheck(obj, type) is true when obj is an instance of type or one of its subclasses. If __new__ returned something that is not an instance of the class being constructed, type_call returns it immediately and __init__ is never called. This is the C realization of the data-model rule below.
  • type = Py_TYPE(obj); — note the re-fetch. __init__ is looked up on the actual type of the returned object, not on the type that was asked for. In normal subclass construction these are the same, but the code is written to honor whatever __new__ actually produced.
  • type->tp_init(obj, args, kwds) — stage three. tp_init (backing __init__) receives the freshly created obj as self and the same original arguments. It returns an int: 0 for success, -1 for failure (with an exception set). It does not return the object — __init__’s Python-level return value is required to be None and is discarded here.
  • Py_SETREF(obj, NULL) — if __init__ failed, the half-built object is dropped and the error propagates.

The function then returns the constructed (and possibly initialized) object. That object is what Cls(...) evaluates to.

Why __init__ returning non-None is a TypeError

type_call ignores tp_init’s payload entirely — it only reads the success/failure int. The “must return None” rule is enforced not here but inside the wrapper that bridges a Python def __init__ to the C tp_init slot (slot_tp_init): it checks that the Python return value is Py_None and raises TypeError: __init__() should return None otherwise. The data model states the rule plainly: “no non-None value may be returned by __init__(); doing so will cause a TypeError to be raised at runtime” (data model).

The Data-Model Rules (the authoritative contract)

The language reference specifies the behavior that type_call implements. The non-obvious rules, each quoted from the docs:

__new__ is a staticmethod, implicitly. “__new__() is a static method (special-cased so you need not declare it as such) that takes the class of which an instance was requested as its first argument.” You write def __new__(cls, *args) without @staticmethod, yet it behaves as one: calling Cls.__new__(Cls) does not bind Cls as self. This special-casing happens when the class is built — CPython wraps a plain-function __new__ as a static method automatically.

The __init__-gating rule. “If __new__() is invoked during object construction and it returns an instance of cls, then the new instance’s __init__() method will be invoked… If __new__() does not return an instance of cls, then the new instance’s __init__() method will not be invoked.” This is exactly the PyObject_TypeCheck gate above.

__init__ returns nothing. “Because __new__() and __init__() work together in constructing objects (__new__() to create it, and __init__() to customize it), no non-None value may be returned by __init__().”

__new__’s intended audience. “__new__() is intended mainly to allow subclasses of immutable types (like int, str, or tuple) to customize instance creation. It is also commonly overridden in custom metaclasses in order to customize class creation.” Outside those two cases, overriding __new__ is rarely necessary — __init__ suffices for mutable objects.

The recommended __new__ body, per the docs: “create a new instance of the class by invoking the superclass’s __new__() method using super().__new__(cls[, ...]) with appropriate arguments and then modifying the newly created instance as necessary before returning it.” You delegate the actual allocation upward to object.__new__ (or int.__new__, etc.), then customize.

tp_alloc: Where the Bytes Come From

__new__ does not itself touch raw memory in the common case. object.__new__ calls the type’s tp_alloc slot, which for almost every Python-defined class is PyType_GenericAlloc. From typeobject.c:

PyObject *
PyType_GenericAlloc(PyTypeObject *type, Py_ssize_t nitems)
{
    PyObject *obj = _PyType_AllocNoTrack(type, nitems);
    if (obj == NULL) {
        return NULL;
    }
    if (_PyType_IS_GC(type)) {
        _PyObject_GC_TRACK(obj);
    }
    return obj;
}

_PyType_AllocNoTrack computes the object’s size from tp_basicsize (the fixed part) plus nitems * tp_itemsize (the variable part, used by variable-length types such as tuple, int, and bytes), asks the allocator for a zero-filled block, writes the type pointer and an initial reference count into the object header, and returns it. If the type is tracked by the cyclic garbage collector (Py_TPFLAGS_HAVE_GC), _PyObject_GC_TRACK then registers it so cycles through it can later be collected. See PyObject and the Object Header for the header layout and Reference Counting for what that initial count means.

And PyType_GenericNew, the default tp_new for a plain class, is a one-liner — it simply forwards to tp_alloc with zero extra items:

PyObject *
PyType_GenericNew(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    return type->tp_alloc(type, 0);
}

So the call chain for a vanilla class with no custom __new__ is: type_call → tp_new (= object.__new__, which calls tp_alloc) → PyType_GenericAlloc → _PyType_AllocNoTrack → the pymalloc allocator. The C-API documentation is explicit that you should reach tp_alloc through these helpers rather than calling the low-level PyObject_New macro yourself: “When populating a type’s tp_alloc slot, PyType_GenericAlloc() is preferred over a custom function” (c-api/allocation).

PyObject_Init is not __init__

The C function PyObject_Init(op, type) initializes the object header (type pointer + refcount) of freshly allocated memory. The docs warn: “Despite its name, this function is unrelated to the object’s __init__() method (tp_init slot). Specifically, this function does not call the object’s __init__() method.” Header-level “init” and Python-level __init__ are entirely different things — tp_init runs much later, in stage three.

object.__new__ and object.__init__: the Excess-Argument Dance

The base implementations of both slots live in typeobject.c as object_new and object_init. They share one quirky job: deciding whether extra constructor arguments are an error. The shared helper is trivially small:

static int
excess_args(PyObject *args, PyObject *kwds)
{
    return PyTuple_GET_SIZE(args) ||
        (kwds && PyDict_Check(kwds) && PyDict_GET_SIZE(kwds));
}

It returns true if any positional or keyword arguments were passed. The subtlety is what each base method does when that happens, because plain object accepts no arguments — yet a subclass that overrides only one of the two methods legitimately should receive arguments. The logic in object_init:

static int
object_init(PyObject *self, PyObject *args, PyObject *kwds)
{
    PyTypeObject *type = Py_TYPE(self);
    if (excess_args(args, kwds)) {
        if (type->tp_init != object_init) {
            PyErr_SetString(PyExc_TypeError,
                "object.__init__() takes exactly one argument (the instance to initialize)");
            return -1;
        }
        if (type->tp_new == object_new) {
            PyErr_Format(PyExc_TypeError,
                "%.200s.__init__() takes exactly one argument (the instance to initialize)",
                type->tp_name);
            return -1;
        }
    }
    return 0;
}

And the mirror image in object_new:

static PyObject *
object_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
    if (excess_args(args, kwds)) {
        if (type->tp_new != object_new) {
            PyErr_SetString(PyExc_TypeError,
                "object.__new__() takes exactly one argument (the type to instantiate)");
            return NULL;
        }
        if (type->tp_init == object_init) {
            PyErr_Format(PyExc_TypeError, "%.200s() takes no arguments",
                         type->tp_name);
            return NULL;
        }
    }
    /* ... abstract-class check, then tp_alloc ... */
}

The cross-checks encode a precise policy. Consider what happens when you call MyClass(1, 2, 3) and MyClass overrides neither __new__ nor __init__: both slots are still object’s, both see the excess args, and object_new’s second branch fires — TypeError: MyClass() takes no arguments. But if MyClass overrides __init__ (so tp_init != object_init) while leaving __new__ as object’s, then object_new sees the arguments, notices __init__ is not the base one, and stays silent — it lets the arguments through, trusting the overridden __init__ to consume them. Symmetrically, object_init keeps quiet when __new__ is overridden. This is why a class that overrides __init__ can take constructor arguments even though it never touches __new__, and why overriding both but ignoring the args raises a complaint only when neither is overridden. After the argument checks, object_new rejects instantiation of abstract base classes (Py_TPFLAGS_IS_ABSTRACT, set when a class still has unimplemented @abstractmethods — see Abstract Base Classes) with Can't instantiate abstract class ... without an implementation for abstract method ..., and finally calls tp_alloc to produce the instance.

Worked Examples

A mutable class: __init__ is enough

class Point:
    def __init__(self, x, y):
        self.x = x          # self already exists; we mutate it
        self.y = y
 
p = Point(3, 4)

Point does not override __new__, so type_call runs object.__new__(Point) (which allocates via PyType_GenericAlloc), the gate passes (the result is a Point), and Point.__init__(p, 3, 4) runs to set the attributes. Nothing surprising — this is 99% of real classes.

An immutable subclass: work must happen in __new__

class PositiveInt(int):
    def __new__(cls, value):
        if value < 0:
            raise ValueError("must be non-negative")
        return super().__new__(cls, value)   # int.__new__ fixes the value NOW
 
x = PositiveInt(7)
print(x + 1)        # 8 — behaves as an int

int is immutable: its integer value is baked in at allocation by int.__new__ and can never change afterward. There is no writable “value” attribute for an __init__ to set. If you tried to do the validation/assignment in __init__, it would be too late — the object’s value is already frozen. This is exactly the “subclasses of immutable types customize instance creation in __new__” case the docs call out. The same applies to str, bytes, tuple, float, frozenset, and complex. See CPython Integer Internals and CPython String Internals for why these values are immutable at the storage level.

Returning a different object: skip __init__ entirely

class Singleton:
    _instance = None
    def __new__(cls):
        if cls._instance is None:
            cls._instance = super().__new__(cls)
        return cls._instance
    def __init__(self):
        print("init runs")
 
a = Singleton()   # prints "init runs"
b = Singleton()   # ALSO prints "init runs" — __init__ runs every time!
print(a is b)     # True

This is the classic trap. Because __new__ returns an instance of Singleton both times, the PyObject_TypeCheck gate passes both times, so __init__ runs on every call — even though the same object is reused. If __init__ does expensive or side-effecting setup, it re-runs on the cached instance. (The cure is to guard __init__, or to do setup in __new__.) Contrast with a __new__ that returns a foreign object:

class Weird:
    def __new__(cls):
        return 42        # NOT an instance of Weird
    def __init__(self):
        print("never runs")
 
print(Weird())   # 42 — __init__ is skipped because the gate fails

Here the gate if (!PyObject_TypeCheck(obj, type)) return obj; short-circuits, and Weird() evaluates to the plain integer 42.

Failure Modes and Common Misunderstandings

  • “__init__ is the constructor.” No — __new__ is. __init__ initializes an object that already exists, already has an id(), and is already wired into the GC. Treating __init__ as the place where the object is “made” leads to confusion the moment you subclass an immutable type or write a singleton/cache.
  • __init__ re-running on cached instances. As the singleton example shows, returning an already-constructed instance from __new__ does not suppress __init__; the gate only checks type, not novelty. Guard with an instance flag if re-initialization is harmful.
  • Mismatched signatures. __new__ and __init__ receive the same arguments. If __new__ consumes some via its own parameters but __init__ is also defined to accept them (or vice versa), you must keep both signatures compatible, or one will raise a TypeError about unexpected arguments. The base-class excess-args policy above is the reason a lone overridden __init__ “just works.”
  • Forgetting super().__new__(cls). A custom __new__ that returns a hand-built object without delegating to a base __new__ skips tp_alloc and the header setup. For Python-level classes you almost always want super().__new__(cls) (or int.__new__(cls, v) for an immutable base) so allocation, the object header, and GC tracking happen correctly.
  • Passing args to object.__new__ directly. object.__new__(MyClass, 1, 2) raises TypeError: object.__new__() takes exactly one argument (the type to instantiate) — the first branch of object_new. The base allocator takes only the class.
  • Metaclass confusion. Because type_call is type.__call__, a metaclass can override __call__ to intercept all construction of its instances (the classes) — a more powerful hook than __new__. See Metaclasses.

See Also

Graph View

Backlinks