vault

CPython Float Internals

13 min read

CPython Float Internals

A Python float is the thinnest possible wrapper around a hardware double: the C struct PyFloatObject is just the universal object header plus a single field, double ob_fval (floatobject.h). All of Python’s floating-point behavior — the precision, the rounding, the 0.1 + 0.2 != 0.3 surprise, the existence of nan and inf and a negative zero — is inherited directly from the IEEE 754 double-precision format and the platform’s floating-point unit. CPython adds three things on top of the raw double: a per-interpreter free list that recycles float objects to avoid allocator churn (pycore_freelist.h), a shortest-round-trip repr built on David Gay’s dtoa so that repr(x) always reads back to the exact same double (pystrtod.c), and a hex format (float.hex) for lossless exchange. This note covers the box and its formatting; the user-facing pitfalls of floating-point math are collected in Floating Point Pitfalls in Python.

Mental Model

Think of a Python float as a labelled envelope around a single 64-bit IEEE 754 value. The envelope is the PyObject machinery from PyObject and the Object Header — a reference count and a type pointer — and the contents are one C double. Nothing about the number is Python’s invention: the bit pattern, the rounding to nearest-even, the value of 0.1, and the behaviour of inf/nan are all defined by the IEEE 754 standard and executed by the CPU’s floating-point unit. CPython’s only real cleverness is at the boundary — turning a double into the shortest decimal string that round-trips, and recycling the envelopes through a free list so that a tight loop creating floats does not hammer the memory allocator.

flowchart TD
    A["Python float (PyFloatObject)"] --> B["PyObject_HEAD<br/>ob_refcnt, ob_type"]
    A --> C["double ob_fval<br/>(one IEEE 754 binary64)"]
    C --> D["bit 63: sign (1 bit)"]
    C --> E["bits 62-52: exponent (11 bits, bias 1023)"]
    C --> F["bits 51-0: mantissa/fraction (52 bits,<br/>+ 1 implicit leading bit = 53 significant)"]
    G["PyFloat_FromDouble"] -->|"pop a recycled box<br/>(free list, max 100)"| A
    A -->|"float_dealloc"| H["push box back on free list<br/>or PyObject_Free if full"]
    A -->|"repr()"| I["dtoa mode 0:<br/>shortest decimal that<br/>rounds back to ob_fval"]

Diagram: a PyFloatObject is a header plus one double, and the double is exactly the IEEE 754 binary64 bit layout. The insight to extract is that Python adds no numeric semantics of its own — its contributions are the free-list recycling on the allocation path and the shortest-round-trip repr on the printing path.

The object layout

The entire struct (floatobject.h):

typedef struct {
    PyObject_HEAD
    double ob_fval;
} PyFloatObject;
 
static inline double PyFloat_AS_DOUBLE(PyObject *op) {
    return _PyFloat_CAST(op)->ob_fval;
}

Line by line:

  • PyObject_HEAD — the universal header: ob_refcnt (reference count) and ob_type (pointer to PyFloat_Type). See PyObject and the Object Header. 16 bytes on a 64-bit build.
  • double ob_fval — the value, a native C double (IEEE 754 binary64). 8 bytes.
  • PyFloat_AS_DOUBLE — an unchecked inline accessor that just returns ob_fval; it “trades safety for speed” by not verifying the argument is actually a float.

That gives a total object size of 24 bytes on 64-bit CPython, which sys.getsizeof confirms on CPython 3.14.5 (Fedora, 64-bit):

>>> import sys
>>> sys.getsizeof(0.0)
24

Unlike int (see CPython Integer Internals), the float object never grows — every float, from 0.0 to 1.7e308, is exactly 24 bytes, because the double is fixed-width.

IEEE 754 binary64, bit by bit

A C double is the IEEE 754 double-precision (“binary64”) format: 64 bits split into three fields, most-significant first.

FieldBitsWidthMeaning
Sign s6310 = positive, 1 = negative
Exponent e62–5211biased exponent, bias 1023
Fraction f51–052the mantissa’s fractional bits

The value of a normal number is reconstructed as:

value = (−1)^s × 1.f × 2^(e − 1023)

Walking every symbol:

  • (−1)^s — the sign: s = 0 gives +1, s = 1 gives −1. Because the sign is one isolated bit, +0.0 and −0.0 are distinct bit patterns even though they compare equal.
  • 1.f — the significand: there is an implicit leading 1 (the “hidden bit”) for normal numbers, so the stored 52 fraction bits actually represent 53 bits of precision. f is interpreted as the binary fraction after the implied 1.. 53 bits of precision is about log₁₀(2⁵³) ≈ 15.95 decimal digits — which is exactly why sys.float_info.mant_dig is 53 and sys.float_info.dig is 15.
  • 2^(e − 1023) — the scale. e is biased: the raw 11-bit field has 1023 subtracted from it to yield the actual exponent. So e = 1023 means 2⁰, e = 1024 means 2¹, and so on. The bias lets the exponent be stored as an unsigned field while representing both positive and negative powers.

The extreme exponent values are reserved for special encodings:

  • e = 0 (all-zero exponent): subnormal numbers (no implicit leading 1, value (−1)^s × 0.f × 2^(−1022)), allowing gradual underflow toward zero; and, with f = 0, signed zero (+0.0 / −0.0).
  • e = 2047 (all-one exponent): with f = 0, infinity (+inf / −inf); with f ≠ 0, NaN (Not a Number).

These ranges are observable through sys.float_info, measured live on CPython 3.14.5:

>>> import sys; sys.float_info
sys.float_info(max=1.7976931348623157e+308, max_exp=1024, max_10_exp=308,
               min=2.2250738585072014e-308, min_exp=-1021, min_10_exp=-307,
               dig=15, mant_dig=53, epsilon=2.220446049250313e-16,
               radix=2, rounds=1)

mant_dig=53 is the 53-bit significand; radix=2 is the binary base; epsilon (the gap between 1.0 and the next representable double, 2⁻⁵²) is the fundamental rounding granularity; rounds=1 means round-to-nearest. CPython exposes these from the platform float.h rather than defining them (Python sys docs).

The load-bearing layout values are corroborated live on CPython 3.14.5: sys.float_info reports mant_dig=53 (52 stored fraction bits + 1 implicit leading bit), radix=2, max_exp=1024 and min_exp=-1021 — exactly the values the 1/11/52 split with bias-1023 reconstruction predicts. The 1/11/52 field widths and the reserved-exponent (e=0 subnormal/zero, e=2047 inf/NaN) encodings are the standard IEEE 754-2019 binary64 definition; the standard text itself is paywalled and was not fetched, but it is not load-bearing here since sys.float_info settles every value the prose relies on.

Allocation and the free list

Creating a float goes through PyFloat_FromDouble (floatobject.c):

PyObject *
PyFloat_FromDouble(double fval)
{
    PyFloatObject *op = _Py_FREELIST_POP(PyFloatObject, floats);
    if (op == NULL) {
        op = PyObject_Malloc(sizeof(PyFloatObject));
        if (!op) {
            return PyErr_NoMemory();
        }
        _PyObject_Init((PyObject*)op, &PyFloat_Type);
    }
    op->ob_fval = fval;
    return (PyObject *) op;
}
  • _Py_FREELIST_POP(PyFloatObject, floats) first tries to pop a previously-freed float box from the free list. If one is available, no allocation happens — the recycled memory is reused.
  • Only on a free-list miss (op == NULL) does it call PyObject_Malloc and initialize the header with _PyObject_Init.
  • Either way it stores fval into ob_fval. The free list is the reason a loop computing millions of intermediate floats does not generate millions of malloc/free calls.

Deallocation is the mirror image:

void
_PyFloat_ExactDealloc(PyObject *obj)
{
    assert(PyFloat_CheckExact(obj));
    _Py_FREELIST_FREE(floats, obj, PyObject_Free);
}
 
static void
float_dealloc(PyObject *op)
{
    if (PyFloat_CheckExact(op))
        _PyFloat_ExactDealloc(op);
    else
        Py_TYPE(op)->tp_free(op);
}
  • _Py_FREELIST_FREE(floats, obj, PyObject_Free) pushes the dead box onto the free list, unless the list is full — in which case it calls PyObject_Free to actually release the memory. Subclasses of float skip the free list and use their type’s tp_free.

Where the free list lives (3.14 consolidation)

Earlier CPython kept per-type free lists in ad-hoc global or thread-local structures. In modern CPython all the small-object free lists are consolidated into a single _Py_freelists struct, and the float list is one member of it (pycore_freelist_state.h):

#  define Py_floats_MAXFREELIST 100
...
struct _Py_freelist {
    void *freelist;       // entries linked through their first word
    Py_ssize_t size;      // count, or -1 if disabled
};
 
struct _Py_freelists {
    struct _Py_freelist floats;
    struct _Py_freelist ints;
    ...
};
  • Py_floats_MAXFREELIST is 100: at most 100 float boxes are cached; beyond that, freed floats are returned to the allocator.
  • A _Py_freelist is an intrusive singly-linked list — freed objects are chained through their own first word (which overlaps ob_refcnt/ob_tid), so the list needs no extra storage.

The critical detail is where this struct is anchored (pycore_freelist.h):

static inline struct _Py_freelists *
_Py_freelists_GET(void)
{
    PyThreadState *tstate = _PyThreadState_GET();
#ifdef Py_GIL_DISABLED
    return &((_PyThreadStateImpl*)tstate)->freelists;
#else
    return &tstate->interp->object_state.freelists;
#endif
}
  • In the default (GIL) build, the float free list lives in per-interpreter state (tstate->interp->object_state.freelists). The GIL serializes access, so a single per-interpreter list is safe and sub-interpreters get their own.
  • In the free-threaded build (Py_GIL_DISABLED), it lives in per-thread state, because without the GIL a shared list would need locking on every alloc/free; giving each thread its own list keeps the fast path lock-free.

Terminology drift

Older write-ups describe “the float free list” as a fixed global array (PyFloat_MAXFREELIST ≈ 100). That count is unchanged, but the location has moved into per-interpreter (or per-thread, free-threaded) state as part of the broader free-list consolidation. Statements that the free list is a single process-global block are outdated for CPython 3.14.

Formatting: shortest round-trip repr

When you print a float, CPython does not dump 17 digits and hope. float_repr calls PyOS_double_to_string with the 'r' format code (floatobject.c):

static PyObject *
float_repr(PyObject *op)
{
    PyFloatObject *v = _PyFloat_CAST(op);
    char *buf = PyOS_double_to_string(PyFloat_AS_DOUBLE(v),
                                'r', 0, Py_DTSF_ADD_DOT_0, NULL);
    ...
}

The 'r' code maps to dtoa mode 0 (pystrtod.c):

    /* repr format */
    case 'r':
        mode = 0;
        ...
    digits = _Py_dg_dtoa(d, mode, precision, &decpt_as_int, &sign, &digits_end);

_Py_dg_dtoa is David M. Gay’s dtoa (Lucent Technologies, 1991/2000/2001), the canonical correctly-rounded binary-to-decimal converter (dtoa.c). Its mode 0 is documented in the source as “shortest string that yields d when read in and rounded to nearest.” That is the round-trip guarantee: repr(x) produces the fewest decimal digits such that float(repr(x)) == x exactly. So repr(0.1) is '0.1' (not 0.1000000000000000055...), but repr(0.1 + 0.2) is '0.30000000000000004' because that 17-digit value is the shortest one that reads back to the actual double result of the addition. The format also flips to exponential notation when the decimal point falls outside a range (specifically decpt <= -4 or decpt > 16), with a comment explaining the threshold was moved from 1e17 to 1e16 to avoid “bogus zeros” in padded reprs.

Before Python 3.1, repr showed 17 digits and str showed 12; since 3.1 both use this shortest-round-trip algorithm, which is why str(x) and repr(x) now agree for floats.

float.hex — exact, lossless interchange

For round-tripping without any decimal-conversion ambiguity, float.hex() emits the value in C99 hexadecimal-float notation (floatobject.c):

static PyObject *
float_hex_impl(PyObject *self)
{
    ...
    if (isnan(x) || isinf(x))
        return float_repr(self);
    if (x == 0.0) {
        if (copysign(1.0, x) == -1.0)
            return PyUnicode_FromString("-0x0.0p+0");
        else
            return PyUnicode_FromString("0x0.0p+0");
    }
    m = frexp(fabs(x), &e);      /* split into mantissa m in [0.5,1) and exponent e */
    ...
}

The output has the form [sign]0x1.<hex-mantissa>p<exponent>, where the mantissa is the exact 53-bit significand written in hex and the p exponent is the power of two. Measured on CPython 3.14.5:

>>> (3.14159).hex()
'0x1.921f9f01b866ep+1'
>>> (-0.1).hex()
'-0x1.999999999999ap-4'
>>> (0.0).hex(), (-0.0).hex()
('0x0.0p+0', '-0x0.0p+0')
>>> float.fromhex('0x1p10')
1024.0

(-0.1).hex() shows the repeating pattern 999...a — 0.1 has no exact binary representation, and the hex makes the truncation visible. Note the code calls float_repr for NaN/inf (so float('nan').hex() is 'nan') and special-cases signed zero via copysign. float.fromhex is the exact inverse, useful for serializing a float and recovering the identical bit pattern.

Special values and comparison gotchas

The special IEEE 754 values produce behaviour that surprises people who think of == as “same value.” All measured live on CPython 3.14.5:

>>> n = float('nan')
>>> n == n          # False  -- NaN is unequal to everything, including itself
False
>>> n != n          # True   -- the canonical "is it NaN" test
True
>>> -0.0 == 0.0     # True   -- positive and negative zero compare equal
True
>>> repr(-0.0)      # '-0.0' -- but they are distinct objects/bit patterns
'-0.0'
>>> import math
>>> math.copysign(1, -0.0), math.copysign(1, 0.0)   # (-1.0, 1.0) -- distinguishable
(-1.0, 1.0)
  • nan != nan. IEEE 754 mandates that any comparison involving NaN (except !=) is false. So nan == nan is False and nan != nan is True. CPython’s float_richcompare implements this faithfully (floatobject.c — the source comment calls comparison “pretty much a nightmare” because of NaN and float-vs-int magnitude edge cases). This breaks containers and sorting: a nan in a list defeats in/sort invariants, and a nan key in a dict can never be looked up via an equal-valued literal.
  • Signed zero. +0.0 == −0.0 is True, but they are different bit patterns. repr distinguishes them and math.copysign can read the sign — relevant for branch-cut-sensitive math (e.g. atan2, complex functions).
  • The identity twist. Because nan != nan, but a single nan object is itself, you can get [n].count(n) == 1 via identity while n == n is False. CPython’s containers fall back to identity before equality, so a nan is findable in the exact list that contains it but not in a fresh list with an equal-valued nan. This is the float analogue of the integer identity surprises in Integer and String Identity Surprises and is detailed in Floating Point Pitfalls in Python.

How this differs from int and from a raw double

  • Versus [[CPython Integer Internals|int]]: float is fixed-width (always 24 bytes, always 53 bits of precision) and inexact (most decimals cannot be represented), where int is variable-width and exact. A large int converted to float can lose precision or overflow to inf.
  • Versus a raw C double: the Python object adds the header and reference count (24 vs 8 bytes) and the boxing indirection, in exchange for being a first-class object. NumPy’s float64 stores unboxed doubles in arrays precisely to avoid this per-element overhead.

Common Misunderstandings

  • “Python floats are arbitrary precision like Python ints.” No. int grows; float is a fixed 64-bit double. For arbitrary-precision decimals use the decimal module; for exact rationals use fractions.Fraction.
  • “0.1 + 0.2 == 0.3 should be true.” It is False (0.30000000000000004) — none of 0.1, 0.2, 0.3 are exactly representable in binary, and the rounded sum differs from the rounded 0.3. This is IEEE 754, not a Python bug. See Floating Point Pitfalls in Python.
  • “repr rounds, so it loses information.” The opposite — repr is engineered (via dtoa mode 0) to be the shortest string that loses no information; float(repr(x)) == x always holds.
  • “NaN equals NaN.” Never under ==. Use math.isnan(x).

Production Notes

The shortest-round-trip repr (David Gay dtoa, mode 0) landed in CPython 3.1 and remains the behaviour in 3.14; it is why float serialization via repr/str is lossless across a save/load round-trip on the same platform. The free-list consolidation into _Py_freelists (per-interpreter under the GIL, per-thread in the free-threaded build) is the modern shape as of 3.14 — code that profiled float allocation against an older “global free list” mental model should re-check against the current per-state layout when reasoning about sub-interpreters or free-threading. For anyone debugging “why does this float print so many digits,” the answer is almost always that the underlying double genuinely differs from the short decimal you typed, and (x).hex() is the fastest way to see the exact stored value.

See Also

Graph View

Backlinks