vault

GIL Mechanics and the Switch Interval

13 min read

GIL Mechanics and the Switch Interval

The Global Interpreter Lock explains why CPython has one lock and what it protects. This note explains the other half: how that lock is actually held and handed from one thread to the next. The current machinery dates to a 2009 rewrite by Antoine Pitrou, shipped in Python 3.2, which scrapped the old “check interval” (drop the lock every 100 bytecodes) in favour of a time-based switch interval (default 5 milliseconds), and added a forced hand-off so a thread that releases the lock cannot immediately grab it back (whatsnew 3.2, lines 2315–2333; ceval_gil.c). Understanding this mechanism explains why a single configuration knob — sys.setswitchinterval() — trades latency against throughput, and why CPU-bound Python threads on a multicore machine used to run slower than on one core.

Mental Model — A Polite Lock With a Timeout

A naive global lock would be a mutex.lock() / mutex.unlock() around each bytecode: correct, but it would let one CPU-bound thread monopolise the interpreter forever, since it would re-grab the lock the instant it released it. The GIL instead behaves like a cooperative scheduler built on a condition variable. A thread that wants the lock does not spin; it waits on a condition variable for up to one switch interval (5 ms by default). If the interval elapses and the current holder has not voluntarily let go, the waiter sets a flag — the gil_drop_request, carried as a bit in the holder’s eval breaker — politely asking the holder to drop the lock at its next safe point. When the holder obliges, a forced-switch mechanism makes it wait until some other thread has actually taken the lock before it is allowed to compete again. The result is a defined, fair-ish rotation rather than a free-for-all.

sequenceDiagram
    participant H as Holder thread (running bytecode)
    participant G as GIL state (locked, cond, last_holder, switch_number)
    participant W as Waiter thread (in take_gil)
    W->>G: locked==1, so COND_TIMED_WAIT(cond, interval=5ms)
    Note over W: timeout fires, switch_number unchanged
    W->>H: set _PY_GIL_DROP_REQUEST_BIT in holder's eval_breaker
    Note over H: at next back-edge / CALL, eval breaker is checked
    H->>G: drop_gil(): locked=0, COND_SIGNAL(cond)
    H->>G: FORCE_SWITCHING: wait on switch_cond until last_holder != me
    G-->>W: wakes from cond wait
    W->>G: locked=1, last_holder=W, switch_number++, COND_SIGNAL(switch_cond)
    G-->>H: holder released to compete again

Diagram: the hand-off protocol. A waiter times out, requests a drop, the holder releases at a safe point and then blocks on switch_cond until last_holder changes — guaranteeing the waiter, not the ex-holder, gets the next turn. The insight: the forced-switch step is what prevents the releasing thread from re-acquiring its own lock and starving everyone else, the exact pathology of the pre-3.2 GIL.

The gil_runtime_state Structure

Everything the GIL needs lives in one struct, _gil_runtime_state, defined in pycore_gil.h (lines 22–61). The fields that matter for hand-off:

struct _gil_runtime_state {
    unsigned long interval;          // switch interval in MICROSECONDS (API uses seconds)
    PyThreadState* last_holder;      // last thread that held / holds the GIL
    int locked;                      // is the GIL taken? (-1 if uninitialized)
    unsigned long switch_number;     // count of GIL hand-offs since startup
    PyCOND_T cond;  PyMUTEX_T mutex; // wait for release; mutex also guards the fields above
    PyCOND_T switch_cond;  PyMUTEX_T switch_mutex;  // forced-switch hand-off (FORCE_SWITCHING)
};

Field by field. interval is the switch interval in microseconds — note the units: the C side stores microseconds, while the Python API (sys.{get,set}switchinterval) speaks seconds, so the default of 5 ms is stored as 5000. last_holder records which PyThreadState most recently held the lock; comparing it to “me” is how a thread detects whether anyone else has been scheduled since it dropped the lock. locked is the actual lock state — the boolean the whole edifice protects, read atomically without the mutex in the eval loop. switch_number is a monotonically increasing counter of hand-offs, used by a waiter to detect “did a switch happen while I was waiting, even though the lock is busy again?” The cond/mutex pair is the primary condition variable: mutex is held only briefly to guard the fields, and cond is what releasing signals so waiters wake. The switch_cond/switch_mutex pair exists only when FORCE_SWITCHING is defined (it is, by default — line 20 of the header) and implements the forced hand-off.

The default is set in ceval_gil.c:

#define DEFAULT_INTERVAL 5000        // line 147 — 5000 microseconds == 5 ms == 0.005 s
 
static void _gil_initialize(struct _gil_runtime_state *gil) {
    gil->locked = -1;                // -1 means "GIL not yet created"
    gil->interval = DEFAULT_INTERVAL;
}

This 5000 is the load-bearing constant: it is exactly the 0.005 seconds that sys.getswitchinterval() returns on a fresh interpreter, and it is verified here against the v3.14.5 source tag (not the unreleased main branch).

Acquiring the GIL — take_gil

When a detaching/attaching thread wants to run Python code, it calls take_gil(tstate) (ceval_gil.c, lines 284–418). Stripped to the scheduling core, the wait loop is:

MUTEX_LOCK(gil->mutex);
int drop_requested = 0;
while (_Py_atomic_load_int_relaxed(&gil->locked)) {          // (1) lock is busy
    unsigned long saved_switchnum = gil->switch_number;      // (2) snapshot switch count
    unsigned long interval = _Py_atomic_load_ulong_relaxed(&gil->interval);
    if (interval < 1) interval = 1;
    int timed_out = 0;
    COND_TIMED_WAIT(gil->cond, gil->mutex, interval, timed_out);   // (3) wait up to `interval` µs
    if (timed_out &&
        _Py_atomic_load_int_relaxed(&gil->locked) &&
        gil->switch_number == saved_switchnum)               // (4) timed out AND no switch happened
    {
        PyThreadState *holder_tstate =
            (PyThreadState*)_Py_atomic_load_ptr_relaxed(&gil->last_holder);
        _Py_set_eval_breaker_bit(holder_tstate, _PY_GIL_DROP_REQUEST_BIT);  // (5) ask holder to drop
        drop_requested = 1;
    }
}

Reading the numbered points: (1) the thread loops as long as locked is set — someone else holds the GIL. (2) before waiting, it snapshots switch_number; this lets it distinguish “the lock is still busy because nobody dropped it” from “the lock changed hands while I waited.” (3) it does a timed wait on the condition variable for one interval of microseconds, releasing gil->mutex while it sleeps; if the holder calls COND_SIGNAL(cond) (i.e. drops the GIL) before the timeout, it wakes early and loops back to recheck locked. (4) the crucial condition: if the wait timed out, the lock is still held, and switch_number is unchanged from the snapshot, then the current holder has been hogging the lock for a full interval without any hand-off occurring. (5) in that case the waiter sets the _PY_GIL_DROP_REQUEST_BIT on the holder’s eval breaker — it cannot force the holder to stop, but it raises a flag the holder will see at its next safe point.

Once the loop finally observes locked == 0, the thread claims the lock (lines 380–387):

_Py_atomic_store_int_relaxed(&gil->locked, 1);               // I hold the GIL now
if (tstate != (PyThreadState*)_Py_atomic_load_ptr_relaxed(&gil->last_holder)) {
    _Py_atomic_store_ptr_relaxed(&gil->last_holder, tstate); // record me as holder
    ++gil->switch_number;                                    // a real hand-off happened
}

Note that switch_number only increments when the new holder differs from last_holder — i.e. only on an actual change of thread, not when the same thread re-acquires. That is exactly the signal the waiter’s check at point (4) keys off.

Requesting and Performing the Drop

The drop request set at point (5) is not a standalone “drop now” command; it is a single bit, _PY_GIL_DROP_REQUEST_BIT, OR-ed into the holder thread’s eval breaker — the per-thread bitfield the bytecode interpreter glances at during execution.

Note

The long comment at the top of ceval_gil.c (lines 20–24) still describes a “volatile boolean variable (gil_drop_request) … checked at every turn of the eval loop.” That phrasing is historical. In current 3.14 the request is the _PY_GIL_DROP_REQUEST_BIT of the eval breaker (set at take_gil line 359, tested in _Py_HandlePending line 1409 and in drop_gil line 259), and the eval breaker is not checked on every instruction — it is checked at back edges of loops and on most calls (the file’s own later comment, lines 1331–1347, says so, as does The eval Breaker). Treat “gil_drop_request” as the name of the concept, implemented today as an eval-breaker bit.

When the holder reaches a safe point and finds the bit set, _Py_HandlePending() does the actual yield (lines 1408–1416):

/* GIL drop request */
if ((breaker & _PY_GIL_DROP_REQUEST_BIT) != 0) {
    _PyThreadState_Detach(tstate);   // drops the GIL → other threads may run now
    _PyThreadState_Attach(tstate);   // immediately tries to re-acquire it
}

The detach releases the GIL (ultimately calling drop_gil), and the attach turns around and re-acquires it. On its own that would let the holder grab the lock straight back — which is where forced switching earns its keep.

Forced Switching — Why the Ex-Holder Waits

The releasing path, drop_gil (ceval_gil.c, lines 215–275), does the obvious part — clear locked, signal cond — and then, when FORCE_SWITCHING is enabled and a drop was requested, does something subtle (lines 258–273):

if (!final_release &&
    _Py_eval_breaker_bit_is_set(tstate, _PY_GIL_DROP_REQUEST_BIT)) {
    MUTEX_LOCK(gil->switch_mutex);
    if (((PyThreadState*)_Py_atomic_load_ptr_relaxed(&gil->last_holder)) == tstate) {
        _Py_unset_eval_breaker_bit(tstate, _PY_GIL_DROP_REQUEST_BIT);
        COND_WAIT(gil->switch_cond, gil->switch_mutex);   // block until someone else takes the GIL
    }
    MUTEX_UNLOCK(gil->switch_mutex);
}

The logic: if I dropped the GIL because I was asked to, I check whether last_holder is still me. If it is, no other thread has taken the lock yet, so I block on switch_cond until one does. The corresponding wake-up is in take_gil (lines 389–392), where the new holder does COND_SIGNAL(gil->switch_cond) right after updating last_holder to itself. Only then is the ex-holder released to compete again.

The file’s own comment explains the purpose (lines 46–49): without this, “one thread would speculatively release the GIL, but still run and end up being the first to re-acquire it, making the ‘timeslices’ much longer than expected.” Forced switching converts a hint (“please drop”) into a guarantee (“you will not run again until someone else has had a turn”), which is what makes the rotation fair on multicore hardware where the OS might otherwise schedule the just-released thread right back onto its hot core.

Why the Rewrite Happened — The Old GIL and the Convoy Effect

Before Python 3.2 the GIL had no notion of time. Instead it used a check interval: a global tick counter, set with the old sys.setcheckinterval(), that caused the running thread to consider releasing the GIL “every 100 bytecode instructions” by default. The problem, as Pitrou argued in his 2009 python-dev design message, is that bytecodes have wildly different durations — a single opcode can be anything from a few nanoseconds (a trivial is not None) to a fraction of a second (a heavy method call). Counting opcodes therefore bears no relationship to wall-clock fairness — 100 trivial opcodes might fly by in a microsecond, forcing pointless lock churn, while 100 heavy opcodes could hold the lock for an age.

Worse was the multicore behaviour David Beazley famously measured (“the Dave Beazley effect”): on a CPU-bound workload, two threads could run dramatically slower than one. The cause was a convoy effect. Under the old scheme, releasing the GIL merely unlocked it; it did nothing to ensure another thread was actually scheduled. On a multicore machine the just-released thread, still hot on its core, would frequently re-acquire the GIL before the waiting thread on another core could even be woken — so the waiter would repeatedly wake, find the lock taken again, and go back to sleep, burning CPU on futile lock operations and system calls. Pitrou’s message reports that on platforms with costly locks this produced roughly a 50% slowdown going from one thread to two on trivial workloads, and that an I/O thread sharing the interpreter with a compute thread could see close to a second of average latency waiting for the GIL.

The 3.2 rewrite addressed both. The What’s New summary is the authoritative one-paragraph statement: the GIL “has been rewritten. Among the objectives were more predictable switching intervals and reduced overhead due to lock contention and the number of ensuing system calls. The notion of a ‘check interval’ to allow thread switches has been abandoned and replaced by an absolute duration expressed in seconds. This parameter is tunable through sys.setswitchinterval. It currently defaults to 5 milliseconds” (whatsnew 3.2, lines 2318–2325). The time-based interval makes switching wall-clock-predictable, and the forced-switch hand-off cures the convoy by guaranteeing the waiter, not the ex-holder, runs next. Pitrou reported the latency case improving from close to a second down to a few milliseconds, with throughput roughly equal or slightly better.

Note

Pitrou’s original python-dev proposal also described priority requests — letting an I/O thread demand an immediate hand-off. These were not merged: the What’s New note records that “‘priority requests’ as exposed in this message have not been kept for inclusion” (whatsnew 3.2, lines 2330–2331), and there is no priority field in _gil_runtime_state. Do not describe priority requests as part of the shipped GIL.

The sys Knob and Why I/O Threads Release the GIL

The switch interval is read and written from Python with sys.getswitchinterval() and sys.setswitchinterval(seconds), both added in 3.2 (sys docs). The setter’s documentation is careful about what the knob does and does not promise: it “determines the ideal duration of the ‘timeslices’ allocated to concurrently running Python threads. Please note that the actual value can be higher, especially if long-running internal functions or methods are used. Also, which thread becomes scheduled at the end of the interval is the operating system’s decision. The interpreter doesn’t have its own scheduler” (sys.rst, lines 1746–1754). Two cautions follow from this. First, the interval is a target, not a guarantee: a single opcode (or a C call that holds the GIL) can run far longer than 5 ms, and the waiter cannot preempt it — it can only set the drop-request bit and wait for the holder to reach a safe point. Second, which thread runs after a hand-off is up to the OS scheduler; CPython only decides when to offer a switch, not who wins it.

This connects directly back to why I/O-bound threads behave so much better than CPU-bound ones under the GIL. A CPU-bound thread holds the GIL continuously and can only be pried off it via the 5 ms timeout-and-request dance described above. An I/O-bound thread, by contrast, voluntarily releases the GIL the moment it blocks, via Py_BEGIN_ALLOW_THREADS / PyEval_SaveThread() (covered in The Global Interpreter Lock and Thread State and the GIL Handshake). Because the release is immediate and explicit rather than forced after a timeout, I/O threads hand the lock off promptly and overlap cleanly — which is exactly why threading is effective for I/O-bound work and useless for CPU-bound pure-Python work. Tuning setswitchinterval shifts the trade-off: a smaller interval switches more often, lowering worst-case latency for a starving thread but raising switching overhead; a larger interval does the reverse. In practice the 5 ms default is rarely worth changing.

See Also

Graph View

Backlinks