`defaultdict` reads `default_factory` as a borrowed reference in `defdict_repr` / `defdict_missing` under free-threading

[Naserume]

Crash report

What happened?

In the free-threaded build, collections.defaultdict reads its default_factory slot as a plain, borrowed reference and then uses the pointed-to object (reprs it, or calls it) without holding a strong reference or taking the object's critical section.

defdict_repr reads dd->default_factory directly and passes it to PyObject_Repr (and Py_ReprEnter/Py_ReprLeave) with no incref and no critical section.

cpython/Modules/_collectionsmodule.c

Lines 2373 to 2410 in a2a8466

	static PyObject *
	defdict_repr(PyObject *op)
	{
	defdictobject *dd = defdictobject_CAST(op);
	PyObject *baserepr;
	PyObject *defrepr;
	PyObject *result;
	baserepr = PyDict_Type.tp_repr(op);
	if (baserepr == NULL)
	return NULL;
	if (dd->default_factory == NULL)
	defrepr = PyUnicode_FromString("None");
	else
	{
	int status = Py_ReprEnter(dd->default_factory);
	if (status != 0) {
	if (status < 0) {
	Py_DECREF(baserepr);
	return NULL;
	}
	defrepr = PyUnicode_FromString("...");
	}
	else {
	defrepr = PyObject_Repr(dd->default_factory);
	Py_ReprLeave(dd->default_factory);
	}
	}
	if (defrepr == NULL) {
	Py_DECREF(baserepr);
	return NULL;
	}
	result = PyUnicode_FromFormat("%s(%U, %U)",
	_PyType_Name(Py_TYPE(dd)),
	defrepr, baserepr);
	Py_DECREF(defrepr);
	Py_DECREF(baserepr);
	return result;
	}

defdict_missing (the __missing__ path taken by dd[key] for an absent key) loads dd->default_factory into a local borrowed pointer and calls it.

cpython/Modules/_collectionsmodule.c

Lines 2233 to 2256 in a2a8466

	static PyObject *
	defdict_missing(PyObject *op, PyObject *key)
	{
	defdictobject *dd = defdictobject_CAST(op);
	PyObject *factory = dd->default_factory;
	PyObject *value;
	if (factory == NULL || factory == Py_None) {
	/* XXX Call dict.__missing__(key) */
	PyObject *tup;
	tup = PyTuple_Pack(1, key);
	if (!tup) return NULL;
	PyErr_SetObject(PyExc_KeyError, tup);
	Py_DECREF(tup);
	return NULL;
	}
	value = _PyObject_CallNoArgs(factory);
	if (value == NULL)
	return value;
	PyObject *result = NULL;
	(void)PyDict_SetDefaultRef(op, key, value, &result);
	// 'result' is NULL, or a strong reference to 'value' or 'op[key]'
	Py_DECREF(value);
	return result;
	}

The writer side is already free-threading-correct. Assigning dd.default_factory goes through PyMember_SetOne (Py_T_OBJECT_EX), which publishes the new value under a critical section with an atomic release store and then decrefs the old value.

cpython/Python/structmember.c

Lines 322 to 327 in a2a8466

	case _Py_T_OBJECT:
	case Py_T_OBJECT_EX:
	Py_BEGIN_CRITICAL_SECTION(obj);
	oldv = *(PyObject **)addr;
	FT_ATOMIC_STORE_PTR_RELEASE(*(PyObject **)addr, Py_XNewRef(v));
	Py_END_CRITICAL_SECTION();

So the defect is entirely on the reader side. The plain borrowed load races the writer's atomic store (a data race on the field), and the borrowed pointer is then used after the writer's Py_XDECREF has already freed the old factory.

When the old factory is referenced only by the defaultdict, a concurrent assignment drops its refcount to zero and func_dealloc runs while a reader is mid-repr or mid-call on it. The reader then reprs or calls a freed object, a use-after-free that crashes(segfault) or corrupts memory.

Reproducer:

import collections
from threading import Thread

dd = collections.defaultdict(lambda: 0)
KEY = "k"

def setter():
    for _ in range(500000):
        dd.default_factory = lambda: 1 

def reprer():
    for _ in range(20000):
        try:
            repr(dd)
        except Exception:
            pass

def getter():
    for _ in range(20000):
        try:
            dd.pop(KEY, None)
            dd[KEY] 
        except Exception:
            pass

threads  = [Thread(target=setter) for _ in range(4)]
threads += [Thread(target=reprer) for _ in range(6)]
threads += [Thread(target=getter) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()

TSAN Report:

==================
WARNING: ThreadSanitizer: data race (pid=1525801)
  Atomic write of size 8 at 0x7fffb66a4370 by thread T1:
    #0 _Py_atomic_store_ptr_release cpython/./Include/cpython/pyatomic_gcc.h:565:3 
    #1 PyMember_SetOne cpython/Python/structmember.c:326:9
    #2 member_set cpython/Objects/descrobject.c:239:12
    #3 _PyObject_GenericSetAttrWithDict cpython/Objects/object.c:2049:19 
    #4 PyObject_GenericSetAttr cpython/Objects/object.c:2120:12
    #5 PyObject_SetAttr cpython/Objects/object.c:1533:15
    #6 _PyEval_EvalFrameDefault cpython/Python/generated_cases.c.h:12146:27 

  Previous read of size 8 at 0x7fffb66a4370 by thread T5:
    #0 defdict_repr cpython/./Modules/_collectionsmodule.c:2384:13 
    #1 PyObject_Repr cpython/Objects/object.c:784:11
    #2 builtin_repr cpython/Python/bltinmodule.c:2677:12
    #3 _PyEval_EvalFrameDefault cpython/Python/generated_cases.c.h:2712:35

SUMMARY: ThreadSanitizer: data race cpython/./Include/cpython/pyatomic_gcc.h:565:3 in _Py_atomic_store_ptr_release
==================
==================
WARNING: ThreadSanitizer: data race (pid=1525801)
  Read of size 8 at 0x7fffb66a4370 by thread T11:
    #0 defdict_missing cpython/./Modules/_collectionsmodule.c:2238:29 
    #1 cfunction_vectorcall_O cpython/Objects/methodobject.c:536:24 
    #2 _PyObject_VectorcallTstate cpython/./Include/internal/pycore_call.h:144:11 
    #3 PyObject_CallOneArg cpython/Objects/call.c:395:12 
    #4 _PyDict_SubscriptKnownHash cpython/Objects/dictobject.c:3722:23 
    #5 _PyDict_Subscript cpython/Objects/dictobject.c:3743:12 
    #6 _PyEval_EvalFrameDefault cpython/Python/generated_cases.c.h:686:35

  Previous atomic write of size 8 at 0x7fffb66a4370 by thread T4:
    #0 _Py_atomic_store_ptr_release cpython/./Include/cpython/pyatomic_gcc.h:565:3 
    #1 PyMember_SetOne cpython/Python/structmember.c:326:9 
    #2 member_set cpython/Objects/descrobject.c:239:12 
    #3 _PyObject_GenericSetAttrWithDict cpython/Objects/object.c:2049:19 
    #4 PyObject_GenericSetAttr cpython/Objects/object.c:2120:12 
    #5 PyObject_SetAttr cpython/Objects/object.c:1533:15
    #6 _PyEval_EvalFrameDefault cpython/Python/generated_cases.c.h:12146:27

SUMMARY: ThreadSanitizer: data race cpython/./Modules/_collectionsmodule.c:2238:29 in defdict_missing
==================
...
==================
WARNING: ThreadSanitizer: data race (pid=1525801)
  Read of size 8 at 0x7fffc00f0d98 by thread T7:
    #0 PyObject_Repr cpython/Objects/object.c:766:9 
    #1 defdict_repr cpython/./Modules/_collectionsmodule.c:2397:23
    #2 PyObject_Repr cpython/Objects/object.c:784:11
    #3 builtin_repr cpython/Python/bltinmodule.c:2677:12 
    #4 _PyEval_EvalFrameDefault cpython/Python/generated_cases.c.h:2712:35

  Previous write of size 8 at 0x7fffc00f0d98 by thread T3:
    #0 Py_SET_TYPE cpython/./Include/object.h:207:17
    #1 _PyObject_Init cpython/./Include/internal/pycore_object.h:482:5 
    #2 _PyObject_GC_New cpython/Python/gc_free_threading.c:2752:5
    #3 PyFunction_NewWithQualName cpython/Objects/funcobject.c:202:28
    #4 PyFunction_New cpython/Objects/funcobject.c:392:12
    #5 _PyEval_EvalFrameDefault cpython/Python/generated_cases.c.h:10844:17 

SUMMARY: ThreadSanitizer: data race cpython/Objects/object.c:766:9 in PyObject_Repr
==================
...
==================
WARNING: ThreadSanitizer: data race (pid=1525801)
  Read of size 8 at 0x7fffb80f0f98 by thread T11:
    #0 _PyVectorcall_FunctionInline cpython/./Include/internal/pycore_call.h:105:5 
    #1 _PyObject_VectorcallTstate cpython/./Include/internal/pycore_call.h:139:12
    #2 _PyObject_CallNoArgs cpython/./Include/internal/pycore_call.h:160:12 
    #3 defdict_missing cpython/./Modules/_collectionsmodule.c:2249:13
    #4 cfunction_vectorcall_O cpython/Objects/methodobject.c:536:24
    #5 _PyObject_VectorcallTstate cpython/./Include/internal/pycore_call.h:144:11
    #6 PyObject_CallOneArg cpython/Objects/call.c:395:12 
    #7 _PyDict_SubscriptKnownHash cpython/Objects/dictobject.c:3722:23
    #8 _PyDict_Subscript cpython/Objects/dictobject.c:3743:12
    #9 PyObject_GetItem cpython/Objects/abstract.c:163:26
    #10 _PyEval_EvalFrameDefault cpython/Python/generated_cases.c.h:67:35

  Previous write of size 8 at 0x7fffb80f0f98 by thread T1:
    #0 PyFunction_NewWithQualName cpython/Objects/funcobject.c:224:20
    #1 PyFunction_New cpython/Objects/funcobject.c:392:12
    #2 _PyEval_EvalFrameDefault cpython/Python/generated_cases.c.h:10844:17

SUMMARY: ThreadSanitizer: data race cpython/./Include/internal/pycore_call.h:105:5 in _PyVectorcall_FunctionInline
==================
...
==================
ThreadSanitizer:DEADLYSIGNAL
==1525801==ERROR: ThreadSanitizer: SEGV on unknown address (pc 0x5555556b79ac bp 0x7fffaf3f98b0 sp 0x7fffaf3f9858 T1525807)
==1525801==The signal is caused by a READ memory access.
==1525801==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
ThreadSanitizer:DEADLYSIGNA

(or this happened)

SUMMARY: ThreadSanitizer: data race /cpython/Objects/object.c:477:40 in _Py_ExplicitMergeRefcount
==================
SystemError: Objects/weakrefobject.c:1023: bad argument to internal function

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/cpython/cpython-tsan/lib/python3.16t/threading.py", line 1218, in _bootstrap_inner
    self._context.run(self.run)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/cpython/cpython-tsan/lib/python3.16t/threading.py", line 1164, in run
    del self._target, self._args, self._kwargs
        ^^^^^^^^^^^^
SystemError: PyObject_SetAttr() must not be called with NULL value and an exception set
SystemError: Objects/weakrefobject.c:1023: bad argument to internal function

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/cpython/cpython-tsan/lib/python3.16t/threading.py", line 1218, in _bootstrap_inner
    self._context.run(self.run)
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "/cpython/cpython-tsan/lib/python3.16t/threading.py", line 1164, in run
    del self._target, self._args, self._kwargs
        ^^^^^^^^^^^^
SystemError: PyObject_SetAttr() must not be called with NULL value and an exception set
ThreadSanitizer: reported 36 warnings

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.16.0a0 free-threading build (tags/v3.15.0b1-614-g7928a8b730b:7928a8b730b, Jul 22 2026, 10:44:15) [Clang 18.1.3 (1ubuntu1)]