gh-133164: Add PyUnstable_Object_IsUniqueReferencedTemporary C API (gh-133170)

After gh-130704, the interpreter replaces some uses of `LOAD_FAST` with
`LOAD_FAST_BORROW` which avoid incref/decrefs by "borrowing" references
on the interpreter stack when the bytecode compiler can determine that
it's safe.

This change broke some checks in C API extensions that relied on
`Py_REFCNT()` of `1` to determine if it's safe to modify an object
in-place. Objects may have a reference count of one, but still be
referenced further up the interpreter stack due to borrowing of
references.

This provides a replacement function for those checks.
`PyUnstable_Object_IsUniqueReferencedTemporary` is more conservative:
it checks that the object has a reference count of one and that it exists as a
unique strong reference in the interpreter's stack of temporary
variables in the top most frame.

See also:

* numpy/numpy#28681

Co-authored-by: Pieter Eendebak <[email protected]>
Co-authored-by: T. Wouters <[email protected]>
Co-authored-by: mpage <[email protected]>
Co-authored-by: Mark Shannon <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>

Author: 6 people

Doc/c-api/object.rst

@@ -613,6 +613,38 @@ Object Protocol
 
    .. versionadded:: 3.14
 
+.. c:function:: int PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *obj)
+
+   Check if *obj* is a unique temporary object.
+   Returns ``1`` if *obj* is known to be a unique temporary object,
+   and ``0`` otherwise.  This function cannot fail, but the check is
+   conservative, and may return ``0`` in some cases even if *obj* is a unique
+   temporary object.
+
+   If an object is a unique temporary, it is guaranteed that the current code
+   has the only reference to the object. For arguments to C functions, this
+   should be used instead of checking if the reference count is ``1``. Starting
+   with Python 3.14, the interpreter internally avoids some reference count
+   modifications when loading objects onto the operands stack by
+   :term:`borrowing <borrowed reference>` references when possible, which means
+   that a reference count of ``1`` by itself does not guarantee that a function
+   argument uniquely referenced.
+
+   In the example below, ``my_func`` is called with a unique temporary object
+   as its argument::
+
+      my_func([1, 2, 3])
+
+   In the example below, ``my_func`` is **not** called with a unique temporary
+   object as its argument, even if its refcount is ``1``::
+
+      my_list = [1, 2, 3]
+      my_func(my_list)
+
+   See also the function :c:func:`Py_REFCNT`.
+
+   .. versionadded:: 3.14
+
 .. c:function:: int PyUnstable_IsImmortal(PyObject *obj)
 
    This function returns non-zero if *obj* is :term:`immortal`, and zero

Doc/c-api/refcounting.rst

@@ -23,6 +23,8 @@ of Python objects.
 
    Use the :c:func:`Py_SET_REFCNT()` function to set an object reference count.
 
+   See also the function :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary()`.
+
    .. versionchanged:: 3.10
       :c:func:`Py_REFCNT()` is changed to the inline static function.
 

Doc/whatsnew/3.14.rst

@@ -89,6 +89,10 @@ If you encounter :exc:`NameError`\s or pickling errors coming out of
 :mod:`multiprocessing` or :mod:`concurrent.futures`, see the
 :ref:`forkserver restrictions <multiprocessing-programming-forkserver>`.
 
+The interpreter avoids some reference count modifications internally when
+it's safe to do so. This can lead to different values returned from
+:func:`sys.getrefcount` and :c:func:`Py_REFCNT` compared to previous versions
+of Python.  See :ref:`below <whatsnew314-refcount>` for details.
 
 New features
 ============
@@ -2215,6 +2219,11 @@ New features
   take a C integer and produce a Python :class:`bool` object. (Contributed by
   Pablo Galindo in :issue:`45325`.)
 
+* Add :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` to determine if an object
+  is a unique temporary object on the interpreter's operand stack. This can
+  be used in some cases as a replacement for checking if :c:func:`Py_REFCNT`
+  is ``1`` for Python objects passed as arguments to C API functions.
+
 
 Limited C API changes
 ---------------------
@@ -2249,6 +2258,17 @@ Porting to Python 3.14
   a :exc:`UnicodeError` object.
   (Contributed by Bénédikt Tran in :gh:`127691`.)
 
+.. _whatsnew314-refcount:
+
+* The interpreter internally avoids some reference count modifications when
+  loading objects onto the operands stack by :term:`borrowing <borrowed reference>`
+  references when possible. This can lead to smaller reference count values
+  compared to previous Python versions. C API extensions that checked
+  :c:func:`Py_REFCNT` of ``1`` to determine if an function argument is not
+  referenced by any other code should instead use
+  :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` as a safer replacement.
+
+
 * Private functions promoted to public C APIs:
 
   * ``_PyBytes_Join()``: :c:func:`PyBytes_Join`.

Include/cpython/object.h

@@ -476,6 +476,11 @@ PyAPI_FUNC(PyRefTracer) PyRefTracer_GetTracer(void**);
  */
 PyAPI_FUNC(int) PyUnstable_Object_EnableDeferredRefcount(PyObject *);
 
+/* Determine if the object exists as a unique temporary variable on the
+ * topmost frame of the interpreter.
+ */
+PyAPI_FUNC(int) PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *);
+
 /* Check whether the object is immortal. This cannot fail. */
 PyAPI_FUNC(int) PyUnstable_IsImmortal(PyObject *);
 

Lib/test/test_capi/test_object.py

@@ -1,4 +1,5 @@
 import enum
+import sys
 import textwrap
 import unittest
 from test import support
@@ -223,5 +224,17 @@ def __del__(self):
             obj = MyObj()
             _testinternalcapi.incref_decref_delayed(obj)
 
+    def test_is_unique_temporary(self):
+        self.assertTrue(_testcapi.pyobject_is_unique_temporary(object()))
+        obj = object()
+        self.assertFalse(_testcapi.pyobject_is_unique_temporary(obj))
+
+        def func(x):
+            # This relies on the LOAD_FAST_BORROW optimization (gh-130704)
+            self.assertEqual(sys.getrefcount(x), 1)
+            self.assertFalse(_testcapi.pyobject_is_unique_temporary(x))
+
+        func(object())
+
 if __name__ == "__main__":
     unittest.main()

Misc/NEWS.d/next/C_API/2025-04-29-19-39-16.gh-issue-133164.W-XTU7.rst

@@ -0,0 +1,5 @@
+Add :c:func:`PyUnstable_Object_IsUniqueReferencedTemporary` function for
+determining if an object exists as a unique temporary variable on the
+interpreter's stack.  This is a replacement for some cases where checking
+that :c:func:`Py_REFCNT` is one is no longer sufficient to determine if it's
+safe to modify a Python object in-place with no visible side effects.

Modules/_testcapi/object.c

@@ -131,6 +131,13 @@ pyobject_enable_deferred_refcount(PyObject *self, PyObject *obj)
     return PyLong_FromLong(result);
 }
 
+static PyObject *
+pyobject_is_unique_temporary(PyObject *self, PyObject *obj)
+{
+    int result = PyUnstable_Object_IsUniqueReferencedTemporary(obj);
+    return PyLong_FromLong(result);
+}
+
 static int MyObject_dealloc_called = 0;
 
 static void
@@ -478,6 +485,7 @@ static PyMethodDef test_methods[] = {
     {"pyobject_print_os_error", pyobject_print_os_error, METH_VARARGS},
     {"pyobject_clear_weakrefs_no_callbacks", pyobject_clear_weakrefs_no_callbacks, METH_O},
     {"pyobject_enable_deferred_refcount", pyobject_enable_deferred_refcount, METH_O},
+    {"pyobject_is_unique_temporary", pyobject_is_unique_temporary, METH_O},
     {"test_py_try_inc_ref", test_py_try_inc_ref, METH_NOARGS},
     {"test_xincref_doesnt_leak",test_xincref_doesnt_leak,        METH_NOARGS},
     {"test_incref_doesnt_leak", test_incref_doesnt_leak,         METH_NOARGS},

Objects/object.c

@@ -15,6 +15,7 @@
 #include "pycore_hamt.h"          // _PyHamtItems_Type
 #include "pycore_initconfig.h"    // _PyStatus_OK()
 #include "pycore_instruction_sequence.h" // _PyInstructionSequence_Type
+#include "pycore_interpframe.h"   // _PyFrame_Stackbase()
 #include "pycore_interpolation.h" // _PyInterpolation_Type
 #include "pycore_list.h"          // _PyList_DebugMallocStats()
 #include "pycore_long.h"          // _PyLong_GetZero()
@@ -2621,6 +2622,29 @@ PyUnstable_Object_EnableDeferredRefcount(PyObject *op)
 #endif
 }
 
+int
+PyUnstable_Object_IsUniqueReferencedTemporary(PyObject *op)
+{
+    if (!_PyObject_IsUniquelyReferenced(op)) {
+        return 0;
+    }
+
+    _PyInterpreterFrame *frame = _PyEval_GetFrame();
+    if (frame == NULL) {
+        return 0;
+    }
+
+    _PyStackRef *base = _PyFrame_Stackbase(frame);
+    _PyStackRef *stackpointer = frame->stackpointer;
+    while (stackpointer > base) {
+        stackpointer--;
+        if (op == PyStackRef_AsPyObjectBorrow(*stackpointer)) {
+            return PyStackRef_IsHeapSafe(*stackpointer);
+        }
+    }
+    return 0;
+}
+
 int
 PyUnstable_TryIncRef(PyObject *op)
 {