Skip to content

Commit aa2ecef

Browse files
[3.12] gh-98154: Clarify Usage of "Reference Count" In the Docs (gh-107552) (#107752)
* gh-98154: Clarify Usage of "Reference Count" In the Docs (gh-107552) PEP 683 (immortal objects) revealed some ways in which the Python documentation has been unnecessarily coupled to the implementation details of reference counts. In the end users should focus on reference ownership, including taking references and releasing them, rather than on how many reference counts an object has. This change updates the documentation to reflect that perspective. It also updates the docs relative to immortal objects in a handful of places. (cherry picked from commit 5dc825d) Co-authored-by: Eric Snow <[email protected]> * Fix a typo. --------- Co-authored-by: Eric Snow <[email protected]>
1 parent e5582bd commit aa2ecef

15 files changed

+162
-90
lines changed

Doc/c-api/allocation.rst

+7-6
Original file line numberDiff line numberDiff line change
@@ -29,12 +29,13 @@ Allocating Objects on the Heap
2929
3030
.. c:macro:: PyObject_New(TYPE, typeobj)
3131
32-
Allocate a new Python object using the C structure type *TYPE* and the
33-
Python type object *typeobj* (``PyTypeObject*``).
34-
Fields not defined by the Python object header
35-
are not initialized; the object's reference count will be one. The size of
36-
the memory allocation is determined from the :c:member:`~PyTypeObject.tp_basicsize` field of
37-
the type object.
32+
Allocate a new Python object using the C structure type *TYPE*
33+
and the Python type object *typeobj* (``PyTypeObject*``).
34+
Fields not defined by the Python object header are not initialized.
35+
The caller will own the only reference to the object
36+
(i.e. its reference count will be one).
37+
The size of the memory allocation is determined from the
38+
:c:member:`~PyTypeObject.tp_basicsize` field of the type object.
3839
3940
4041
.. c:macro:: PyObject_NewVar(TYPE, typeobj, size)

Doc/c-api/arg.rst

+11-6
Original file line numberDiff line numberDiff line change
@@ -293,8 +293,10 @@ Other objects
293293

294294
``O`` (object) [PyObject \*]
295295
Store a Python object (without any conversion) in a C object pointer. The C
296-
program thus receives the actual object that was passed. The object's reference
297-
count is not increased. The pointer stored is not ``NULL``.
296+
program thus receives the actual object that was passed. A new
297+
:term:`strong reference` to the object is not created
298+
(i.e. its reference count is not increased).
299+
The pointer stored is not ``NULL``.
298300

299301
``O!`` (object) [*typeobject*, PyObject \*]
300302
Store a Python object in a C object pointer. This is similar to ``O``, but
@@ -378,7 +380,8 @@ inside nested parentheses. They are:
378380
mutually exclude each other.
379381

380382
Note that any Python object references which are provided to the caller are
381-
*borrowed* references; do not decrement their reference count!
383+
*borrowed* references; do not release them
384+
(i.e. do not decrement their reference count)!
382385

383386
Additional arguments passed to these functions must be addresses of variables
384387
whose type is determined by the format string; these are used to store values
@@ -613,8 +616,10 @@ Building values
613616
Convert a C :c:type:`Py_complex` structure to a Python complex number.
614617
615618
``O`` (object) [PyObject \*]
616-
Pass a Python object untouched (except for its reference count, which is
617-
incremented by one). If the object passed in is a ``NULL`` pointer, it is assumed
619+
Pass a Python object untouched but create a new
620+
:term:`strong reference` to it
621+
(i.e. its reference count is incremented by one).
622+
If the object passed in is a ``NULL`` pointer, it is assumed
618623
that this was caused because the call producing the argument found an error and
619624
set an exception. Therefore, :c:func:`Py_BuildValue` will return ``NULL`` but won't
620625
raise an exception. If no exception has been raised yet, :exc:`SystemError` is
@@ -624,7 +629,7 @@ Building values
624629
Same as ``O``.
625630
626631
``N`` (object) [PyObject \*]
627-
Same as ``O``, except it doesn't increment the reference count on the object.
632+
Same as ``O``, except it doesn't create a new :term:`strong reference`.
628633
Useful when the object is created by a call to an object constructor in the
629634
argument list.
630635

Doc/c-api/buffer.rst

+5-2
Original file line numberDiff line numberDiff line change
@@ -102,7 +102,9 @@ a buffer, see :c:func:`PyObject_GetBuffer`.
102102
.. c:member:: PyObject *obj
103103
104104
A new reference to the exporting object. The reference is owned by
105-
the consumer and automatically decremented and set to ``NULL`` by
105+
the consumer and automatically released
106+
(i.e. reference count decremented)
107+
and set to ``NULL`` by
106108
:c:func:`PyBuffer_Release`. The field is the equivalent of the return
107109
value of any standard C-API function.
108110

@@ -454,7 +456,8 @@ Buffer-related functions
454456
455457
.. c:function:: void PyBuffer_Release(Py_buffer *view)
456458
457-
Release the buffer *view* and decrement the reference count for
459+
Release the buffer *view* and release the :term:`strong reference`
460+
(i.e. decrement the reference count) to the view's supporting object,
458461
``view->obj``. This function MUST be called when the buffer
459462
is no longer being used, otherwise reference leaks may occur.
460463

Doc/c-api/bytes.rst

+2-2
Original file line numberDiff line numberDiff line change
@@ -184,8 +184,8 @@ called with a non-bytes parameter.
184184
.. c:function:: void PyBytes_ConcatAndDel(PyObject **bytes, PyObject *newpart)
185185
186186
Create a new bytes object in *\*bytes* containing the contents of *newpart*
187-
appended to *bytes*. This version decrements the reference count of
188-
*newpart*.
187+
appended to *bytes*. This version releases the :term:`strong reference`
188+
to *newpart* (i.e. decrements its reference count).
189189
190190
191191
.. c:function:: int _PyBytes_Resize(PyObject **bytes, Py_ssize_t newsize)

Doc/c-api/exceptions.rst

+2-1
Original file line numberDiff line numberDiff line change
@@ -110,7 +110,8 @@ For convenience, some of these functions will always return a
110110
111111
This is the most common way to set the error indicator. The first argument
112112
specifies the exception type; it is normally one of the standard exceptions,
113-
e.g. :c:data:`PyExc_RuntimeError`. You need not increment its reference count.
113+
e.g. :c:data:`PyExc_RuntimeError`. You need not create a new
114+
:term:`strong reference` to it (e.g. with :c:func:`Py_INCREF`).
114115
The second argument is an error message; it is decoded from ``'utf-8'``.
115116
116117

Doc/c-api/init_config.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -1109,7 +1109,7 @@ PyConfig
11091109
11101110
.. c:member:: int show_ref_count
11111111
1112-
Show total reference count at exit?
1112+
Show total reference count at exit (excluding immortal objects)?
11131113
11141114
Set to ``1`` by :option:`-X showrefcount <-X>` command line option.
11151115

Doc/c-api/intro.rst

+33-26
Original file line numberDiff line numberDiff line change
@@ -287,60 +287,67 @@ true if (and only if) the object pointed to by *a* is a Python list.
287287
Reference Counts
288288
----------------
289289

290-
The reference count is important because today's computers have a finite (and
291-
often severely limited) memory size; it counts how many different places there
292-
are that have a reference to an object. Such a place could be another object,
293-
or a global (or static) C variable, or a local variable in some C function.
294-
When an object's reference count becomes zero, the object is deallocated. If
295-
it contains references to other objects, their reference count is decremented.
296-
Those other objects may be deallocated in turn, if this decrement makes their
297-
reference count become zero, and so on. (There's an obvious problem with
298-
objects that reference each other here; for now, the solution is "don't do
299-
that.")
290+
The reference count is important because today's computers have a finite
291+
(and often severely limited) memory size; it counts how many different
292+
places there are that have a :term:`strong reference` to an object.
293+
Such a place could be another object, or a global (or static) C variable,
294+
or a local variable in some C function.
295+
When the last :term:`strong reference` to an object is released
296+
(i.e. its reference count becomes zero), the object is deallocated.
297+
If it contains references to other objects, those references are released.
298+
Those other objects may be deallocated in turn, if there are no more
299+
references to them, and so on. (There's an obvious problem with
300+
objects that reference each other here; for now, the solution
301+
is "don't do that.")
300302

301303
.. index::
302304
single: Py_INCREF()
303305
single: Py_DECREF()
304306

305-
Reference counts are always manipulated explicitly. The normal way is to use
306-
the macro :c:func:`Py_INCREF` to increment an object's reference count by one,
307-
and :c:func:`Py_DECREF` to decrement it by one. The :c:func:`Py_DECREF` macro
307+
Reference counts are always manipulated explicitly. The normal way is
308+
to use the macro :c:func:`Py_INCREF` to take a new reference to an
309+
object (i.e. increment its reference count by one),
310+
and :c:func:`Py_DECREF` to release that reference (i.e. decrement the
311+
reference count by one). The :c:func:`Py_DECREF` macro
308312
is considerably more complex than the incref one, since it must check whether
309313
the reference count becomes zero and then cause the object's deallocator to be
310-
called. The deallocator is a function pointer contained in the object's type
311-
structure. The type-specific deallocator takes care of decrementing the
312-
reference counts for other objects contained in the object if this is a compound
314+
called. The deallocator is a function pointer contained in the object's type
315+
structure. The type-specific deallocator takes care of releasing references
316+
for other objects contained in the object if this is a compound
313317
object type, such as a list, as well as performing any additional finalization
314318
that's needed. There's no chance that the reference count can overflow; at
315319
least as many bits are used to hold the reference count as there are distinct
316320
memory locations in virtual memory (assuming ``sizeof(Py_ssize_t) >= sizeof(void*)``).
317321
Thus, the reference count increment is a simple operation.
318322

319-
It is not necessary to increment an object's reference count for every local
320-
variable that contains a pointer to an object. In theory, the object's
323+
It is not necessary to hold a :term:`strong reference` (i.e. increment
324+
the reference count) for every local variable that contains a pointer
325+
to an object. In theory, the object's
321326
reference count goes up by one when the variable is made to point to it and it
322327
goes down by one when the variable goes out of scope. However, these two
323328
cancel each other out, so at the end the reference count hasn't changed. The
324329
only real reason to use the reference count is to prevent the object from being
325330
deallocated as long as our variable is pointing to it. If we know that there
326331
is at least one other reference to the object that lives at least as long as
327-
our variable, there is no need to increment the reference count temporarily.
332+
our variable, there is no need to take a new :term:`strong reference`
333+
(i.e. increment the reference count) temporarily.
328334
An important situation where this arises is in objects that are passed as
329335
arguments to C functions in an extension module that are called from Python;
330336
the call mechanism guarantees to hold a reference to every argument for the
331337
duration of the call.
332338

333339
However, a common pitfall is to extract an object from a list and hold on to it
334-
for a while without incrementing its reference count. Some other operation might
335-
conceivably remove the object from the list, decrementing its reference count
340+
for a while without taking a new reference. Some other operation might
341+
conceivably remove the object from the list, releasing that reference,
336342
and possibly deallocating it. The real danger is that innocent-looking
337343
operations may invoke arbitrary Python code which could do this; there is a code
338344
path which allows control to flow back to the user from a :c:func:`Py_DECREF`, so
339345
almost any operation is potentially dangerous.
340346

341347
A safe approach is to always use the generic operations (functions whose name
342348
begins with ``PyObject_``, ``PyNumber_``, ``PySequence_`` or ``PyMapping_``).
343-
These operations always increment the reference count of the object they return.
349+
These operations always create a new :term:`strong reference`
350+
(i.e. increment the reference count) of the object they return.
344351
This leaves the caller with the responsibility to call :c:func:`Py_DECREF` when
345352
they are done with the result; this soon becomes second nature.
346353

@@ -356,7 +363,7 @@ to objects (objects are not owned: they are always shared). "Owning a
356363
reference" means being responsible for calling Py_DECREF on it when the
357364
reference is no longer needed. Ownership can also be transferred, meaning that
358365
the code that receives ownership of the reference then becomes responsible for
359-
eventually decref'ing it by calling :c:func:`Py_DECREF` or :c:func:`Py_XDECREF`
366+
eventually releasing it by calling :c:func:`Py_DECREF` or :c:func:`Py_XDECREF`
360367
when it's no longer needed---or passing on this responsibility (usually to its
361368
caller). When a function passes ownership of a reference on to its caller, the
362369
caller is said to receive a *new* reference. When no ownership is transferred,
@@ -414,9 +421,9 @@ For example, the above two blocks of code could be replaced by the following
414421

415422
It is much more common to use :c:func:`PyObject_SetItem` and friends with items
416423
whose references you are only borrowing, like arguments that were passed in to
417-
the function you are writing. In that case, their behaviour regarding reference
418-
counts is much saner, since you don't have to increment a reference count so you
419-
can give a reference away ("have it be stolen"). For example, this function
424+
the function you are writing. In that case, their behaviour regarding references
425+
is much saner, since you don't have to take a new reference just so you
426+
can give that reference away ("have it be stolen"). For example, this function
420427
sets all items of a list (actually, any mutable sequence) to a given item::
421428

422429
int

Doc/c-api/module.rst

+1-1
Original file line numberDiff line numberDiff line change
@@ -529,7 +529,7 @@ state:
529529
.. note::
530530
531531
Unlike other functions that steal references, ``PyModule_AddObject()``
532-
only decrements the reference count of *value* **on success**.
532+
only releases the reference to *value* **on success**.
533533
534534
This means that its return value must be checked, and calling code must
535535
:c:func:`Py_DECREF` *value* manually on error.

Doc/c-api/object.rst

+7-6
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@ Object Protocol
1515
.. c:macro:: Py_RETURN_NOTIMPLEMENTED
1616
1717
Properly handle returning :c:data:`Py_NotImplemented` from within a C
18-
function (that is, increment the reference count of NotImplemented and
19-
return it).
18+
function (that is, create a new :term:`strong reference`
19+
to NotImplemented and return it).
2020

2121

2222
.. c:function:: int PyObject_Print(PyObject *o, FILE *fp, int flags)
@@ -320,11 +320,12 @@ Object Protocol
320320
321321
When *o* is non-``NULL``, returns a type object corresponding to the object type
322322
of object *o*. On failure, raises :exc:`SystemError` and returns ``NULL``. This
323-
is equivalent to the Python expression ``type(o)``. This function increments the
324-
reference count of the return value. There's really no reason to use this
323+
is equivalent to the Python expression ``type(o)``.
324+
This function creates a new :term:`strong reference` to the return value.
325+
There's really no reason to use this
325326
function instead of the :c:func:`Py_TYPE()` function, which returns a
326-
pointer of type :c:expr:`PyTypeObject*`, except when the incremented reference
327-
count is needed.
327+
pointer of type :c:expr:`PyTypeObject*`, except when a new
328+
:term:`strong reference` is needed.
328329
329330
330331
.. c:function:: int PyObject_TypeCheck(PyObject *o, PyTypeObject *type)

0 commit comments

Comments
 (0)