Skip to content

bpo-43908: Document Static Types in the C API #25710

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 29, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Doc/c-api/type.rst
Original file line number Diff line number Diff line change
Expand Up @@ -111,7 +111,7 @@ Type Objects

.. versionchanged:: 3.10
:c:func:`PyType_GetSlot` can now accept all types.
Previously, it was limited to heap types.
Previously, it was limited to :ref:`heap types <heap-types>`.

.. c:function:: PyObject* PyType_GetModule(PyTypeObject *type)

Expand Down Expand Up @@ -153,7 +153,7 @@ The following functions and structs are used to create

.. c:function:: PyObject* PyType_FromModuleAndSpec(PyObject *module, PyType_Spec *spec, PyObject *bases)

Creates and returns a heap type object from the *spec*
Creates and returns a :ref:`heap type <heap-types>` from the *spec*
(:const:`Py_TPFLAGS_HEAPTYPE`).

The *bases* argument can be used to specify base classes; it can either
Expand Down
99 changes: 58 additions & 41 deletions Doc/c-api/typeobj.rst
Original file line number Diff line number Diff line change
Expand Up @@ -486,12 +486,16 @@ type objects) *must* have the :attr:`ob_size` field.
PyObject* PyObject._ob_prev

These fields are only present when the macro ``Py_TRACE_REFS`` is defined.
Their initialization to ``NULL`` is taken care of by the ``PyObject_HEAD_INIT``
macro. For statically allocated objects, these fields always remain ``NULL``.
For dynamically allocated objects, these two fields are used to link the object
into a doubly-linked list of *all* live objects on the heap. This could be used
for various debugging purposes; currently the only use is to print the objects
that are still alive at the end of a run when the environment variable

Their initialization to ``NULL`` is taken care of by the
``PyObject_HEAD_INIT`` macro. For :ref:`statically allocated objects
<static-types>`, these fields always remain ``NULL``. For :ref:`dynamically
allocated objects <heap-types>`, these two fields are used to link the
object into a doubly-linked list of *all* live objects on the heap.

This could be used for various debugging purposes; currently the only uses
are the :func:`sys.getobjects` function and to print the objects that are
still alive at the end of a run when the environment variable
:envvar:`PYTHONDUMPREFS` is set.

**Inheritance:**
Expand All @@ -502,10 +506,11 @@ type objects) *must* have the :attr:`ob_size` field.
.. c:member:: Py_ssize_t PyObject.ob_refcnt

This is the type object's reference count, initialized to ``1`` by the
``PyObject_HEAD_INIT`` macro. Note that for statically allocated type objects,
the type's instances (objects whose :attr:`ob_type` points back to the type) do
*not* count as references. But for dynamically allocated type objects, the
instances *do* count as references.
``PyObject_HEAD_INIT`` macro. Note that for :ref:`statically allocated type
objects <static-types>`, the type's instances (objects whose :attr:`ob_type`
points back to the type) do *not* count as references. But for
:ref:`dynamically allocated type objects <heap-types>`, the instances *do*
count as references.

**Inheritance:**

Expand Down Expand Up @@ -540,8 +545,9 @@ PyVarObject Slots

.. c:member:: Py_ssize_t PyVarObject.ob_size

For statically allocated type objects, this should be initialized to zero. For
dynamically allocated type objects, this field has a special internal meaning.
For :ref:`statically allocated type objects <static-types>`, this should be
initialized to zero. For :ref:`dynamically allocated type objects
<heap-types>`, this field has a special internal meaning.

**Inheritance:**

Expand All @@ -566,11 +572,13 @@ and :c:type:`PyType_Type` effectively act as defaults.)
:class:`T` defined in module :mod:`M` in subpackage :mod:`Q` in package :mod:`P`
should have the :c:member:`~PyTypeObject.tp_name` initializer ``"P.Q.M.T"``.

For dynamically allocated type objects, this should just be the type name, and
For :ref:`dynamically allocated type objects <heap-types>`,
this should just be the type name, and
the module name explicitly stored in the type dict as the value for key
``'__module__'``.

For statically allocated type objects, the tp_name field should contain a dot.
For :ref:`statically allocated type objects <static-types>`,
the *tp_name* field should contain a dot.
Everything before the last dot is made accessible as the :attr:`__module__`
attribute, and everything after the last dot is made accessible as the
:attr:`~definition.__name__` attribute.
Expand Down Expand Up @@ -725,7 +733,7 @@ and :c:type:`PyType_Type` effectively act as defaults.)
always inherited. If it's not, then the subclass won't use
:ref:`vectorcall <vectorcall>`, except when
:c:func:`PyVectorcall_Call` is explicitly called.
This is in particular the case for `heap types`_
This is in particular the case for :ref:`heap types <heap-types>`
(including subclasses defined in Python).


Expand Down Expand Up @@ -1116,7 +1124,7 @@ and :c:type:`PyType_Type` effectively act as defaults.)

**Inheritance:**

This flag is never inherited by heap types.
This flag is never inherited by :ref:`heap types <heap-types>`.
For extension types, it is inherited whenever
:c:member:`~PyTypeObject.tp_descr_get` is inherited.

Expand Down Expand Up @@ -1163,9 +1171,9 @@ and :c:type:`PyType_Type` effectively act as defaults.)

**Inheritance:**

This bit is inherited for *static* subtypes if
This bit is inherited for :ref:`static subtypes <static-types>` if
:c:member:`~PyTypeObject.tp_call` is also inherited.
`Heap types`_ do not inherit ``Py_TPFLAGS_HAVE_VECTORCALL``.
:ref:`Heap types <heap-types>` do not inherit ``Py_TPFLAGS_HAVE_VECTORCALL``.

.. versionadded:: 3.9

Expand All @@ -1181,7 +1189,8 @@ and :c:type:`PyType_Type` effectively act as defaults.)

This bit is set for type objects that are immutable: type attributes cannot be set nor deleted.

:c:func:`PyType_Ready` automatically applies this flag to static types.
:c:func:`PyType_Ready` automatically applies this flag to
:ref:`static types <static-types>`.

**Inheritance:**

Expand Down Expand Up @@ -1250,9 +1259,8 @@ and :c:type:`PyType_Type` effectively act as defaults.)
:c:func:`local_traverse` to have these specific names; don't name them just
anything.

Heap-allocated types (:const:`Py_TPFLAGS_HEAPTYPE`, such as those created
with :c:func:`PyType_FromSpec` and similar APIs) hold a reference to their
type. Their traversal function must therefore either visit
Instances of :ref:`heap-allocated types <heap-types>` hold a reference to
their type. Their traversal function must therefore either visit
:c:func:`Py_TYPE(self) <Py_TYPE>`, or delegate this responsibility by
calling ``tp_traverse`` of another heap-allocated type (such as a
heap-allocated superclass).
Expand Down Expand Up @@ -1667,8 +1675,8 @@ and :c:type:`PyType_Type` effectively act as defaults.)

**Default:**

This slot has no default. For static types, if the field is
``NULL`` then no :attr:`__dict__` gets created for instances.
This slot has no default. For :ref:`static types <static-types>`, if the
field is ``NULL`` then no :attr:`__dict__` gets created for instances.


.. c:member:: initproc PyTypeObject.tp_init
Expand Down Expand Up @@ -1703,7 +1711,7 @@ and :c:type:`PyType_Type` effectively act as defaults.)

**Default:**

For static types this field does not have a default.
For :ref:`static types <static-types>` this field does not have a default.


.. c:member:: allocfunc PyTypeObject.tp_alloc
Expand Down Expand Up @@ -1754,14 +1762,15 @@ and :c:type:`PyType_Type` effectively act as defaults.)

**Inheritance:**

This field is inherited by subtypes, except it is not inherited by static types
whose :c:member:`~PyTypeObject.tp_base` is ``NULL`` or ``&PyBaseObject_Type``.
This field is inherited by subtypes, except it is not inherited by
:ref:`static types <static-types>` whose :c:member:`~PyTypeObject.tp_base`
is ``NULL`` or ``&PyBaseObject_Type``.

**Default:**

For static types this field has no default. This means if the
slot is defined as ``NULL``, the type cannot be called to create new
instances; presumably there is some other way to create
For :ref:`static types <static-types>` this field has no default.
This means if the slot is defined as ``NULL``, the type cannot be called
to create new instances; presumably there is some other way to create
instances, like a factory function.


Expand Down Expand Up @@ -1803,7 +1812,7 @@ and :c:type:`PyType_Type` effectively act as defaults.)

(The only example of this are types themselves. The metatype,
:c:data:`PyType_Type`, defines this function to distinguish between statically
and dynamically allocated types.)
and :ref:`dynamically allocated types <heap-types>`.)

**Inheritance:**

Expand Down Expand Up @@ -1949,10 +1958,10 @@ objects on the thread which called tp_dealloc will not violate any assumptions
of the library.


.. _heap-types:
.. _static-types:

Heap Types
----------
Static Types
------------

Traditionally, types defined in C code are *static*, that is,
a static :c:type:`PyTypeObject` structure is defined directly in code
Expand All @@ -1972,12 +1981,20 @@ Also, since :c:type:`PyTypeObject` is not part of the :ref:`stable ABI <stable>`
any extension modules using static types must be compiled for a specific
Python minor version.

An alternative to static types is *heap-allocated types*, or *heap types*
for short, which correspond closely to classes created by Python's
``class`` statement.

.. _heap-types:

Heap Types
----------

An alternative to :ref:`static types <static-types>` is *heap-allocated types*,
or *heap types* for short, which correspond closely to classes created by
Python's ``class`` statement. Heap types have the :const:`Py_TPFLAGS_HEAPTYPE`
flag set.

This is done by filling a :c:type:`PyType_Spec` structure and calling
:c:func:`PyType_FromSpecWithBases`.
:c:func:`PyType_FromSpec`, :c:func:`PyType_FromSpecWithBases`,
or :c:func:`PyType_FromModuleAndSpec`.


.. _number-structs:
Expand Down Expand Up @@ -2489,7 +2506,7 @@ include common usage you may encounter. Some demonstrate tricky corner
cases. For more examples, practical info, and a tutorial, see
:ref:`defining-new-types` and :ref:`new-types-topics`.

A basic static type::
A basic :ref:`static type <static-types>`::

typedef struct {
PyObject_HEAD
Expand Down Expand Up @@ -2596,7 +2613,7 @@ to create instances (e.g. uses a separate factory func)::
.tp_repr = (reprfunc)myobj_repr,
};

The simplest static type (with fixed-length instances)::
The simplest :ref:`static type <static-types>` with fixed-length instances::

typedef struct {
PyObject_HEAD
Expand All @@ -2607,7 +2624,7 @@ The simplest static type (with fixed-length instances)::
.tp_name = "mymod.MyObject",
};

The simplest static type (with variable-length instances)::
The simplest :ref:`static type <static-types>` with variable-length instances::

typedef struct {
PyObject_VAR_HEAD
Expand Down
5 changes: 4 additions & 1 deletion Doc/includes/typestruct.h
Original file line number Diff line number Diff line change
Expand Up @@ -35,12 +35,14 @@ typedef struct _typeobject {

const char *tp_doc; /* Documentation string */

/* Assigned meaning in release 2.0 */
/* call function for all accessible objects */
traverseproc tp_traverse;

/* delete references to contained objects */
inquiry tp_clear;

/* Assigned meaning in release 2.1 */
/* rich comparisons */
richcmpfunc tp_richcompare;

Expand All @@ -55,6 +57,7 @@ typedef struct _typeobject {
struct PyMethodDef *tp_methods;
struct PyMemberDef *tp_members;
struct PyGetSetDef *tp_getset;
// Strong reference on a heap type, borrowed reference on a static type
struct _typeobject *tp_base;
PyObject *tp_dict;
descrgetfunc tp_descr_get;
Expand All @@ -76,5 +79,5 @@ typedef struct _typeobject {
unsigned int tp_version_tag;

destructor tp_finalize;

vectorcallfunc tp_vectorcall;
} PyTypeObject;
3 changes: 2 additions & 1 deletion Doc/whatsnew/3.10.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1668,7 +1668,8 @@ New Features
slot.
(Contributed by Hai Shi in :issue:`41832`.)

* The :c:func:`PyType_GetSlot` function can accept static types.
* The :c:func:`PyType_GetSlot` function can accept
:ref:`static types <static-types>`.
(Contributed by Hai Shi and Petr Viktorin in :issue:`41073`.)

* Add a new :c:func:`PySet_CheckExact` function to the C-API to check if an
Expand Down
5 changes: 3 additions & 2 deletions Doc/whatsnew/3.8.rst
Original file line number Diff line number Diff line change
Expand Up @@ -352,7 +352,8 @@ PEP 590: Vectorcall: a fast calling protocol for CPython
:ref:`vectorcall` is added to the Python/C API.
It is meant to formalize existing optimizations which were already done
for various classes.
Any static type implementing a callable can use this protocol.
Any :ref:`static type <static-types>` implementing a callable can use this
protocol.

This is currently provisional.
The aim is to make it fully public in Python 3.9.
Expand Down Expand Up @@ -2040,7 +2041,7 @@ Changes in the C API
This makes types created through :c:func:`PyType_FromSpec` behave like
other classes in managed code.

Statically allocated types are not affected.
:ref:`Statically allocated types <static-types>` are not affected.

For the vast majority of cases, there should be no side effect.
However, types that manually increase the reference count after allocating
Expand Down
7 changes: 4 additions & 3 deletions Doc/whatsnew/3.9.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1124,7 +1124,7 @@ Changes in the Python API
Changes in the C API
--------------------

* Instances of heap-allocated types (such as those created with
* Instances of :ref:`heap-allocated types <heap-types>` (such as those created with
:c:func:`PyType_FromSpec` and similar APIs) hold a reference to their type
object since Python 3.8. As indicated in the "Changes in the C API" of Python
3.8, for the vast majority of cases, there should be no side effect but for
Expand All @@ -1147,7 +1147,8 @@ Changes in the C API

If your traverse function delegates to ``tp_traverse`` of its base class
(or another type), ensure that ``Py_TYPE(self)`` is visited only once.
Note that only heap types are expected to visit the type in ``tp_traverse``.
Note that only :ref:`heap type <heap-types>` are expected to visit the type
in ``tp_traverse``.

For example, if your ``tp_traverse`` function includes:

Expand All @@ -1160,7 +1161,7 @@ Changes in the C API
.. code-block:: c

#if PY_VERSION_HEX >= 0x03090000
// This was not needed before Python 3.9 (Python issue 35810 and 40217)
// This was not needed before Python 3.9 (bpo-35810 and bpo-40217)
if (base->tp_flags & Py_TPFLAGS_HEAPTYPE) {
// a heap type's tp_traverse already visited Py_TYPE(self)
} else {
Expand Down
2 changes: 2 additions & 0 deletions Include/cpython/object.h
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,8 @@ typedef struct {
* backwards-compatibility */
typedef Py_ssize_t printfunc;

// If this structure is modified, Doc/includes/typestruct.h should be updated
// as well.
struct _typeobject {
PyObject_VAR_HEAD
const char *tp_name; /* For printing, in format "<module>.<name>" */
Expand Down