Skip to content

use Py_REFCNT(obj) instead of obj->ob_refcnt (allows building in nogil) #201

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

jimkring
Copy link

@jimkring jimkring commented Sep 6, 2023

here’s a suggestion from @colesbury:

This looks like it needs to use Py_REFCNT(obj) instead of obj->ob_refcnt. The reference count field changed (it's now two fields in "nogil Python), so you can't access them directly. The Python docs now say to use Py_REFCNT instead of ->ob_refcnt so maybe we can get this change upstreamed.

use Py_REFCNT(obj) instead of obj->ob_refcnt
@nascheme
Copy link

I think assuming refcnt == 1 means that it's safe to mutate it is not safe. It's not safe in both the free-threaded and the regular build. I don't know how much performance would be lost by disabling this. You might check that Python version is < 3.13 since the upcoming release will have more ways to break this assumption (was already broken in <= 3.12, I think).

There is a new API being proposed for CPython:
python/cpython#133170

@colesbury
Copy link

colesbury commented Apr 30, 2025

The Py_REFCNT(obj) == 1 check is fine here. There obj comes from a PyBytes_FromStringAndSize() call. There are only two cases:

  • PyBytes_FromStringAndSize() returns a new bytes object with refcount of 1. Nobody else has access to it, so it's safe to resize in-place.
  • PyBytes_FromStringAndSize() returns an interned, immortal bytes object. The refcount is not one (it's some large value) and we don't resize it.

There's no concern about racing accesses to the refcount here in the free threading build. It's a lot more straightforward than the NumPy because the PyBytesObject is coming from python-zstandard code, not from some Python call.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants