[BUG] Wheel ABI tag is wrong when building with debug build of Python 3.13 on Windows

[adang1345]

setuptools version

75.1.0

Python version

Python 3.13.0

OS

Windows

Additional environment information

No response

Description

If I execute python_d.exe setup.py bdist_wheel using the debug build of Python 3.13 on Windows, then the wheel that is produced has a name that looks like simpleext-0.0.1-cp313-cp313-win_amd64.whl. The ABI tag is set to cp313 instead of the expected value of cp313d. If I execute the same command using the debug build of Python 3.12, the ABI tag is correctly set to cp312d, producing a wheel filename like simpleext-0.0.1-cp312-cp312d-win_amd64.whl.

The cause of this problem is that Python 3.13 on Windows now sets a value for the SOABI configuration variable, while previous Python versions did not. Debug and release builds of Python 3.13 on Windows have the same SOABI value, and the following line does not take this into account.

setuptools/setuptools/command/bdist_wheel.py

Line 128 in 66a8aee

abi = "cp" + soabi.split("-")[1]

I propose that we modify this section to

    elif soabi and impl == "cp" and soabi.startswith("cp"):
        # Windows
        abi = soabi.split("-")[0]
        if hasattr(sys, "gettotalrefcount"):
            # using debug build; append "d" flag
            abi += "d"

Expected behavior

The ABI tag is set to cp313d, producing a wheel filename like simpleext-0.0.1-cp313-cp313d-win_amd64.whl.

How to Reproduce

Install 64-bit Python 3.13.0 on Windows with the "Download debug binaries" option enabled. Create a file setup.py with the following contents.

from setuptools import setup, Extension
import os

setup(
    name='simpleext',
    version='0.0.1',
    description='Simple extension module',
    zip_safe=False,
    ext_modules=[Extension(
        'simpleext', ['simpleext.c'],
    )]
)

In the same directory, create a file simpleext.c with the following contents.

#define PY_SSIZE_T_CLEAN
#include <Python.h>

static PyObject *simpleext_donothing(PyObject *self, PyObject *args)
{
   Py_RETURN_NONE;
}

static PyMethodDef SimpleExtMethods[] = {
    {"donothing", simpleext_donothing, METH_NOARGS, NULL},
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef simpleextmodule = {
    PyModuleDef_HEAD_INIT,
    "simpleext",
    NULL,
    -1,
    SimpleExtMethods
};

PyMODINIT_FUNC PyInit_simpleext(void)
{
    return PyModule_Create(&simpleextmodule);
}

Execute python_d.exe setup.py bdist_wheel. Note that the wheel file that is created is named simpleext-0.0.1-cp313-cp313-win_amd64.whl.

Output

running bdist_wheel
running build
running build_ext
building 'simpleext' extension
"C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\bin\HostX86\x64\cl.exe" /c /nologo /O2 /W3 /GL /DNDEBUG /MD "-IC:\Program Files\Python313\include" "-IC:\Program Files\Python313\Include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\ATLMFC\include" "-IC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include" "-IC:\Program Files (x86)\Windows Kits\10\include\10.0.22621.0\ucrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\um" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\shared" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\winrt" "-IC:\Program Files (x86)\Windows Kits\10\\include\10.0.22621.0\\cppwinrt" "-IC:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um" /TcC:\Users\adang\Desktop\simpleext\simpleext.c /Fo\Users\adang\Desktop\simpleext\simpleext.obj
simpleext.c
creating C:\Users\adang\Desktop\simpleext\build\lib.win-amd64-cpython-313-pydebug
"C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\bin\HostX86\x64\link.exe" /nologo /INCREMENTAL:NO /LTCG /DLL /MANIFEST:EMBED,ID=2 /MANIFESTUAC:NO "/LIBPATH:C:\Program Files\Python313\libs" "/LIBPATH:C:\Program Files\Python313" "/LIBPATH:C:\Program Files\Python313\PCbuild\amd64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\ATLMFC\lib\x64" "/LIBPATH:C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.41.34120\lib\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\lib\um\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\lib\10.0.22621.0\ucrt\x64" "/LIBPATH:C:\Program Files (x86)\Windows Kits\10\\lib\10.0.22621.0\\um\x64" /EXPORT:PyInit_simpleext \Users\adang\Desktop\simpleext\simpleext.obj /OUT:build\lib.win-amd64-cpython-313-pydebug\simpleext_d.cp313-win_amd64.pyd /IMPLIB:\Users\adang\Desktop\simpleext\simpleext_d.cp313-win_amd64.lib
   Creating library \Users\adang\Desktop\simpleext\simpleext_d.cp313-win_amd64.lib and object \Users\adang\Desktop\simpleext\simpleext_d.cp313-win_amd64.exp
Generating code
Finished generating code
C:\Program Files\Python313\Lib\site-packages\setuptools\_distutils\cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
installing to build\bdist.win-amd64\wheel
running install
running install_lib
creating build\bdist.win-amd64\wheel
copying build\lib.win-amd64-cpython-313-pydebug\simpleext_d.cp313-win_amd64.pyd -> build\bdist.win-amd64\wheel\.
running install_egg_info
running egg_info
creating simpleext.egg-info
writing simpleext.egg-info\PKG-INFO
writing dependency_links to simpleext.egg-info\dependency_links.txt
writing top-level names to simpleext.egg-info\top_level.txt
writing manifest file 'simpleext.egg-info\SOURCES.txt'
reading manifest file 'simpleext.egg-info\SOURCES.txt'
writing manifest file 'simpleext.egg-info\SOURCES.txt'
Copying simpleext.egg-info to build\bdist.win-amd64\wheel\.\simpleext-0.0.1-py3.13.egg-info
running install_scripts
creating build\bdist.win-amd64\wheel\simpleext-0.0.1.dist-info\WHEEL
creating 'dist\simpleext-0.0.1-cp313-cp313-win_amd64.whl' and adding 'build\bdist.win-amd64\wheel' to it
adding 'simpleext_d.cp313-win_amd64.pyd'
adding 'simpleext-0.0.1.dist-info/METADATA'
adding 'simpleext-0.0.1.dist-info/WHEEL'
adding 'simpleext-0.0.1.dist-info/top_level.txt'
adding 'simpleext-0.0.1.dist-info/RECORD'
removing build\bdist.win-amd64\wheel

[abravalheri]

Thank you very much @adang1345 for having a look on this.
Were the CPython devs notified about the value of SOABI not having the d parameter in this circumstance? What would be their advice (we can merge the PR - thank you very much - but maybe this is something that should also be present in the value of SOABI?)

[adang1345]

I asked about this at https://discuss.python.org/t/soabi-for-debug-version-of-python-3-13-on-windows/66987 but didn't get a response. I'll ask again on the CPython bug tracker and see whether someone responds there.

[abravalheri]

Thank you very much @adang1345, let's see what they advise. If they plan to tackle this in a future release we might have to change the PR to also check for duplication (together with a TODO note for removal once an specific version of Python becomes the minimum requirement)

[adang1345]

CPython developers have said that the lack of a d flag is intentional and is not something that will be changed: python/cpython#125591

[abravalheri]

Thank you very much @adang1345 for having a look on this.

So if I understood correctly, the CPython conversation:

1. The correct wheel nomenclature for this case is not to append a d flag to the cpXXX, but actually change the base name of the file and append _d.
2. They also said that debug builds do not make much sense unless debugging CPython itself:

   Using a debug build of CPython is only really necessary to debug CPython itself. If you believe that CPython does not contain the bug, you don't need its debug build.

3. The last piece of information is that cp312d seems to be simply ignored by the Python interpreter and other tools, so apparently it is a "innocuous" change.

Now, there are a couple of options on what to do with this information:

1. Try to use the _d naming convention as mentioned by CPython devs.
   I am not 100% that the setuptools code-base will allow that without an overhaul.
   Also, I don't know how other tools will react to this change (PyPI, pip, etc...), so I am a bit worried of pursuing it.
2. Pretend Windows follows the same convention as Linux, and implement the suggested change in PR Fix wheel ABI tag for debug Python 3.13 on Windows #4676 (motivated by the fact that the change seems to be "innocuous").
3. Do nothing, and not include the d flag (motivated by CPython's comments on it not being that useful).

I have a couple of questions, consider the Windows wheel as they are built today when you use python_d.exe: simpleext-0.0.1-cp313-cp313-win_amd64.whl. @adang1345, when you install this wheel as it is, can you still import the modules normally and work with them? Can you run the same modules/wheel installation in python.exe even when they were built with python_d.exe? What happens if you manually rename the relevant files (e.g. wheel, maybe dlls and pyds) to have the cp313d as requested, will they still work properly with python_d.exe/python.exe?

@jaraco do you have any thoughts on the matter?

[adang1345]

Your understanding of the CPython conversation is not quite right; let me clarify and add a few points.

1. Appending _d to the name of the extension module and having a d flag in the wheel ABI tag have two different effects. Appending _d to the name of the extension module affects whether it can be loaded using the import statement; it can be loaded with python_d.exe but not python.exe. setuptools can already do this, executing python_d.exe setup.py build_ext --debug currently appends _d to the end of extension modules and builds them using the CPython debug ABI. Having a d flag in the wheel ABI tag affects whether the wheel can be installed; if this flag is present, then the wheel can be installed using python_d.exe -m pip install but not python.exe -m pip install.
2. In most use cases, debug builds are meant for debugging Python itself. However, I have a use case where I want an extension module to link to a C++ DLL that is built in debug mode, which requires me to build the extension module in debug mode as well to adhere to C++ ABI restrictions. I understand that this is not a common use case and I could probably make changes at my end to avoid doing this.
3. The cp312d is not ignored. When d is present in the ABI tag, this ensures that the wheel can be installed only when running python_d.exe -m pip install and not python.exe -m pip install.

I took another look at my current workflow, and I'm actually calling python_d.exe setup.py build_ext --debug bdist_wheel to create the wheel.

• With Python 3.12, this creates a wheel named simpleext-0.0.1-cp312-cp312d-win_amd64.whl containing the extension module named simpleext_d.cp312-win_amd64.pyd. I can install the wheel with python_d.exe -m pip install but not python.exe -m pip install. I can run import simpleext with python_d.exe but not python.exe.
• With Python 3.13, this creates a wheel named simpleext-0.0.1-cp313-cp313-win_amd64.whl containing the extension module named simpleext_d.cp313-win_amd64.pyd. I can install the wheel with python_d.exe -m pip install or python.exe -m pip install. I can run import simpleext with python_d.exe but not python.exe; this part has not changed.

So practically speaking, the only thing that the d flag in the wheel filename does is tell pip that the CPython debug ABI is required, which is why pip must be running in a python_d.exe process in order to install a wheel with this flag.

[abravalheri]

Ah ok, thank you very much for explaining.

So it should be fine if we preserve the behaviour we see in 3.12, via the provided PR (since the major point of interaction here is pip not the import system).

[abravalheri]

I will probably release something next week (I try to avoid cutting a release over the weekend),