Please consider undeprecating setup_requires

I have run into an issue with the deprecation of setup_requires: the recommended alternative appears to be PEP 517/build-system.requires, but that comes with its own issues:

• apparently it breaks wheels unless you use special options like --no-binary. This breaks very common tools like tox which no longer can install a project if this package is a dependency - also, why would you want to not have wheels? this is wrong, my apologies

• setup_requires isn't installed when invoking hooks.get_requires_for_build_wheel() as called by pip/pep517. However the new build-system.requires is. In practice this is an issue when putting a Cython project as a build dependency for .pxd imports: a .pxd dep is only actually required when building the wheel (where setup_requires will provide it) but not for basic package analysis (where build-system.requires per design provides it too, setup_requires won't). In practice since "providing it" means building the whole package, as a result obtaining any package metadata is greatly slowed down

• pyproject.tomls build-system.requires is declarative but not exhaustive. the problem with that is that the dependencies are then spread over multiple places, e.g. no longer just requirements.txt parsed by setup.py - which can't be parsed similarly in the declarative pyproject.toml file with its static build_system.requires

Edit: a lot of this was initially quite misinformed so I corrected the reasons

EDIT FROM THE FUTURE: this part is somewhat irrelevant because I just accidentally broke something (so fully my mistake for the distraction, apologies 🙈 ) jump down here for remaining issue: #1742 (comment)

---

By the way since setup_requires is deprecated and therefore apparently it still uses easy_install because nobody fixed that (I suppose makes sense if it's deprecated) it doesn't work with Cython(!) because it breaks it with its sandbox: cython/cython#2730

But as written above, using pep518 breaks tox! And just putting it into install_requires works for all pip invocations outside I've seen, but also breaks in tox because through some random chance pip there decides to build the project itself before its deps which of course doesn't work if .pxd imports are needed (that requires the deps to be present in site-packages)

So as a result it appears it is currently impossible to rely on a dependency for .pxd cross-package import and have that project as a dependency for another that builds in tox. Isn't that a major issue? How is any of this supposed to work? All I'm seeing is two mechanisms, one deprecated and unable to deal with Cython, the other having really weird restrictions that break tox and wheels and whatnot, and now I can choose between the devil and the deep blue sea!

Edit: meant pep518 ("build-system".requires) and not pep517 (library)

Edit 2: most of this was wrong, hence the correction 😄

[benoit-pierre]

The alternative to setup_requires is PEP 518, not necessarily PEP 517. What's the issue with pip's PEP 518/518 support and wheels?

[benoit-pierre]

It would help if you provided a reproducible example of the issue you are running into.

[benoit-pierre]

Assuming the issue is with your nettools package, using https://github.com/JonasT/nettools/archive/a8b9236f985d468e49953f818dad3dcb1bbe2b77.zip with an added minimal pyproject.toml works fine:

[build-system]
requires = ["setuptools", "wheel", "cython"]

That is pip wheel . from a clean virtualenv (with only pip available) successfully build a wheel.

[benoit-pierre]

Note that setuptools won't include pyproject.toml automatically, so you'll need a manifest entry if you want the source distribution to be complete, in MANIFEST.in:

include pyproject.toml

Yes that message appears if you use it in a dep that itself is a build requires dep of your main project in tox. Edit from way later: the message appeared because my package was broken 😄 apologies from future me

[benoit-pierre]

Again, can you point to a reproducible example?

I'll try to. This involves 3 projects with interdependencies and tox, so I'm not sure how quickly I can find out what constellation breaks it in detail. But it seems to be related to how tox builds the deps, and possibly --only-binary and other options it uses to build the wheel

Edit: also, the other two disadvantages I listed still remain. Declarative can be both good but also a huge disadvantage, and having dependencies somewhat arbitrarily split (there is a criteria of course but it's not very transparent) between two files and wildly different formats just is a little ugly

Edit 2: (I just had a broken package. Realized why later, see below!)

The exact error I believe was this @ tox/wheel: Could not build wheels for wobblui which use PEP 517 and cannot be installed directly.

@benoit-pierre Okay, so my MANIFEST.in was wrong and this lead to problems! Now I fixed that, and it almost works, but I get a really strange error on a dependent package and basically it doesn't work for some reason:

pypa/packaging-problems#252
(this issue now breaks it both inside and outside of tox)

But I suppose my ticket here wasn't that constructive given the wheel thing was just wrong, and the other points are admittedly a bit personal taste (I just don't think the multiple files or forcibly declarative approach just for half the things mixes well with how setuptools works right now) so I'll close the ticket for now even though it doesn't actually work, but see the above issue link for that

Edit: unintentional comment spam from my side (sorry again 😬 ) ends with this comment of my past self, next comment explains true issue more in-depth

For what it's worth, I discovered using PEP518 (build-system.requires) instead of setup_requires has the following problem:

• setup_requires is added to returned value of hooks.get_requires_for_build_wheel() as called by pip/pep517, but in practice not installed if one just needs to run setup.py and obtain hooks.get_requires_for_build_wheel() without actually building the wheel - which is necessary e.g. to obtain package metadata like dependencies.
• and build-project.requires always is, per design, installed first, even when just hooks.get_requires_for_build_wheel() is run and the wheel is never built, affecting obtaining package metadata

This is a problem because it slows down obtaining basic package metadata greatly if you put in a build dependency that builds slowly. And many .pxd cimport targets tend to be larger Cython libs with native code compilation that is very slow. Some of my projects went from a few seconds metadata extraction to more than five minutes! And in some environments like https://github.com/kivy/python-for-android this metadata analysis is necessary to automatically map to cross-compile patches and other tasks which turns abhorrently slow due to this.

I'm therefore reopening, and suggesting setup_requires is undeprecated (is it even still deprecated?) or something else is found to provide that behavior, also see discussion here: https://discuss.python.org/t/support-for-build-and-run-time-dependencies/1513/61?u=jtt