Add dependency groups to pyproject.toml file

[Julfried]

Type of Changes

	Type
✓	🔨 Refactoring

Description

Follow up to #10288. Introducing dependency groups in pyproject.toml and cleaning up the file a little bit. Once pip fully supports PEP735, we should update the documentation and remove all the requirement.txt files.

With this change you are able to install dependencies like this:

uv pip install -e . --group docs # installs docs requirements
uv pip install -e .[spelling] --group docs --group dev # installs pre-commit and docs requirements + spelling extras

One thing I am currently unsure is how the dependencies in .readthedocs.yaml would be handled, when the requirement files in the doc folder are removed?

[Pierre-Sassoulas]

Thank you for handling this ! I have some nits and some discussions of actual changes for other maintainers to chime in :)

[Julfried]

Applied your suggestion, I think the failing pylint run seems like a caching issue to me, because it installs pylint but does not find the executeable. What do you think?

[DanielNoord]

Very cool! Thanks!

[DanielNoord]

Assuming CI passes this LGTM!

[Pierre-Sassoulas]

We probably have to change the file based caching based on the requirement files too ? Not sure if the pipeline is failing because of the changes or because the cache is already broken.

[Julfried]

Yeah I think the cache is broken. I see that the pylint step is using pre-commit again. Maybe we should replace that?

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.90%. Comparing base (6faf5b4) to head (cdf3544).
Report is 84 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main   #10332   +/-   ##
=======================================
  Coverage   95.90%   95.90%           
=======================================
  Files         176      176           
  Lines       19108    19108           
=======================================
  Hits        18325    18325           
  Misses        783      783           

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Julfried]

Ok, actually I have no idea why this fails here.

[github-actions]

🤖 According to the primer, this change has no effect on the checked open source code. 🤖🎉

This comment was generated for commit 39b9052

[Julfried]

Found it! I accidentally deleted the entry point scripts and urls section when editing the pyproject.toml, which caused resulted this weird error when installing pylint with pip install .

Now the pyproject.toml should be complete again. The pypy error seems unrelated, because it already passed on previous runs.

[Pierre-Sassoulas]

Thank you for being so persistent @Julfried :)

[cdce8p]

Thanks for the PR. Please note though that dependabot does not support dependency-groups either. See dependabot/dependabot-core#10847.

I tested it on my fork by reverting the typing-extensions bump. The PR dependabot created updates only requirements_test_min.txt: cdce8p#4.

On that note, it'll likely be a challenge to keep them in sync for the time being as the version requirements have already diverted.

pylint/requirements_test_min.txt

Line 4 in 70559b8

typing-extensions~=4.13

pylint/pyproject.toml

Line 106 in 70559b8

"typing-extensions~=4.12",

So, even while I'm in favor of moving to dependency-groups, I do wonder it it would be better to revert this one for the time being and wait for better tooling support.

[Julfried]

Ok I was not aware of the current limitation of dependabot. However since we still left the requirements.txt files and also the the documentation still refers to them, I think reverting is not needed for now. Once all tooling fully supports PEP735 we can update docs and remove the requirements.txt files. What do you think?

[cdce8p]

However since we still left the requirements.txt files and also the documentation still refers to them, I think reverting is not needed for now.

Yeah, it's probably not needed. However I think we should remember that the requirement files are the source of truth (at least for the time being). Mainly, so we don't update the CI pipeline just yet. As for dependabot PRs they will require more attention now. Alternative we could just ignore the dependency groups for now and don't keep them in sync but that's a case where I'd feel more comfortable reverting this PR for the time being.

Once all tooling fully supports PEP735 we can update docs and remove the requirements.txt files. What do you think?

Once the support is there, mainly in pip and dependabot, this should be fine. This will still take some time though as dependabot seems to move rather slowly. Maybe there is some more urgency there once the pip support is released.

[Julfried]

Thanks for the heads-up! Since CI and docs still rely on the requirements.txt files, I think we're fine for now. We can treat the dependency groups as optional until tooling catches up, and fully switch once support is there. That said, I’m also fine with reverting if you think it’s cleaner. What do you think @Pierre-Sassoulas?

[Pierre-Sassoulas]

Let's revert and re-revert when the time is right. It's better if we deal with upgrading dependency group from the requirement only once during the re-revert. (Thank you for the review Marc, I wrongly assumed dependabot would 'just work')

[Julfried]

Sounds good, lets do that👍

[Pierre-Sassoulas]

Reverted temporarily in #10342