having a way resolver outputs a "number of ticks", and being able to limit that number

[stonebig]

What's the problem this feature will solve?

give the builder a way to "stop" the resolver when things are obviously not going the usual way versus previous builds

Describe the solution you'd like

I'm building from a local drive a huge list of requirement.
When all goes well, the resolver is "expectedly" behaving about the same each time,
When not, the resolver takes way more time, maybe would take a day but I kill it before, and I loose that "way more" time.

nota: after that initial big and long try, implementing a constraint.txt file and retrying, and nothing else, allows to pin-point the issue. (as 'pip check' can't see it comes from a dask[complete] constraint)

What would avoid this at minimal initial time cost is:

• to have the resolver counting its "steps" to resolve a request, and output it
• so that when asking a pip install, a parameter limiting the number of steps would allow to "break" spontaneously when we are out of the "normal" situation.

so, request is to have pip install --verbose :

• to include, just before the summary, or as it runs, a "steps" or "number of ticks" spending,
• being able via a --steps_limit to put an upper bound to it.

Advantage vs a time limit or depth limit:

• no dependancy on the cpu speed
• no change in the algorithm itself.

Alternative Solutions

this is an alternative to #10932

Additional context

I can try to experiment a counter if pip devs gives have an idea of how what/where could be counted.
It must be in a place where the "never ending resolution" is increasing it continously.

the "number of steps" displaying and the "breakage" could be done at the current --verbose output line writing moment.

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[pradyunsg]

What version of pip are you using?

[pfmoore]

This doesn't sound substantially different from most of the previous proposals. While an opaque "number of steps" might be harder for people to guess at, there's no obvious way to force people to only supply a value that they got from the report of a previous run, and I'm pretty sure we'd end up with misguided advice on the internet to "add --steps_limit=20000" or something, just to avoid long runtimes.

[stonebig]

What version of pip are you using?

The last one

[stonebig]

This doesn't sound substantially different from most of the previous proposals. While an opaque "number of steps" might be harder for people to guess at, there's no obvious way to force people to only supply a value that they got from the report of a previous run, and I'm pretty sure we'd end up with misguided advice on the internet to "add --steps_limit=20000" or something, just to avoid long runtimes.

Given my number of package and my iterative approach, that I suppose is also the typical use case of many, that number, while meaningless between each different use case, may be statistically significant for each independent repetitive use case

[stonebig]

A simpler approach you may prefer is to limit the number of "taking more time than usual" existing --verbose message when things are going wrong.

That would be even less invasive. I have this message twice in my fail build, and it's still way later to me, but it would be an improvement from existing. looking where the code is for that message

[stonebig]

I see there is this hard-coded

https://github.com/pypa/pip/blob/main/src/pip/_internal/resolution/resolvelib/resolver.py#L91

[stonebig]

the verbose mode is there

https://github.com/pypa/pip/blob/main/src/pip/_internal/resolution/resolvelib/reporter.py#L43

[pfmoore]

I see there is this hard-coded

This has been discussed endlessly in previous issues. Please review the previous discussions to understand why we're not going to expose that limit to the user.

[stonebig]

would you accept that in pip reporter the messages_at_reject_count for '8:' is slightly different from '1:' ?

[pfmoore]

would you accept that in pip reporter the messages_at_reject_count for '8:' is slightly different from '1:' ?

I don't personally see the point (if there's 2 messages, you've hit 8 - it's basically just a "still working..." message, after all) but I'd be fine with it saying "pip is still looking at..." (rest of message stays the same) if it bothers you.

[notatallshaw]

I think this is basically a duplicate of #10417, although this has been discussed in other threads.

I would argue that since Pip 23.1+ some of the previous arguments about user supportability are less meaningful as a user is far less likely to get stuck backtracking for a long period of time and therefore far fewer users would go looking for this option. A max backtracking option could be added with the text:

Sets the limit on the number of steps the resolver will take to backtrack to find a solution to the requirements. Advanced option to help investigate long backtracking issues, not advised to keep permanently as there is a non-linear relationship between the number of backtrack steps and any given resolution solution. Steps to investigate long backtracking times may include adding lower bounds on requirements, adding requirements iteratively, changing the order of the requirements (requirements added earlier are backtracked later all other things equal).

@stonebig As for the long backtracking problem itself, can you please provide a set of reproducible requirements that causes long backtracking times on Pip 23.1+. In general I'd be interested to see what causes long backtracking times, and in particular I want a few examples to see when PyPi backfills PEP 658 metadata to see if it's possible to use that as a preference for backtracking #12035. and if it helps.

[notatallshaw]

I will point out is pre-23.1 when I investigated backtracking issues I do manually edit the Pip source code to print out the number of backtracking steps (usually once every 100 or 1000 steps for long backtracking issues), and also what packages Pip is currently trying to resolve on if it's changed from the previous time it printed that message (I keep a set of the last print out).

This, and a general understanding of the backtracking algorithm, often allowed me to figure out where the actual "knot" causing the long backtracking times were, where often in these cases Pip was getting caught looking at the wrong place. It was not easy to find the "knot" without editing source code.

I am not sure this specific technique will apply to post-23.1 long backtracking time issues where there is a resolution but the information would still be useful. I have only come across the one so far in #12028 and it gave me the idea for #12035, but I think also in that example it wasn't useful is Pip's lack of telling the user why it's backtracking on that requirement (i.e. a conflict between awscli and pylint over colorama). I'm not sure how this would be added but similar to the same information that is given from ResolutionImpossible but perhaps after a certain number of backtrack steps, number of versions of a package backtracked through, or amount of time?

I think getting access to this message early of why backtracking is taking so long and/or failing was my original motivation for writing #10417, this information is still useful today and it would be good for a way to access it without editing the source code of Pip.