Speed up os.path

[nineteendo]

Feature or enhancement

Proposal:

There are some minor optimisations possible in os.path, including some fast returns.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Links to previous discussion of this feature:

• gh-117201: Handle leading // for posixpath.commonpath #117202

Linked PRs

• gh-117349: Micro-optimize a few os.path functions #117350
• gh-117349: Speedup os.path #117610

[nineteendo]

@barneygale, can you reopen this?

[AlexWaygood]

Hey @nineteendo. In my opinion, optimisations for os.path should be welcome where they don't lead to significant deteriorations in style -- these are low-level functions that can often be used many times across a codebase, so improving performance of these functions can be pretty significant for end users of Python. Quoting my comment from #117350 (comment), however:

My advice for the future, however, would definitely be to work on small, focused PRs that have isolated, easily measurable changes. PRs like that are much easier for us to review, and you should find the contributing experience less frustrating as a result.

If you have a specific optimisation in mind (that improves performance by 5% or more, perhaps) for a specific function, and you're confident that the full test suite passes with the optimisation applied, then I would advise opening an issue regarding just that one optimisation. The issue should describe the proposed optimisation properly ("fast returns" isn't really a sufficient description here :-). Any PRs linked to that issue should tackle the specific optimisation proposed in the issue, and nothing else. Generic issues that propose "optimising os.path", and PRs that make changes to multiple functions at once, aren't particularly helpful ways to engage in CPython development.

[nineteendo]

If you have a specific optimisation in mind (that improves performance by 5% or more, perhaps) for a specific function, and you're confident that the full test suite passes with the optimisation applied, then I would advise opening an issue regarding just that one optimisation.

If the same optimisation can be applied to several functions, can I make a single issue?

Any PRs linked to that issue should tackle the specific optimisation proposed in the issue, and nothing else.

Does that include further optimisations to the modified functions, besides the proposed one? Or do I need to make a separate issue?

[AlexWaygood]

If the same optimisation can be applied to several functions, can I make a single issue?

I think that would be fine. But I'd make sure to limit any PR so that it only changes functions where that optimisation actually makes a difference :-)

Does that include further optimisations to the modified functions, besides the proposed one? Or do I need to make a separate issue?

It's hard to say in general. Sometimes we have one issues to track multiple changes to a function or an area of code. Sometimes we're more atomic, and have a different issue to track each specific change. But whatever the case, it's important to have each issue clearly define the work you're proposing to do, and why you think it will be beneficial for users of CPython. If you want to optimise os.path, then you need to clearly state which optimisations you're planning in the issue before filing a PR. If you later see the potential for followup PRs with different optimisations, then yes, it would probably be best to open a new issue detailing the new optimisation, rather than reusing the old issue.