Asymptotically faster int(string)

[tim-one]

Proposal:

EDIT: I made better progress on this than I expected, reducing the cutoff dramatically to about 3.4 million digits, by revisiting and improving the "cached reciprocal approximation" approach. Still not really compelling, but ... what the heck 😉.

Way back when we were first working on ideas for faster int <-> decimal string conversions, @pochmann worked up a function that used the decimal module to, in effect, convert from base 10 to base 256. This is "unnatural", in that it requires multiplying and dividing by large powers of 2, which decimal isn't naturally suited to. But decimal's * and / are asymptotically superior to CPython's, so at some point it could be expected to win.

Alas, the crossover point was too high to be of much real interest. I then worked on ways to replace its division with multiplication by a cached reciprocal approximation instead, fixing up (usually small) errors afterwards. This reduced the crossover point significantly, but still too high to be of much real interest. And the code grew increasingly complex.

While working on _pylong.py across the last few days on a different issue, I had a different idea: there's no actual need to divide by powers of 2 at all. Since 2*5 = 10, division by 2**k is the same as multiplying by 5**k (exact) and then dividing by 10**k (also exact in decimal, and dirt cheap: the module's scaleb() function, which just adjusts the internal exponent).

This yields short and reasonably clear code with a much lower crossover value, at about 26 million digits. That's still too high to be compelling, but I want to record the progress in case someone else is intrigued enough to pursue it (I'm out of time for this for now). So I intend to add the (working, but unused) code to _pylong.py.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs

• gh-118750: Asymptotically faster int(string) #118751