Get the Pyston macrobenchmarks working on `main`

[ericsnowcurrently]

• Get 0.29.x working on Python 3.11 cython/cython#4465
• Build failure with CPython main (3.11). unbit/uwsgi#2365

[gvanrossum]

@brandtbucher Could you contribute the specific list of benchmark failures that you've experienced? Maybe we need a separate issue for each (unless they're all just "needs to use the fixed Cython release" :-).

[brandtbucher]

Yep!

2021-12-11 01:49:11,016: 9 benchmarks failed:
2021-12-11 01:49:11,016: - aiohttp
2021-12-11 01:49:11,016: - djangocms
2021-12-11 01:49:11,016: - genshi
2021-12-11 01:49:11,016: - gevent_hub
2021-12-11 01:49:11,016: - gunicorn
2021-12-11 01:49:11,016: - kinto
2021-12-11 01:49:11,016: - mypy
2021-12-11 01:49:11,016: - pylint
2021-12-11 01:49:11,016: - pytorch_alexnet_inference

Give me a moment to gather the individual failure messages...

[brandtbucher]

The logs are too long to leave in a comment. I've dumped the failures into a Gist instead:

https://gist.github.com/brandtbucher/9c1368c43f31b1d31c1efbd3d2fdf5e8

[gvanrossum]

So the "gist" is:

• aiohttp: can't build dependency uvloop
• djangocms: ditto Pillow
• gevent_hub: ditto greenlets
• gunicorn: depends on aiohttp (!)
• kinto: failed to build dep uWSGI
• mypy: failure building typed-ast (IIRC I've seen a fix, maybe a faulty version pin?)
• pytorch_alexnet_inference: can't find torch 1.5.1

Runtime failures:

• genshi: instantiates types.CodeType() with incorrect parameters (should be a simple fix)
• pylint: uses inspect.formatargspecs; 3.10 docs state: "Deprecated since version 3.5: Use signature() and Signature Object, which provide a better introspecting API for callables."

[gvanrossum]

I propose to create separate TODO issues for each. I suspect the "In Review" column of the Project board might come in handy once we start working down the list...

[brandtbucher]

• aiohttp and gunicorn look like they have been fixed via Cython: FIX: change c includes to account for upstream changes cython/cython#4428
• djangocms requires libjpeg to install Pillow (but we may be able to just pass --disable-jpeg instead when building to get around that): https://pillow.readthedocs.io/en/latest/installation.html#external-libraries
• genshi has an open issue: Genshi doesn't work on Python 3.11: code expected at least 18 arguments, got 16 edgewall/genshi#43
• pylint failure is via wrapt, which has an open issue: Python 3.11: ImportError: cannot import name 'formatargspec' from 'inspect' GrahamDumpleton/wrapt#196

[gvanrossum]

Do you mind if I assign this to you? Somebody needs to track all those fixes, nudge them along if needed, and update the requirements.txt files in PyPerformance once the fixes have been released. Since you tracked down so many of these already maybe we can make it official that you are the "whip" for this topic.

[brandtbucher]

Yup, I can herd these.

[JelleZijlstra]

mypy pins an overly old version of typed-ast. The current release (0.910) has typed_ast >= 1.4.0, < 1.5.0 and typed-ast 1.5.1 has the fixes for 3.11.

typed-ast is an optional dependency for mypy though, only necessary on Python <3.8 or to typecheck Python 2 code. Perhaps you can get away with removing it from your benchmark.

[gvanrossum]

Benchmarks usually pin some older version of a package since we want benchmark numbers to be comparable over time. (Though sometimes we have to just toss old data anyway.)

[ericsnowcurrently]

Also see python/pyperformance#113.