Fails to compile with Apple Silicon compiler

[gvanrossum]

This combination fails:

• Intel Mac (UPDATED)
• macOS 11.2.2
• Xcode 12.4
• Python 3.10a6

[gvanrossum]

Some errors:

gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -Iast27/Include -I/Library/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c ast27/Custom/typed_ast.c -o build/temp.macosx-10.9-universal2-3.10/ast27/Custom/typed_ast.o
In file included from ast27/Custom/typed_ast.c:1:
In file included from /Library/Frameworks/Python.framework/Versions/3.10/include/python3.10/Python.h:11:
In file included from /Library/Developer/CommandLineTools/usr/lib/clang/11.0.0/include/limits.h:37:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/limits.h:63:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:807:2: error: Unsupported architecture
#error Unsupported architecture
 ^
In file included from ast27/Custom/typed_ast.c:1:
In file included from /Library/Frameworks/Python.framework/Versions/3.10/include/python3.10/Python.h:11:
In file included from /Library/Developer/CommandLineTools/usr/lib/clang/11.0.0/include/limits.h:37:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/limits.h:64:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/limits.h:8:2: error: architecture not supported
#error architecture not supported
 ^
In file included from ast27/Custom/typed_ast.c:1:
In file included from /Library/Frameworks/Python.framework/Versions/3.10/include/python3.10/Python.h:25:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types.h:27:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:33:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/_types.h:34:2: error: architecture not supported
#error architecture not supported
 ^
In file included from ast27/Custom/typed_ast.c:1:
In file included from /Library/Frameworks/Python.framework/Versions/3.10/include/python3.10/Python.h:25:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h:64:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h:71:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types.h:27:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:55:9: error: unknown type name '__int64_t'; did you mean '__int128_t'?
typedef __int64_t       __darwin_blkcnt_t;      /* total blocks */
        ^
note: '__int128_t' declared here
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h:56:9: error: unknown type name '__int32_t'; did you mean '__int128_t'?
typedef __int32_t       __darwin_blksize_t;     /* preferred block size */
        ^

(And more, but nothing more revealing.)

[hauntsaninja]

Note, if you just want to build x86 wheels, I suspect setting export ARCHFLAGS="-arch x86_64" should work, a la https://github.com/joerick/cibuildwheel/blob/c9451facbd7a4bb54d580a093c9e381d524c6170/cibuildwheel/macos.py#L248

[gvanrossum]

Ah, thanks. Though won't that leave mypy users on Apple Silicon high and dry? (Hm, mypyc probably doesn't support that yet either, so it may require using Intel emulation mode anyway?)

[ned-deily]

I just tried it on my Intel Mac running macOS 11.2.2 and it seemed to compile OK. The issue may be that you seem to be using header files from the Command Line Tools rather than Xcode itself and it may be that the Command LIne Tools are out of date. Try this:

$ xcode-select -p
/Applications/Xcode.app/Contents/Developer

If not Xcode, then:

$ sudo xcode-select -s /Applications/Xcode.app/Contents/Developer

You can also try to update the Command Line Tools (they should be sufficient if you don't use the Xcode.app itself):

$ xcode-select --install

[hauntsaninja]

I believe mypy recently started building Apple Silicon wheels: python/mypy#10114 / https://github.com/python/mypy/pull/10145/files Switching over to cibuildwheel earlier made this really easy; we should switch typed-ast too.
Finally, I opened python/mypy#10183 today to drop the required typed-ast dependency (if you want to review ;-) ).

[ned-deily]

(To clarify, I didn't try the whole build process, just setup.py build and it did not get the errors Guido saw. This was also using a python.org 3.10ax universal2 Python so it compiled for both Intel and Apple Silicon.)

[gvanrossum]

Thanks @ned-deily, changing the path to use the Xcode headers seems to have worked. But running xcode-select --install did not fix anything.

I'm guessing I can tell a universal compilation is happening because I get the warning about an unreachable abort() call in ast.c twice. :-)

@hauntsaninja Should I just push a 3.10 macOS wheel without bumping the typed_ast version? All that happened since the 1.4.2 release is build stuff. (OTOH the source tarball wouldn't compile on macOS 11, so maybe a version bump is advisable?)

[hauntsaninja]

I'd bump the version due to the source tarball changing. Note it's not just macOS 11, #155 is anyone trying to compile for 3.10 from source.

[gvanrossum]

Okay. Unfortunately I am rather out of time to work on this soon. :-(

[hauntsaninja]

Maybe I'll bother you about this on May 3rd :-)
(In other good news, mypy master no longer requires typed-ast on Python 3.10)

[hauntsaninja]

Closing, since this seemed to be some env issue. #163 should get cibuildwheel to make us Apple Silicon wheels.