Problems with tests on Fedora #153
Comments
|
I had to read through the messages a few times, but I came to the conclusion that the Message/Encrypted/Decrypted prints are just red herrings. There is only one test failure, and that's the mypy check. What surprises me is that it fails for you, while for me on both Mint Linux and Windows 10 the tests succeed. Do you have any environment variables set that may influence mypy's behaviour? |
|
I got same problem on Debian(sid). This is CI build log. |
|
I was looking at updating Fedora packages as well... looking around I found the issue below which looks like a similar error, but not having looked at the python-rsa code I can't say if it's the same. |
|
I'd love to get this fixed, but I can't reproduce this locally, it all works fine for me on Ubuntu 18.04, Linux Mint, and Windows 10. |
|
A few things to note, the offending lines are: I would only expect this to happen on python 3.6+, so part of why you are or aren't seeing this could depend on what versions of python you have on Mint/Windows. I am not certain I'm following completely, but it seems that in Python 3.6 a Keccak SHA3 implementation was used. Later a FIPS 202 SHA3 implementation came along and both were kept and there might be some additional information about which implmentation is being used. As a test I imported I don't think this is a proper fix, but seems to point that the test errors might indeed be legitimate. |
|
Hi, I'm not sure which variables can affect those behavior, but I tested the following procedure that can be run on any machine that is able to execute a docker container: I hope this helps to debug the problem :-) |
|
I don't really understand. This doesn't cause an error... |
|
grasping at straws but I see multiple mypy issues wrt to update: |
|
I believe the root cause of the "non-determinism" is which version of mypy you're running. Adding a type hint like this seems to avoid the mypy error: HASH_METHODS: Dict[str, Callable[[], hashlib._Hash]] = {
"MD5": hashlib.md5,
"SHA-1": hashlib.sha1,
"SHA-224": hashlib.sha224,
"SHA-256": hashlib.sha256,
"SHA-384": hashlib.sha384,
"SHA-512": hashlib.sha512,
}
if sys.version_info >= (3, 6):
# Python 3.6 introduced SHA3 support.
HASH_METHODS.update(
{
"SHA3-256": hashlib.sha3_256,
"SHA3-384": hashlib.sha3_384,
"SHA3-512": hashlib.sha3_512
}
)though... this root cause can't be right since the PR I referenced is fro 2017 and the latest version of mypy can run into the issue, but it goes away if you specify |
|
I think I've tracked it down to python/typeshed#3626, where in PY 3.7, it's |
|
I've figured out why Fedora was acting weirdly! The typeshed commit I mentioned that avoids the issue in 3.7 was commited around Jan 22, 2020 and was vendored in mypy==0.770 (Mar 9, 2020). From @Fale 's logs from above:
So it's expected that 1 and 2 would have run into issues. For 0.780 and even the lastest version of python3-mypy (0.782-2.fc33), we should have had the python/typeshed#3626 fix, but it looks like python3-mypy-0.782-2.fc33 did not vendor this correctly: https://bugzilla.redhat.com/show_bug.cgi?id=1929522 Ultimately, it looks like mypy's inference gets a bit wonky, so a type hint like I noted above seems like the best way to handle this, especially since the issue is present if we update setup.cfg to have mypy target 3.8 or 3.9. |
As captured in python/typeshed#1663, the types for SHA-1 and SHA-2 family of functions are callables that return a Hash instance, whilst the SHA-3 family of functions are Hash `type`s (at least in Python 3.6). Mixing the two kinds of functions together in a dictionary confuses mypy's type inference as noted in sybrenstuvel#153, so we instead add an annotation as a hint. Also, update test_my.py to match the python version set by tox.ini in CI instead of always targeting Python 3.7 (as configured in setup.cfg) to validate the types in all supported Python 3.x versions. This fix also avoids the issue with the older mypy releases for Python 3.6 / Python 3.7 found in distro repos... ... for Ubuntu: ``` docker run \ -v $(pwd):/tmp/rsa \ -w /tmp/rsa ubuntu:18.04 \ /bin/bash -c 'apt-get update -qqy \ && apt-get install -qqy python3-pyasn1 python3-setuptools python3-mypy \ && python3 setup.py test' ``` ... and for Fedora: ``` docker run \ -v $(pwd):/tmp/rsa \ -w /tmp/rsa docker.io/fedora \ /bin/bash -c 'dnf -y install wget python3-devel python3-pyasn1 python3-setuptools python3-mypy \ && python3 setup.py test' ``` Fixes sybrenstuvel#153
As captured in python/typeshed#1663, the types for SHA-1 and SHA-2 family of functions are callables that return a Hash instance, whilst the SHA-3 family of functions are Hash `type`s (at least in Python 3.6). Mixing the two kinds of functions together in a dictionary confuses mypy's type inference as noted in #153, so we instead add an annotation as a hint. Also, update test_my.py to match the python version set by tox.ini in CI instead of always targeting Python 3.7 (as configured in setup.cfg) to validate the types in all supported Python 3.x versions. This fix also avoids the issue with the older mypy releases for Python 3.6 / Python 3.7 found in distro repos... ... for Ubuntu: ``` docker run \ -v $(pwd):/tmp/rsa \ -w /tmp/rsa ubuntu:18.04 \ /bin/bash -c 'apt-get update -qqy \ && apt-get install -qqy python3-pyasn1 python3-setuptools python3-mypy \ && python3 setup.py test' ``` ... and for Fedora: ``` docker run \ -v $(pwd):/tmp/rsa \ -w /tmp/rsa docker.io/fedora \ /bin/bash -c 'dnf -y install wget python3-devel python3-pyasn1 python3-setuptools python3-mypy \ && python3 setup.py test' ``` Fixes #153
As captured in python/typeshed#1663, the types for SHA-1 and SHA-2 family of functions are callables that return a Hash instance, whilst the SHA-3 family of functions are Hash `type`s (at least in Python 3.6). Mixing the two kinds of functions together in a dictionary confuses mypy's type inference as noted in #153, so we instead add an annotation as a hint. Also, update test_my.py to match the python version set by tox.ini in CI instead of always targeting Python 3.7 (as configured in setup.cfg) to validate the types in all supported Python 3.x versions. This fix also avoids the issue with the older mypy releases for Python 3.6 / Python 3.7 found in distro repos... ... for Ubuntu: ``` docker run \ -v $(pwd):/tmp/rsa \ -w /tmp/rsa ubuntu:18.04 \ /bin/bash -c 'apt-get update -qqy \ && apt-get install -qqy python3-pyasn1 python3-setuptools python3-mypy \ && python3 setup.py test' ``` ... and for Fedora: ``` docker run \ -v $(pwd):/tmp/rsa \ -w /tmp/rsa docker.io/fedora \ /bin/bash -c 'dnf -y install wget python3-devel python3-pyasn1 python3-setuptools python3-mypy \ && python3 setup.py test' ``` Fixes #153
Hi,
I'm updating the Fedora package, but I'm not able to make the tests work. I'm not sure if it's a problem with how Fedora builds Python or there is something else going on.
I would say that the majority of tests works properly, but the SHA3 ones fail with an error like the following (the type it expects does change on the various Python versions):
This happens with:
Do you have any idea on the possible cause and work around?
Thanks a lot!
The text was updated successfully, but these errors were encountered: