gh-135401: Test AWS-LC as a cryptography library in CI #135402
Conversation
This reverts commit 7d37e6a.
AA-Turner
changed the title
Misc/NEWS.d/next/Tests/2025-06-11-16-52-49.gh-issue-135401.ccMXmL.rst
Outdated
Show resolved
Hide resolved
.github/workflows/build.yml
Outdated
| with: | ||
| path: ./multissl/aws-lc/${{ matrix.awslc_ver }} | ||
| key: ${{ matrix.os }}-multissl-aws-lc-${{ matrix.awslc_ver }} | ||
| # TODO [childw] can we use env.* instead of env vars here? |
There was a problem hiding this comment.
I'd suggest for the initial version, keep as similar to the OpenSSL job/workflow, and then perhaps update both at once afterwards?
There was a problem hiding this comment.
Fair enough. I'll remove the TODOs. Perhaps we can leave this comment unresolved as a reminder for me to clean up both (if tenable) if/after this PR has been merged.
Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
…XmL.rst Co-authored-by: Adam Turner <[email protected]>
WillChilds-Klein
changed the title
|
Can you cherry-pick 8f4a0eb and make a separate PR please? TiA. |
|
Ok, the failure is because HMAC-SHA3 isn't supported in AWS-LC. I don't know if the ValueError is actually on my side or fired from OpenSSL and I'm just converting the message, but improving that message would be nice. |
|
Looks like it's coming from python. This |
|
Ok so it fell back to the default error message (i.e. there was no reason we could extract) |
This PR implements HMAC over truncated SHA3 variants as specified in [NIST SP 800-224](https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-224.ipd.pdf#section.3). We do so without externalizing any SHA3/keccak internals, including its context struct. This is done intentionally, as OpenSSL [has not yet externalized its SHA3/keccak context struct](https://github.com/openssl/openssl/blob/be7467f5a0aa098531597b95a71be6d7c2a463c7/include/internal/sha3.h#L40) and we want to leave the door open to future interoperability. To work around this in `hmac.h`'s `md_ctx_union`, we hard-code the context struct size and add a compile-time assertion that it does not grow larger than the hard-coded value. SHA3 is unique in supported HMAC digests in that, due to differences between its sponge construction and others' Merkle-Dåmgard constructions, it does not support pre-computed keys. To accommodate this difference, we refactor the relevant code generation macros and relevant unit tests. The HMAC-SHA3 feature gap was discovered in a somewhat roundabout way. While preparing a [pull request](python/cpython#135402) to add AWS-LC to upstream CPython's CI, I discovered that CPython's `./configure` script's compile probe failed to detect `libcrypto` support for linking `hashlib`. The compile probe [referenced](https://github.com/python/cpython/blob/59963e866a1bb8128a50cd53d1b13eeab03df06e/configure#L30869) `NID_blake2b512`, which AWS-LC does not support. The consequence of this was that CPython used its HACL implementations for `hashlib` instead of linking AWS-LC. This did not affect our `ssl` integration, as AWS-LC always uses its own hash functions for TLS.
|
Ah yes the error is due to multissltests. We only use tags but the script could be extended to support exact commits maybe? |
I suppose it could, but that would require taking a test container dependency on |
WillChilds-Klein
requested review from
gpshead,
erlend-aasland,
corona10,
ezio-melotti and
hugovk
as code owners
| @@ -628,7 +703,7 @@ jobs: | |||
| - build-windows-msi | |||
| - build-macos | |||
| - build-ubuntu | |||
| - build-ubuntu-ssltests | |||
| - build-ubuntu-ssltests-openssl | |||
There was a problem hiding this comment.
ISTM we should include the new AWS-LC variant in each of these three places as well. Being included in allowed-failures and allowed-skips it can't block a merge, but it will ensure that there's a result before a PR is merged.
There was a problem hiding this comment.
I see. For allowed-failures and allowed-skips, I agree. But what about all-required-green? It looks like that job is "required" to pass, and will fail if any of its members do not pass.
So, it looks like including -awslc there would effectively make it a "required" check. I'm open to this, but @gpshead expressed a preference for making this test non-required at first, which seems reasonable.
There was a problem hiding this comment.
Oh, I see! needs here does not mean "needs to have succeeded", just "needs to have completed". From GitHub's docs:
Use jobs.<job_id>.needs to identify any jobs that must complete successfully before this job will run ... If a job fails or is skipped, all jobs that need it are skipped unless the jobs use a conditional expression that causes the job to continue.
Your proposal makes sense. I'll add -awslc in those 3 places.
Notes
This PR extends
multissltests.py'sAbstractBuilderclass to fetch AWS-LC v1.55.0 and build it using CMake and GNU make. To do this, we addcmakeas a GitHub Runner dependency in.github/workflows/posix-deps-apt.sh. We also update CPython'sconfigureandconfigure.acscripts to swap out BLAKE2 (not tracked for standardization) in favor of SHA-512 when detectinglibcryptocompilation compatibility forhashlib.Finally, a new CI workflow uses this update to dynamically link AWS-LC against CPython, perform a linkage check, and run CPython's
ssltests.pyin CPython's public CI. This differs from AWS-LC's own CPython integration test where we statically link the CPython executable to AWS-LC.The new CI check is not marked as "required", but if the community wants to make it "required" for future PRs that can be done by adding a list item for
build-ubuntu-ssltests-awslchere.Please feel free to file an issue with the AWS-LC team here for assistance in troubleshooting any CI failures of the new check.
Testing