HADOOP-19527. Skip AAL tests if encryption is set. #7592
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Changes to skip AAL vectored test if encryption is anything other than SSE-S3. |
|
what about sse-kms? I use that, normally |
|
🎊 +1 overall
This message was automatically generated. |
|
@steveloughran this change will skip the The tests were failing because for these encryption methods, the eTag is not the md5 of the object content. Because AAL caches the HEAD, the eTag being used on the GET was from a previous creation of the file used for the vectoredIo tests if you run these tests with no encryption set, it passes, as then the eTag is the md5, and so is always the same. More details in the JIRA ticket. |
steveloughran
requested changes
Apr 9, 2025
| * @param configuration configuration | ||
| * @throws IOException if the secret lookup fails. | ||
| */ | ||
| public static void skipIfEncryptionSet(Configuration configuration) throws IOException { |
There was a problem hiding this comment.
change title to skipForAnyEncryptionExceptSSES3()
| // https://github.com/awslabs/analytics-accelerator-s3/issues/218 | ||
| try { | ||
| skipIfEncryptionSet(conf); | ||
| } catch (IOException e) { |
There was a problem hiding this comment.
do this in the method itself
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
|
🎊 +1 overall
This message was automatically generated. |
ahmarsuhail
added a commit
to ahmarsuhail/hadoop
that referenced
this pull request
Apr 11, 2025
Contributed by Ahmar Suhail.
4 tasks
steveloughran
pushed a commit
that referenced
this pull request
Apr 25, 2025
Contributed by Ahmar Suhail.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of PR
When encryption is set to anything other than AES256, eTag behaviour changes, which leads to some test failures. See https://issues.apache.org/jira/browse/HADOOP-19527 for more details.
Also skip AAL tests if CSE is enabled as that is not supported currently.
How was this patch tested?
Ran both
ITestS3AContractAnalyticsStreamVectoredReadandITestS3AAnalyticsAcceleratorStreamReadingwith different encryption configurations.Tests pass when nothing is set or if
fs.s3a.encryption.algorithmis set to AES256.ITestS3AContractAnalyticsStreamVectoredRead is skipped when encryption is set to SSE-KMS (or anything else)
ITestS3AAnalyticsAcceleratorStreamReading is skipped when encryption is set to CSE-KMS.
For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?