HADOOP-19091: Add support for Tez to MagicS3GuardCommitter

[VenkatSNarayanan]

This commit adds a MRv1 wrapper for the MagicS3GuardCommitter so applications like Hive can use it.

How was this patch tested?
The ITest suite was run with the changes in the us-east1 region. Two tests were added to test the code to handle Tez-generated UUIDs, and to check that the output path generated by the JobContext codepath matches the prefix of the one generated by the TaskAttemptContext codepath.

Exactly one test failed, with a timeout error. I've seen this error before, and I don't think it has anything to do with my change.

[ERROR] org.apache.hadoop.fs.s3a.impl.ITestConnectionTimeouts.testObjectUploadTimeouts -- Time elapsed: 18.71 s <<< FAILURE!
java.lang.AssertionError:
[Duration of write]
Expecting:
<PT16.212S>
to be less than:

at org.apache.hadoop.fs.s3a.impl.ITestConnectionTimeouts.testObjectUploadTimeouts(ITestConnectionTimeouts.java:247)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at java.util.ArrayList.forEach(ArrayList.java:1259)

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	8m 54s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	8m 17s		Maven dependency ordering for branch
+1 💚	mvninstall	15m 43s		trunk passed
+1 💚	compile	8m 33s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	compile	8m 37s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	checkstyle	1m 40s		trunk passed
+1 💚	mvnsite	1m 7s		trunk passed
+1 💚	javadoc	0m 58s		trunk passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javadoc	0m 56s		trunk passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
-1 ❌	spotbugs	0m 58s	/branch-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core in trunk has 178 extant spotbugs warnings.
-1 ❌	spotbugs	0m 50s	/branch-spotbugs-hadoop-tools_hadoop-aws-warnings.html	hadoop-tools/hadoop-aws in trunk has 188 extant spotbugs warnings.
+1 💚	shadedclient	14m 30s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 22s		Maven dependency ordering for patch
+1 💚	mvninstall	0m 45s		the patch passed
+1 💚	compile	8m 8s		the patch passed with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04
+1 💚	javac	8m 8s		the patch passed
+1 💚	compile	8m 33s		the patch passed with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
+1 💚	javac	8m 33s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 33s		the patch passed
+1 💚	mvnsite	1m 7s		the patch passed
-1 ❌	javadoc	0m 30s	/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-aws-jdkUbuntu-21.0.7+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 generated 7 new + 926 unchanged - 0 fixed = 933 total (was 926)
-1 ❌	javadoc	0m 27s	/results-javadoc-javadoc-hadoop-tools_hadoop-aws-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04.txt	hadoop-tools_hadoop-aws-jdkUbuntu-17.0.15+6-Ubuntu-0ubuntu120.04 with JDK Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04 generated 6 new + 848 unchanged - 0 fixed = 854 total (was 848)
-1 ❌	spotbugs	0m 57s	/new-spotbugs-hadoop-tools_hadoop-aws.html	hadoop-tools/hadoop-aws generated 3 new + 188 unchanged - 0 fixed = 191 total (was 188)
+1 💚	shadedclient	14m 32s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	6m 38s		hadoop-mapreduce-client-core in the patch passed.
+1 💚	unit	2m 15s		hadoop-aws in the patch passed.
+1 💚	asflicense	0m 33s		The patch does not generate ASF License warnings.
		121m 42s

Reason	Tests
SpotBugs	module:hadoop-tools/hadoop-aws
	Unknown bug pattern CT_CONSTRUCTOR_THROW in new org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter(Path, JobContext) At AbstractS3ACommitter.java:new org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitter(Path, JobContext) At AbstractS3ACommitter.java:[line 262]
	Unknown bug pattern CT_CONSTRUCTOR_THROW in new org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter(Path, JobContext) At MagicS3GuardCommitter.java:new org.apache.hadoop.fs.s3a.commit.magic.MagicS3GuardCommitter(Path, JobContext) At MagicS3GuardCommitter.java:[line 98]
	Unknown bug pattern CT_CONSTRUCTOR_THROW in new org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter(Path, JobContext) At StagingCommitter.java:new org.apache.hadoop.fs.s3a.commit.staging.StagingCommitter(Path, JobContext) At StagingCommitter.java:[line 153]

Subsystem	Report/Notes
Docker	ClientAPI=1.52 ServerAPI=1.52 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8085/1/artifact/out/Dockerfile
GITHUB PR	#8085
JIRA Issue	HADOOP-19091
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux 00030654a077 5.15.0-156-generic #166-Ubuntu SMP Sat Aug 9 00:02:46 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / 7e923c5
Default Java	Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Multi-JDK versions	/usr/lib/jvm/java-21-openjdk-amd64:Ubuntu-21.0.7+6-Ubuntu-0ubuntu120.04 /usr/lib/jvm/java-17-openjdk-amd64:Ubuntu-17.0.15+6-Ubuntu-0ubuntu120.04
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8085/1/testReport/
Max. process+thread count	1594 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-tools/hadoop-aws U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-8085/1/console
versions	git=2.25.1 maven=3.9.11 spotbugs=4.9.7
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.