Yarn 11391

[lnbest0707]

Description of PR

The patch reuses the resolver introduced from the HDFS side and apply similar logic to YARN RM service addresses resolving.
To utilize the YARN DNS support and use DNS as endpoint, simply upgrade the hadoop binary and revise configs from, for example:

yarn.resourcemanager.address.rm1 rm1_address:8032 yarn.resourcemanager.scheduler.address.rm1 rm1_address:8030 yarn.resourcemanager.resource-tracker.address.rm1 rm1_address:8031 yarn.resourcemanager.admin.address.rm1 rm1_address:8033 yarn.resourcemanager.webapp.address.rm1 rm1_address:8088 yarn.resourcemanager.webapp.https.address.rm1 rm1_address:8090 yarn.resourcemanager.address.rm2 rm2_address:8032 yarn.resourcemanager.scheduler.address.rm2 rm2_address:8030 yarn.resourcemanager.resource-tracker.address.rm2 rm2_address:8031 yarn.resourcemanager.admin.address.rm2 rm2_address:8033 yarn.resourcemanager.webapp.address.rm2 rm2_address:8088 yarn.resourcemanager.webapp.https.address.rm2 rm2_address:8090 yarn.resourcemanager.ha.rm-ids rm1,rm2 to: yarn.resourcemanager.address.rm1 rm_multi_a_dns:8032 yarn.resourcemanager.scheduler.address.rm1 rm_multi_a_dns:8030 yarn.resourcemanager.resource-tracker.address.rm1 rm_multi_a_dns:8031 yarn.resourcemanager.admin.address.rm1 rm_multi_a_dns:8033 yarn.resourcemanager.webapp.address.rm1 rm_multi_a_dns:8088 yarn.resourcemanager.webapp.https.address.rm1 rm_multi_a_dns:8090 yarn.resourcemanager.ha.rm-ids rm1 yarn.resourcemanager.ha.resolve-needed true yarn.resourcemanager.ha.resolver.useFQDN true # required in secure mode yarn.resourcemanager.ha.refresh-period-ms 180000 # 3 min
where rm_multi_a_dns is a multi-A DNS record for rm1_address and rm2_address. This means the following output on the terminal.

$ dig +short <rm_multi_a_dns> | xargs -n +1 dig +short -x | sort
<rm1_address>
<rm2_address>
For the newly introduced flags, please refer to yarn-default.xml.

How was this patch tested?

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
• Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
• If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
• If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

[hadoop-yetus]

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 51s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	xmllint	0m 0s		xmllint was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	test4tests	0m 0s		The patch appears to include 2 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	15m 16s		Maven dependency ordering for branch
+1 💚	mvninstall	30m 32s		trunk passed
+1 💚	compile	25m 10s		trunk passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	compile	21m 52s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	checkstyle	4m 0s		trunk passed
+1 💚	mvnsite	3m 23s		trunk passed
-1 ❌	javadoc	1m 8s	/branch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in trunk failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 14s		trunk passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	spotbugs	6m 36s		trunk passed
+1 💚	shadedclient	24m 30s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 25s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 20s		the patch passed
+1 💚	compile	24m 28s		the patch passed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04
+1 💚	javac	24m 28s		the patch passed
+1 💚	compile	21m 41s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
+1 💚	javac	21m 41s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
-0 ⚠️	checkstyle	3m 58s	/results-checkstyle-root.txt	root: The patch generated 2 new + 184 unchanged - 0 fixed = 186 total (was 184)
+1 💚	mvnsite	3m 19s		the patch passed
-1 ❌	javadoc	0m 59s	/patch-javadoc-hadoop-common-project_hadoop-common-jdkUbuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.txt	hadoop-common in the patch failed with JDK Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04.
+1 💚	javadoc	2m 12s		the patch passed with JDK Private Build-1.8.0_352-8u352-ga-1~20.04-b08
-1 ❌	spotbugs	2m 2s	/new-spotbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.html	hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0)
+1 💚	shadedclient	24m 30s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	18m 2s		hadoop-common in the patch passed.
+1 💚	unit	1m 6s		hadoop-yarn-api in the patch passed.
+1 💚	unit	5m 24s		hadoop-yarn-common in the patch passed.
+1 💚	asflicense	0m 52s		The patch does not generate ASF License warnings.
		257m 7s

Reason	Tests
SpotBugs	module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
	Inconsistent synchronization of org.apache.hadoop.yarn.client.ConfiguredRMFailoverProxyProvider.rmServiceIds; locked 90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:90% of time Unsynchronized access at RequestHedgingRMFailoverProxyProvider.java:[line 147]

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5277/1/artifact/out/Dockerfile
GITHUB PR	#5277
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient spotbugs checkstyle codespell detsecrets xmllint
uname	Linux b0e74c34ddff 4.15.0-197-generic #208-Ubuntu SMP Tue Nov 1 17:23:37 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / e7d2a31
Default Java	Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.17+8-post-Ubuntu-1ubuntu220.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_352-8u352-ga-1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5277/1/testReport/
Max. process+thread count	3134 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-5277/1/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.