HADOOP-16810. Increase entropy on precommit Linux VMs #2697

amahussein · 2021-02-12T23:29:29Z

HADOOP-16810: Increase entropy to improve cryptographic randomness on precommit Linux VMs
In my comment on MAPREDUCE-7079
This test case has been failing for ever.

When it timeout, MRAppMaster and some YarnChild processes remain running in the background. Therefore, the JVM running the tests fail due to OOM. No one notices that this unit test case has failed because the QA reports the unit tests that failed, but not timeout.
It works for Mac OS X, but never works for Linux running on a virtual Box. It only works on the latter by disabling MRJobConfig.MR_ENCRYPTED_INTERMEDIATE_DATA.

In this PR:

the DOCKER_EXTRAARGS are added to hadoop.sh to pass the random mount
~~the version 0.10.0 is not on the release page. So, this is upgrading the Yetus to a released version 0.13.0.~~
adding the mount parameter to start start-build-env.sh

hadoop-yetus · 2021-02-12T23:31:51Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/1/console in case of problems.

hadoop-yetus · 2021-02-13T01:54:08Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	22m 3s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 54s		trunk passed
+1 💚	mvnsite	24m 57s		trunk passed
+1 💚	shadedclient	13m 28s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	19m 30s		the patch passed
+1 💚	mvnsite	19m 41s		the patch passed
-1 ❌	shellcheck	0m 1s	/diff-patch-shellcheck.txt	The patch generated 1 new + 2 unchanged - 0 fixed = 3 total (was 2)
+1 💚	shelldocs	0m 20s		The patch generated 0 new + 0 unchanged - 236 fixed = 0 total (was 236)
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
-1 ❌	shadedclient	1m 52s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	5m 39s	/patch-unit-root.txt	root in the patch failed.
+1 💚	asflicense	1m 2s		The patch does not generate ASF License warnings.
		143m 31s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/1/artifact/out/Dockerfile
GITHUB PR	#2697
Optional Tests	dupname asflicense mvnsite unit shellcheck shelldocs
uname	Linux 745cc0178d92 4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `f9a073c`
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/1/testReport/
Max. process+thread count	578 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/1/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

aajisaka · 2021-02-13T06:36:44Z

Let's upgrade Yetus 0.13.0 in the Jenkinsfile instead of yetus-wrapper to use Yetus 0.13.0 in the pre-commit job.
Note that Yetus 0.13.0 dropped Python 2 support and I want to merge #1738 first.

hadoop-yetus · 2021-02-15T15:32:40Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/2/console in case of problems.

amahussein · 2021-02-15T15:45:17Z

Let's upgrade Yetus 0.13.0 in the Jenkinsfile instead of yetus-wrapper to use Yetus 0.13.0 in the pre-commit job.
Note that Yetus 0.13.0 dropped Python 2 support and I want to merge #1738 first.

Thanks @aajisaka ! I removed the version change.
Hopefully, the DOCKER_EXTRAARGS will be picked correctly by yetus.

hadoop-yetus · 2021-02-15T17:34:40Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 40s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 46s		trunk passed
+1 💚	mvnsite	24m 43s		trunk passed
+1 💚	shadedclient	13m 26s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	19m 56s		the patch passed
+1 💚	mvnsite	20m 54s		the patch passed
-1 ❌	shellcheck	0m 1s	/diff-patch-shellcheck.txt	The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)
+1 💚	shelldocs	0m 17s		There were no new shelldocs issues.
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
-1 ❌	shadedclient	1m 44s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	5m 48s	/patch-unit-root.txt	root in the patch failed.
+1 💚	asflicense	0m 53s		The patch does not generate ASF License warnings.
		123m 20s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/2/artifact/out/Dockerfile
GITHUB PR	#2697
Optional Tests	dupname asflicense mvnsite unit shellcheck shelldocs
uname	Linux ee2aea590a8b 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `bad6038`
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/2/testReport/
Max. process+thread count	535 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/2/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-02-17T15:14:17Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/3/console in case of problems.

amahussein · 2021-02-17T15:31:08Z

I think that we can test that change by removing the "-Djava.security.egd=file:/dev/./urandom" from TestMRIntermediateDataEncryption. If the build did not hang forever, it implies that the DOCKER_EXTRAARGS were picked by Yetus (see MAPREDUCE-7079)

hadoop-yetus · 2021-02-17T17:35:59Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 43s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	34m 28s		trunk passed
+1 💚	mvnsite	25m 59s		trunk passed
+1 💚	shadedclient	14m 51s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	27m 55s		the patch passed
+1 💚	mvnsite	26m 56s		the patch passed
-1 ❌	shellcheck	0m 0s	/diff-patch-shellcheck.txt	The patch generated 1 new + 1 unchanged - 0 fixed = 2 total (was 1)
+1 💚	shelldocs	0m 17s		There were no new shelldocs issues.
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
-1 ❌	shadedclient	1m 56s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	6m 57s	/patch-unit-root.txt	root in the patch failed.
+0 🆗	asflicense	0m 44s		ASF License check generated no output?
		143m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/3/artifact/out/Dockerfile
GITHUB PR	#2697
Optional Tests	dupname asflicense mvnsite unit shellcheck shelldocs
uname	Linux 13a1355e6c4e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0ed9e36`
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/3/testReport/
Max. process+thread count	535 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/3/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-02-17T22:56:33Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/4/console in case of problems.

hadoop-yetus · 2021-02-18T01:21:19Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 32s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 25s		trunk passed
+1 💚	mvnsite	24m 37s		trunk passed
+1 💚	shadedclient	13m 22s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	19m 49s		the patch passed
+1 💚	mvnsite	19m 34s		the patch passed
-1 ❌	shellcheck	0m 0s	/diff-patch-shellcheck.txt	The patch generated 3 new + 0 unchanged - 1 fixed = 3 total (was 1)
+1 💚	shelldocs	0m 18s		There were no new shelldocs issues.
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	shadedclient	13m 14s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	19m 2s		root in the patch passed.
+1 💚	asflicense	1m 3s		The patch does not generate ASF License warnings.
		146m 1s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/4/artifact/out/Dockerfile
GITHUB PR	#2697
Optional Tests	dupname asflicense mvnsite unit shellcheck shelldocs
uname	Linux 0e506e982959 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e391844`
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/4/testReport/
Max. process+thread count	538 (vs. ulimit of 5500)
modules	C: . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/4/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-02-18T16:45:30Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/5/console in case of problems.

hadoop-yetus · 2021-02-19T09:39:14Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	14m 46s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚		0m 0s	test4tests	The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 42s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 25s		trunk passed
+1 💚	compile	20m 35s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	18m 8s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	3m 56s		trunk passed
+1 💚	mvnsite	25m 7s		trunk passed
+1 💚	shadedclient	13m 19s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	7m 22s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	7m 51s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+0 🆗	spotbugs	61m 10s		Both FindBugs and SpotBugs are enabled, using SpotBugs.
+1 💚	spotbugs	32m 42s		trunk passed
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	mvninstall	20m 4s		the patch passed
+1 💚	compile	20m 14s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	20m 14s		the patch passed
+1 💚	compile	18m 13s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	18m 13s		the patch passed
+1 💚	checkstyle	3m 52s		the patch passed
+1 💚	mvnsite	20m 3s		the patch passed
-1 ❌	shellcheck	0m 2s	/diff-patch-shellcheck.txt	The patch generated 3 new + 0 unchanged - 1 fixed = 3 total (was 1)
+1 💚	shelldocs	0m 17s		There were no new shelldocs issues.
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	shadedclient	13m 23s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	7m 25s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	7m 47s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
-1 ❌	spotbugs	0m 48s	/patch-spotbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt	hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient cannot run computeBugHistory from spotbugs
-1 ❌	spotbugs	31m 45s	/patch-spotbugs-root.txt	root cannot run computeBugHistory from spotbugs
			_ Other Tests _
-1 ❌	unit	684m 42s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 48s		The patch does not generate ASF License warnings.
		1015m 1s

Reason	Tests
Failed junit tests	hadoop.yarn.server.router.clientrm.TestFederationClientInterceptor
	hadoop.yarn.client.api.impl.TestAMRMClient
	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor
	hadoop.hdfs.server.balancer.TestBalancer

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/5/artifact/out/Dockerfile
GITHUB PR	#2697
Optional Tests	dupname asflicense mvnsite unit shellcheck shelldocs compile javac javadoc mvninstall shadedclient spotbugs checkstyle
uname	Linux 629055b536c8 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e391844`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/5/testReport/
Max. process+thread count	2962 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/5/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 spotbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-02-19T17:59:42Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/6/console in case of problems.

hadoop-yetus · 2021-02-19T23:26:40Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚		0m 0s	test4tests	The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 55s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 12s		trunk passed
+1 💚	compile	20m 48s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	18m 10s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	3m 53s		trunk passed
+1 💚	mvnsite	25m 4s		trunk passed
+1 💚	shadedclient	13m 40s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	7m 25s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	7m 43s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+0 🆗	spotbugs	31m 48s		Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	32m 31s		trunk passed
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	mvninstall	20m 19s		the patch passed
+1 💚	compile	20m 8s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	20m 8s		the patch passed
+1 💚	compile	18m 14s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	18m 14s		the patch passed
+1 💚	checkstyle	3m 56s		the patch passed
+1 💚	mvnsite	20m 0s		the patch passed
-1 ❌	shellcheck	0m 1s	/diff-patch-shellcheck.txt	The patch generated 2 new + 0 unchanged - 1 fixed = 2 total (was 1)
+1 💚	shelldocs	0m 18s		There were no new shelldocs issues.
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
-1 ❌	shadedclient	1m 50s		patch has errors when building and testing our client artifacts.
+1 💚	javadoc	7m 21s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	7m 55s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	findbugs	32m 54s		the patch passed
			_ Other Tests _
-1 ❌	unit	23m 38s	/patch-unit-root.txt	root in the patch failed.
+1 💚	asflicense	1m 10s		The patch does not generate ASF License warnings.
		327m 56s

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/6/artifact/out/Dockerfile
GITHUB PR	#2697
Optional Tests	dupname asflicense mvnsite unit shellcheck shelldocs compile javac javadoc mvninstall shadedclient findbugs checkstyle
uname	Linux 37bbf93c4ef4 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `d28b6f9`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/6/testReport/
Max. process+thread count	3150 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/6/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2021-02-20T02:00:39Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/7/console in case of problems.

hadoop-yetus · 2021-02-20T20:00:38Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 33s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 1s		No case conflicting files found.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚		0m 0s	test4tests	The patch appears to include 1 new or modified test files.
			_ trunk Compile Tests _
+0 🆗	mvndep	14m 45s		Maven dependency ordering for branch
+1 💚	mvninstall	20m 25s		trunk passed
+1 💚	compile	22m 15s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	compile	20m 57s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	checkstyle	5m 13s		trunk passed
+1 💚	mvnsite	30m 13s		trunk passed
+1 💚	shadedclient	17m 48s		branch has no errors when building and testing our client artifacts.
+1 💚	javadoc	9m 47s		trunk passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	9m 47s		trunk passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+0 🆗	spotbugs	42m 25s		Used deprecated FindBugs config; considering switching to SpotBugs.
+1 💚	findbugs	43m 9s		trunk passed
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 32s		Maven dependency ordering for patch
+1 💚	mvninstall	24m 33s		the patch passed
+1 💚	compile	21m 17s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javac	21m 17s		the patch passed
+1 💚	compile	23m 21s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	javac	23m 21s		the patch passed
+1 💚	checkstyle	4m 58s		the patch passed
+1 💚	mvnsite	24m 52s		the patch passed
-1 ❌	shellcheck	0m 1s	/diff-patch-shellcheck.txt	The patch generated 3 new + 0 unchanged - 1 fixed = 3 total (was 1)
+1 💚	shelldocs	0m 16s		There were no new shelldocs issues.
+1 💚	whitespace	0m 0s		The patch has no whitespace issues.
+1 💚	shadedclient	17m 55s		patch has no errors when building and testing our client artifacts.
+1 💚	javadoc	9m 34s		the patch passed with JDK Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04
+1 💚	javadoc	9m 26s		the patch passed with JDK Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
+1 💚	findbugs	43m 54s		the patch passed
			_ Other Tests _
-1 ❌	unit	698m 35s	/patch-unit-root.txt	root in the patch passed.
+1 💚	asflicense	1m 53s		The patch does not generate ASF License warnings.
		1081m 13s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks
	hadoop.yarn.server.resourcemanager.nodelabels.TestRMNodeLabelsManager
	hadoop.tools.dynamometer.TestDynamometerInfra
	hadoop.yarn.sls.TestReservationSystemInvariants

Subsystem	Report/Notes
Docker	ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/7/artifact/out/Dockerfile
GITHUB PR	#2697
Optional Tests	dupname asflicense mvnsite unit shellcheck shelldocs compile javac javadoc mvninstall shadedclient findbugs checkstyle
uname	Linux c0bc17bd2092 4.15.0-60-generic #67-Ubuntu SMP Thu Aug 22 16:55:30 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `4891e68`
Default Java	Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.10+9-Ubuntu-0ubuntu1.20.04 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_282-8u282-b08-0ubuntu1~20.04-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/7/testReport/
Max. process+thread count	3707 (vs. ulimit of 5500)
modules	C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient . U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/7/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0 findbugs=4.0.6
Powered by	Apache Yetus 0.13.0-SNAPSHOT https://yetus.apache.org

This message was automatically generated.

asf-cloudbees-jenkins-ci-hadoop · 2021-02-22T14:48:35Z

(!) A patch to the testing environment has been detected.
Re-executing against the patched versions to perform further tests.
The console is at https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-2697/8/console in case of problems.

amahussein force-pushed the hadoop-16810 branch from 9036886 to ade6c68 Compare February 15, 2021 15:30

amahussein force-pushed the hadoop-16810 branch from ade6c68 to bb5af47 Compare February 17, 2021 15:11

amahussein force-pushed the hadoop-16810 branch from bb5af47 to cb5b0d6 Compare February 17, 2021 22:54

amahussein force-pushed the hadoop-16810 branch from 1299c26 to 6a26af2 Compare February 19, 2021 17:57

amahussein force-pushed the hadoop-16810 branch from 6a26af2 to 85dff3c Compare February 20, 2021 01:58

amahussein added 3 commits February 22, 2021 08:34

HADOOP-16810. Increase entropy on precommit Linux VMs

6c77ec9

HADOOP-16810. check intermediate encryption works

1b2770a

HADOOP-16810. remove double quotes

2c8a417

amahussein force-pushed the hadoop-16810 branch from 85dff3c to 2c8a417 Compare February 22, 2021 14:45

HADOOP-16810. Increase entropy on precommit Linux VMs #2697

Are you sure you want to change the base?

HADOOP-16810. Increase entropy on precommit Linux VMs #2697

Uh oh!

Conversation

amahussein commented Feb 12, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

hadoop-yetus commented Feb 12, 2021

Uh oh!

hadoop-yetus commented Feb 13, 2021

Uh oh!

aajisaka commented Feb 13, 2021

Uh oh!

hadoop-yetus commented Feb 15, 2021

Uh oh!

amahussein commented Feb 15, 2021

Uh oh!

hadoop-yetus commented Feb 15, 2021

Uh oh!

hadoop-yetus commented Feb 17, 2021

Uh oh!

amahussein commented Feb 17, 2021

Uh oh!

hadoop-yetus commented Feb 17, 2021

Uh oh!

hadoop-yetus commented Feb 17, 2021

Uh oh!

hadoop-yetus commented Feb 18, 2021

Uh oh!

hadoop-yetus commented Feb 18, 2021

Uh oh!

hadoop-yetus commented Feb 19, 2021

Uh oh!

hadoop-yetus commented Feb 19, 2021

Uh oh!

hadoop-yetus commented Feb 19, 2021

Uh oh!

hadoop-yetus commented Feb 20, 2021

Uh oh!

hadoop-yetus commented Feb 20, 2021

Uh oh!

asf-cloudbees-jenkins-ci-hadoop bot commented Feb 22, 2021

Uh oh!

Uh oh!

amahussein commented Feb 12, 2021 •

edited

Loading