HADOOP-16547. make sure that s3guard prune sets up the FS

[steveloughran]

initial patch; not done the full testing yet

Change-Id: Iaf71561cef6c797a3c66fed110faf08da6cac361

[steveloughran]

manually verified this patch works through DT games, as well as ITest run. All good

[bgaborg]

If we want this to be stable, then we need a test that actually we will have a filesystem after running maybeInitFilesystem. Please add a test so we don't remove this fix by accident.

[steveloughran]

I'm adding the test with the caveat that because only certain configs could trigger it (no fs.s3a.ddb.region) even if there was a regression you can't be confident it is valid. Though if it fails for someone, we can be happy that they have found that regression.

[steveloughran]

tested: s3 ireland; didnt rerun the manual tests this time.

[bgaborg]

Thanks @steveloughran for extending the test for prune, but it would be better to have the method itself tested: the protected boolean maybeInitFilesystem, so if you could add a new test asserting that this method creates a filesystem.
That way it would be more stable, and we would test if the change it that method works.

[steveloughran]

will do

[steveloughran]

added tests; tested s3 ireland

[hadoop-yetus]

🎊 +1 overall

Vote	Subsystem	Runtime	Comment
0	reexec	78	Docker mode activated.
		_ Prechecks _
+1	dupname	0	No case conflicting files found.
+1	@author	0	The patch does not contain any @author tags.
+1	test4tests	0	The patch appears to include 1 new or modified test files.
		_ trunk Compile Tests _
+1	mvninstall	1200	trunk passed
+1	compile	32	trunk passed
+1	checkstyle	24	trunk passed
+1	mvnsite	36	trunk passed
+1	shadedclient	784	branch has no errors when building and testing our client artifacts.
+1	javadoc	27	trunk passed
0	spotbugs	59	Used deprecated FindBugs config; considering switching to SpotBugs.
+1	findbugs	56	trunk passed
		_ Patch Compile Tests _
+1	mvninstall	32	the patch passed
+1	compile	26	the patch passed
+1	javac	26	the patch passed
+1	checkstyle	18	the patch passed
+1	mvnsite	31	the patch passed
+1	whitespace	0	The patch has no whitespace issues.
+1	shadedclient	844	patch has no errors when building and testing our client artifacts.
+1	javadoc	23	the patch passed
+1	findbugs	61	the patch passed
		_ Other Tests _
+1	unit	70	hadoop-aws in the patch passed.
+1	asflicense	29	The patch does not generate ASF License warnings.
		3465

Subsystem	Report/Notes
Docker	Client=19.03.0 Server=19.03.0 base: https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/6/artifact/out/Dockerfile
GITHUB PR	#1402
Optional Tests	dupname asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle
uname	Linux f402829dea9e 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	personality/hadoop.sh
git revision	trunk / 419dd0f
Default Java	1.8.0_222
Test Results	https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/6/testReport/
Max. process+thread count	320 (vs. ulimit of 5500)
modules	C: hadoop-tools/hadoop-aws U: hadoop-tools/hadoop-aws
Console output	https://builds.apache.org/job/hadoop-multibranch/job/PR-1402/6/console
versions	git=2.7.4 maven=3.3.9 findbugs=3.1.0-RC1
Powered by	Apache Yetus 0.10.0 http://yetus.apache.org

This message was automatically generated.

[bgaborg]

Thanks for the tests @steveloughran, looks good to me.
Tested against ireland. Results: one known flaky testConcurrentTableCreations failure (not related) and testMR failures (not related, known).
+1