Skip to content

Commit f348a8b

Browse files
WillymontazWilliam Montaz
andauthored
Fix method getStatus in ViewFileSystem (apache#7)
This method was not present in HDP2 and has been added in HDP3. But instead of providing res.remainingPath to the res.targetFileSystem, it provides the complete path given as argument to the method. Detail of the issue: - when ViewFsFileSystem acts on a given path, it tries to 'resolve' the path to get the underlying FileSystem that can access it. For instance, for a path like /tmp/a/b or viewfs://root/tmp/a/b, due to our configuration, the resolved fs is a ChRootedFileSystem, that targets the URI hdfs://root-preprod-pa4/tmp (which it self relies on a distributed filesystem that targets the couple of namenodes for the root namespace). - such a ChRootedFileSystem expects to be given either absolute paths (no scheme nor authority) or paths targeting the same scheme://authority, and it will then add the path prefix to the path (as chroot would do). For instance, if the ChRootedFileSystem URI is hdfs://root-preprod-pa4/tmp, it will transform: - /a/b/c to /tmp/a/b/c and will use the DistributedFileSystem for hdfs://root-preprod-pa4 to perform the remote call - hdfs://root-preprod-pa4/a/b/c to /tmp/a/b/c (and same as above) - /tmp/a/b/c or hdfs://root-preprod-pa4/tmp to /tmp/tmp/a/b/c (note the double /tmp, that was just for fun...) In the end, if get status is called on a viewfs file system that will use some underlying ChRootedFileSystem, using an absolute path, it will not fail, but preprend a prefix, causing most of the time undesired path (the /tmp/tmp example above), and if it is called using a scheme+authority (viewfs://root/tmp/a/b), it will fail, because it is not the scheme expected by the ChRootedFileSystem. As a matter of fact, the resolve method does the job of finding the underlying filesystem and remove path prefixes in the case of ChRootedFileSystem and puts the result in the remainingPath variable. Co-authored-by: William Montaz <[email protected]>
1 parent 420aabf commit f348a8b

File tree

1 file changed

+1
-1
lines changed
  • hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs

1 file changed

+1
-1
lines changed

hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFileSystem.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -996,7 +996,7 @@ public FsStatus getStatus(Path p) throws IOException {
996996
}
997997
InodeTree.ResolveResult<FileSystem> res = fsState.resolve(
998998
getUriPath(p), true);
999-
return res.targetFileSystem.getStatus(p);
999+
return res.targetFileSystem.getStatus(res.remainingPath);
10001000
}
10011001

10021002
/**

0 commit comments

Comments
 (0)