Add listDirectory() to Path type and deprecate listFiles() (#6581)

Signed-off-by: Ben Sherman <[email protected]>

Author: bentsherman

docs/reference/stdlib-namespaces.md

@@ -83,7 +83,7 @@ The global namespace contains globally available constants and functions.
   This function returns a collection if the glob pattern yields zero or multiple files. Use `files()` to get a collection of files.
   :::
 
-`files( filePattern: String, [options] ) -> List<Path>`
+`files( filePattern: String, [options] ) -> Iterable<Path>`
 : Get a collection of files from a file name or glob pattern. Supports the same options as `file()`.
 : See also: {ref}`channel.fromPath <channel-path>`.
 

docs/reference/stdlib-types.md

@@ -509,30 +509,32 @@ The following operations are supported for paths:
 `<< : (Path, String)`
 : Appends text to a file without replacing existing content. Equivalent to `append()`.
 
-<h3>Getting attributes</h3>
+<h3>Filesystem attributes</h3>
 
-The following methods are useful for getting attributes of a path:
+The following properties are available:
 
-`exists() -> Boolean`
-: Returns `true` if the path exists.
+`baseName: String`
+: The path name without its extension, e.g. `/some/path/file.tar.gz` -> `file.tar`.
 
-`getBaseName() -> String`
-: Gets the path name without its extension, e.g. `/some/path/file.tar.gz` -> `file.tar`.
+`extension: String`
+: The path extension, e.g. `/some/path/file.txt` -> `txt`.
 
-`getExtension() -> String`
-: Gets the path extension, e.g. `/some/path/file.txt` -> `txt`.
+`name: String`
+: The path name, e.g. `/some/path/file.txt` -> `file.txt`.
 
-`getName() -> String`
-: Gets the path name, e.g. `/some/path/file.txt` -> `file.txt`.
+`parent: Path`
+: The path parent path, e.g. `/some/path/file.txt` -> `/some/path`.
 
-`getSimpleName() -> String`
-: Gets the path name without any extension, e.g. `/some/path/file.tar.gz` -> `file`.
+`scheme: String`
+: The path URI scheme, e.g. `s3://some-bucket/hello.txt` -> `s3`.
 
-`getParent() -> Path`
-: Gets the path parent path, e.g. `/some/path/file.txt` -> `/some/path`.
+`simpleName: String`
+: The path name without any extension, e.g. `/some/path/file.tar.gz` -> `file`.
 
-`getScheme() -> String`
-: Gets the path URI scheme, e.g. `s3://some-bucket/hello.txt` -> `s3`.
+The following methods are available for getting filesystem attributes:
+
+`exists() -> Boolean`
+: Returns `true` if the path exists.
 
 `isDirectory() -> Boolean`
 : Returns `true` if the path is a directory.
@@ -578,32 +580,17 @@ The following methods are useful for getting attributes of a path:
 
 The following methods are available for reading files:
 
-`eachByte( action: (byte) -> () )`
-: Iterates over the file, applying the specified closure to each byte.
-
 `eachLine( action: (String) -> () )`
 : Iterates over the file, applying the specified closure to each line.
 
-`getBytes() -> byte[]`
-: Returns the file content as a byte array.
-
 `getText() -> String`
 : Returns the file content as a string.
 
-`newInputStream() -> InputStream`
-: Returns an [InputStream](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/InputStream.html) object to read a binary file.
-
-`newReader() -> Reader`
-: Returns a [Reader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/Reader.html) object to read a text file.
-
 `readLines() -> List<String>`
 : Reads the file line by line and returns the content as a list of strings.
 
-`withInputStream( action: (InputStream) -> () )`
-: Opens a file for reading and lets you access it with an [InputStream](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/InputStream.html) object.
-
-`withReader( action: (Reader) -> () )`
-: Opens a file for reading and lets you access it with a [Reader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/Reader.html) object.
+`withReader( action: (BufferedReader) -> () )`
+: Invokes the given closure with a [BufferedReader](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/BufferedReader.html), which can be used to read the file one line at a time using the `readLine()` method.
 
 <h3>Writing</h3>
 
@@ -612,30 +599,9 @@ The following methods are available for writing to files:
 `append( text: String )`
 : Appends text to a file without replacing existing content.
 
-`newOutputStream() -> OutputStream`
-: Creates an [OutputStream](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/OutputStream.html) object that allows you to write binary data to a file.
-
-`newPrintWriter() -> PrintWriter`
-: Creates a [PrintWriter](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/PrintWriter.html) object that allows you to write formatted text to a file.
-
-`newWriter() -> Writer`
-: Creates a [Writer](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/Writer.html) object that allows you to save text data to a file.
-
-`setBytes( bytes: byte[] )`
-: Writes a byte array to a file. Equivalent to setting the `bytes` property.
-
 `setText( text: String )`
 : Writes text to a file. Equivalent to setting the `text` property.
 
-`withOutputStream( action: (OutputStream) -> () )`
-: Applies the specified closure to an [OutputStream](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/OutputStream.html) object, closing it when finished.
-
-`withPrintWriter( action: (PrintWriter) -> () )`
-: Applies the specified closure to a [PrintWriter](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/PrintWriter.html) object, closing it when finished.
-
-`withWriter( action: (Writer) -> () )`
-: Applies the specified closure to a [Writer](https://docs.oracle.com/en/java/javase/17/docs/api/java.base/java/io/Writer.html) object, closing it when finished.
-
 `write( text: String )`
 : Writes a string to a file, replacing any existing content.
 
@@ -691,12 +657,6 @@ The following methods are available for manipulating files and directories in a
 `getPermissions() -> String`
 : Returns a file's permissions using the [symbolic notation](http://en.wikipedia.org/wiki/File_system_permissions#Symbolic_notation), e.g. `'rw-rw-r--'`.
 
-`list() -> List<String>`
-: Returns the first-level elements (files and directories) of a directory as a list of strings.
-
-`listFiles() -> List<Path>`
-: Returns the first-level elements (files and directories) of a directory as a list of Paths.
-
 `mkdir() -> Boolean`
 : Creates a directory at the given path, returning `true` if the directory is created successfully, and `false` otherwise:
 
@@ -759,24 +719,23 @@ The following methods are available for manipulating files and directories in a
 
 The following methods are available for listing and traversing directories:
 
-`eachDir( action: (Path) -> () )`
-: Iterates through first-level directories only.
-
-`eachDirMatch( nameFilter: String, action: (Path) -> () )`
-: Iterates through directories whose names match the given filter.
-
-`eachDirRecurse( action: (Path) -> () )`
-: Iterates through directories depth-first (regular files are ignored).
-
 `eachFile( action: (Path) -> () )`
 : Iterates through first-level files and directories.
 
-`eachFileMatch( nameFilter: String, action: (Path) -> () )`
-: Iterates through files and directories whose names match the given filter.
-
 `eachFileRecurse( action: (Path) -> () )`
 : Iterates through files and directories depth-first.
 
+`listDirectory() -> Iterable<Path>`
+: :::{versionadded} 26.04.0
+  :::
+: Returns the first-level elements (files and directories) in a directory.
+
+`listFiles() -> Iterable<Path>`
+: :::{deprecated}
+  Use `listDirectory()` instead.
+  :::
+: Returns the first-level elements (files and directories) in a directory.
+
 <h3>Splitting files</h3>
 
 The following methods are available for splitting and counting the records in files:

modules/nextflow/src/main/groovy/nextflow/processor/TaskPath.groovy

@@ -334,11 +334,14 @@ final class TaskPath implements Path, PathEscapeAware {
         FilesEx.or(getFileName(),other)
     }
 
-
     String getPermissions() {
         FilesEx.getPermissions(target)
     }
 
+    Collection<Path> listDirectory() {
+        FilesEx.listDirectory(target)
+    }
+
     boolean setPermissions( String permissions ) {
         throw new UnsupportedOperationException()
     }

modules/nextflow/src/test/groovy/nextflow/processor/TaskPathTest.groovy

@@ -87,7 +87,6 @@ class TaskPathTest extends Specification {
 
         cleanup:
         folder.deleteDir()
-
     }
 
     def 'should validate equals method' () {
@@ -111,10 +110,8 @@ class TaskPathTest extends Specification {
         new TaskPath(p1, 'foo.txt').equals(t2)
 
         !t1.equals(t3)
-
     }
 
-
     def 'should validate operator equality' () {
         given:
         def p1 = Paths.get('/foo/bar/baz.txt')
@@ -138,7 +135,6 @@ class TaskPathTest extends Specification {
 
         expect:
         new TaskPath(Paths.get('/foo')).size() == 0
-
     }
 
     @Ignore
@@ -159,10 +155,8 @@ class TaskPathTest extends Specification {
 
         cleanup:
         folder.deleteDir()
-
     }
 
-
     def 'should serialised task path' () {
 
         given:
@@ -174,7 +168,22 @@ class TaskPathTest extends Specification {
         copy.equals(p)
     }
 
-}
+    def 'should support directory listing' () {
 
+        given:
+        def folder = Files.createTempDirectory('test')
+        folder.resolve('hello.txt').text = 'Hello world'
 
+        when:
+        def path = new TaskPath(folder, 'test')
+        def result = path.listDirectory()
+
+        then:
+        result.size() == 1
+        result[0].name == 'hello.txt'
+
+        cleanup:
+        folder.deleteDir()
+    }
 
+}

modules/nf-commons/src/main/nextflow/extension/FilesEx.groovy

@@ -626,7 +626,7 @@ class FilesEx {
      * @return A list of strings or {@code null} if the path is not a folder
      */
     static String[] list(Path self) {
-        listFiles(self).collect { getName(it) } as String[]
+        return listFiles0(self).collect { getName(it) } as String[]
     }
 
     /**
@@ -637,33 +637,48 @@ class FilesEx {
      * @return A list of {@code Path} or {@code null} if the path is not a folder
      */
     static Path[] listFiles(Path self, @ClosureParams(value = FromString.class, options = ["java.nio.file.Path", "java.nio.file.Path,java.nio.file.attribute.BasicFileAttributes"]) Closure<Boolean> filter=null) {
+        return listFiles0(self, filter) as Path[]
+    }
+
+    /**
+     * List the files in a directory.
+     *
+     * This method is preferred over `listFiles()` in the Nextflow language
+     * because it returns a Collection<Path> instead of a Path[].
+     *
+     * @param self
+     */
+    static Collection<Path> listDirectory(Path self) {
+        return listFiles0(self)
+    }
+
+    private static Collection<Path> listFiles0(Path self, Closure<Boolean> filter=null) {
 
         if( !self.isDirectory() )
             return null
 
-        def result = []
+        final result = []
+
         Files.walkFileTree(self, new SimpleFileVisitor<Path>() {
 
             FileVisitResult visitFile(Path file, BasicFileAttributes attrs) {
-                if( filter==null || invokeFilter(filter,file,attrs) )
-                    result.add( file )
+                if( filter == null || invokeFilter(filter, file, attrs) )
+                    result.add(file)
                 FileVisitResult.CONTINUE
             }
 
             FileVisitResult preVisitDirectory(Path dir, BasicFileAttributes attrs) {
-                if( self == dir )
+                if( self == dir ) {
                     FileVisitResult.CONTINUE
-
+                }
                 else {
                     result.add(dir)
                     FileVisitResult.SKIP_SUBTREE
                 }
             }
-
         } )
 
-        return result as Path[]
-
+        return result
     }
 
     @CompileStatic