Update documentation to reflect the primacy of 'files()' and remove the caveats about traversability. Add new caveats about limited support for resource readers.

Author: jaraco

importlib_resources/docs/changelog.rst

@@ -2,12 +2,14 @@
  importlib_resources NEWS
 ==========================
 
-1.1.0 (2020-01-19)
+1.1.0 (2020-02-16)
 ==================
 * Add support for retrieving resources from subdirectories of packages
-  through the new ``get()`` function, which returns a ``Traversable``
+  through the new ``files()`` function, which returns a ``Traversable``
   object with ``joinpath`` and ``read_*`` interfaces matching those
-  of ``pathlib.Path`` objects.
+  of ``pathlib.Path`` objects. This new function supersedes all of the
+  previous functionality as it provides a more general-purpose access
+  to a package's resources.
 
 1.0.2 (2018-11-01)
 ==================

importlib_resources/docs/index.rst

@@ -7,17 +7,16 @@ in Python packages.  It provides functionality similar to ``pkg_resources``
 `Basic Resource Access`_ API, but without all of the overhead and performance
 problems of ``pkg_resources``.
 
-In our terminology, a *resource* is a file that is located within an
-importable `Python package`_.  Resources can live on the file system, in a zip
-file, or in any place that has a loader_ supporting the appropriate API for
-reading resources.  Directories are not resources.
-
-``importlib_resources`` is a backport of Python 3.7's standard library
-`importlib.resources`_ module for Python 2.7, and 3.4 through 3.6.  Users of
-Python 3.7 and beyond are encouraged to use the standard library module, and
-in fact for these versions, ``importlib_resources`` just shadows that module.
+In our terminology, a *resource* is a file tree that is located within an
+importable `Python package`_.  Resources can live on the file system or in a
+zip file, with limited support for loader_ supporting the appropriate API for
+reading resources.
+
+``importlib_resources`` is a backport of Python 3.9's standard library
+`importlib.resources`_ module for Python 2.7, and 3.5 through 3.8.  Users of
+Python 3.9 and beyond are encouraged to use the standard library module.
 Developers looking for detailed API descriptions should refer to the Python
-3.7 standard library documentation.
+3.9 standard library documentation.
 
 The documentation here includes a general :ref:`usage <using>` guide and a
 :ref:`migration <migration>` guide for projects that want to adopt

importlib_resources/docs/migration.rst

@@ -17,11 +17,6 @@ access`_ APIs:
 * ``pkg_resources.resource_listdir()``
 * ``pkg_resources.resource_isdir()``
 
-Keep in mind that ``pkg_resources`` defines *resources* to include
-directories.  ``importlib_resources`` does not treat directories as resources;
-since only files are allowed as resources, file names in the
-``importlib_resources`` API may *not* include path separators (e.g. slashes).
-
 
 pkg_resources.resource_filename()
 =================================
@@ -34,9 +29,11 @@ that ``pkg_resources()`` also *implicitly* cleans up this temporary file,
 without control over its lifetime by the programmer.
 
 ``importlib_resources`` takes a different approach.  Its equivalent API is the
-``path()`` function, which returns a context manager providing a
-:py:class:`pathlib.Path` object.  This means users have both the flexibility
-and responsibility to manage the lifetime of the temporary file.  Note though
+``files()`` function, which returns a Traversable object implementing a
+subset of the
+:py:class:`pathlib.Path` interface suitable for reading the contents and
+provides a wrapper for creating a temporary file on the system in a
+context whose lifetime is managed by the user.  Note though
 that if the resource is *already* on the file system, ``importlib_resources``
 still returns a context manager, but nothing needs to get cleaned up.
 
@@ -46,7 +43,8 @@ Here's an example from ``pkg_resources()``::
 
 The best way to convert this is with the following idiom::
 
-    with importlib_resources.path('my.package', 'resource.dat') as path:
+    ref = importlib_resources.files('my.package') / 'resource.dat'
+    with importlib_resources.trees.as_file(ref) as path:
         # Do something with path.  After the with-statement exits, any
         # temporary file created will be immediately cleaned up.
 
@@ -56,8 +54,9 @@ to stick around for a while?  One way of doing this is to use an
 
     from contextlib import ExitStack
     file_manager = ExitStack()
+    ref = importlib_resources.files('my.package') / 'resource.dat'
     path = file_manager.enter_context(
-        importlib_resources.path('my.package', 'resource.dat'))
+        importlib_resources.trees.as_file(ref))
 
 Now ``path`` will continue to exist until you explicitly call
 ``file_manager.close()``.  What if you want the file to exist until the
@@ -67,8 +66,9 @@ process exits, or you can't pass ``file_manager`` around in your code?  Use an
     import atexit
     file_manager = ExitStack()
     atexit.register(file_manager.close)
+    ref = importlib_resources.files('my.package') / 'resource.dat'
     path = file_manager.enter_context(
-        importlib_resources.path('my.package', 'resource.dat'))
+        importlib_resources.trees.as_file(ref))
 
 Assuming your Python interpreter exits gracefully, the temporary file will be
 cleaned up when Python exits.
@@ -86,7 +86,8 @@ bytes.  E.g.::
 
 The equivalent code in ``importlib_resources`` is pretty straightforward::
 
-    with importlib_resources.open_binary('my.package', 'resource.dat') as fp:
+    ref = importlib_resources.files('my.package').joinpath('resource.dat')
+    with ref.open() as fp:
         my_bytes = fp.read()
 
 
@@ -103,7 +104,8 @@ following example is often written for clarity as::
 
 This can be easily rewritten like so::
 
-    contents = importlib_resources.read_binary('my.package', 'resource.dat')
+    ref = importlib_resources.files('my.package').joinpath('resource.dat')
+    contents = f.read_bytes()
 
 
 pkg_resources.resource_listdir()
@@ -117,19 +119,18 @@ but it does not recurse into subdirectories, e.g.::
 
 This is easily rewritten using the following idiom::
 
-    for entry in importlib_resources.contents('my.package.subpackage'):
-        print(entry)
+    for entry in importlib_resources.files('my.package.subpackage').iterdir():
+        print(entry.name)
 
 Note:
 
-* ``pkg_resources`` does not require ``subpackage`` to be a Python package,
-  but ``importlib_resources`` does.
-* ``importlib_resources.contents()`` returns an iterator, not a concrete
-  sequence.
+* ``Traversable.iterdir()`` returns *all* the entries in the
+  subpackage, i.e. both resources (files) and non-resources (directories).
+* ``Traversable.iterdir()`` returns additional traversable objects, which if
+  directories can also be iterated over (recursively).
+* ``Traversable.iterdir()``, like ``pathlib.Path`` returns an iterator, not a
+  concrete sequence.
 * The order in which the elements are returned is undefined.
-* ``importlib_resources.contents()`` returns *all* the entries in the
-  subpackage, i.e. both resources (files) and non-resources (directories).  As
-  with ``pkg_resources.listdir()`` it does not recurse.
 
 
 pkg_resources.resource_isdir()
@@ -141,20 +142,10 @@ a package is a directory or not::
     if pkg_resources.resource_isdir('my.package', 'resource'):
         print('A directory')
 
-Because ``importlib_resources`` explicitly does not define directories as
-resources, there's no direct equivalent.  However, you can ask whether a
-particular resource exists inside a package, and since directories are not
-resources you can infer whether the resource is a directory or a file.  Here
-is a way to do that::
+The ``importlib_resources`` equivalent is straightforward::
 
-    from importlib_resources import contents, is_resource
-    if 'resource' in contents('my.package') and \
-              not is_resource('my.package', 'resource'):
-      print('It must be a directory')
-
-The reason you have to do it this way and not just call
-``not is_resource('my.package', 'resource')`` is because this conditional will
-also return False when ``resource`` is not an entry in ``my.package``.
+    if importlib_resources.files('my.package').joinpath('resource').isdir():
+        print('A directory')
 
 
 .. _`basic resource access`: http://setuptools.readthedocs.io/en/latest/pkg_resources.html#basic-resource-access

importlib_resources/docs/using.rst

@@ -23,6 +23,8 @@ If you have a file system layout such as::
         one/
             __init__.py
             resource1.txt
+            resources1/
+                resource1.1.txt
         two/
             __init__.py
             resource2.txt
@@ -41,27 +43,40 @@ Each import statement gives you a Python *module* corresponding to the
 packages since packages are just special module instances that have an
 additional attribute, namely a ``__path__`` [#fn2]_.
 
-In this analogy then, resources are just files within a package directory, so
+In this analogy then, resources are just files or directories contained in a
+package directory, so
 ``data/one/resource1.txt`` and ``data/two/resource2.txt`` are both resources,
-as are the ``__init__.py`` files in all the directories.  However the package
-directories themselves are *not* resources; anything that contains other
-things (i.e. directories) are not themselves resources.
-
-Resources are always accessed relative to the package that they live in.  You
-cannot access a resource within a subdirectory inside a package.  This means
-that ``resource1.txt`` is a resource within the ``data.one`` package, but
-neither ``resource2.txt`` nor ``two/resource2.txt`` are resources within the
-``data`` package.  If a directory isn't a package, it can't be imported and
-thus can't contain resources.
-
-Even when this hierarchical structure isn't represented by physical files and
-directories, the model still holds.  So zip files can contain packages and
-resources, as could databases or other storage medium.  In fact, while
-``importlib_resources`` supports physical file systems and zip files by
-default, anything that can be loaded with a Python import system `loader`_ can
-provide resources, as long as the loader implements the `ResourceReader`_
-abstract base class.
+as are the ``__init__.py`` files in all the directories.
 
+Resources are always accessed relative to the package that they live in.
+``resource1.txt`` and ``resources1/resource1.1.txt`` are resources within
+the ``data.one`` package, and
+``two/resource2.txt`` is a resource within the
+``data`` package.
+
+
+Caveats
+=======
+
+Subdirectory Access
+-------------------
+
+Prior to importlib_resources 1.1 and the ``files()`` API, resources that were
+not direct descendents of a package's folder were inaccessible through the
+API, so in the example above ``resources1/resource1.1`` is not a resource of
+the ``data.one`` package and ``two/resource2.txt`` is not a resource of the
+``data`` package. Therefore, if subdirectory access is required, use the
+``files()`` API.
+
+Resource Reader Support
+-----------------------
+
+Due to the limitations on resource readers to access files beyond direct
+descendents of a package, the ``files()`` API does not rely
+on the importlib ResourceReader interface and thus only supports resources
+exposed by the built-in path and zipfile loaders. If support for arbitrary
+resource readers is required, the other API functions still support loading
+those resources.
 
 Example
 =======
@@ -93,25 +108,22 @@ This requires you to make Python packages of both ``email/tests`` and
 ``email/tests/data``, by placing an empty ``__init__.py`` files in each of
 those directories.
 
-**This is a requirement for importlib_resources too!**
-
 The problem with the ``pkg_resources`` approach is that, depending on the
-structure of your package, ``pkg_resources`` can be very inefficient even to
-just import.  ``pkg_resources`` is a sort of grab-bag of APIs and
-functionalities, and to support all of this, it sometimes has to do a ton of
-work at import time, e.g. to scan every package on your ``sys.path``.  This
+packages in your environment, ``pkg_resources`` can be expensive
+just to import.  This behavior
 can have a serious negative impact on things like command line startup time
 for Python implement commands.
 
-``importlib_resources`` solves this by being built entirely on the back of the
+``importlib_resources`` solves this performance challenge by being built
+entirely on the back of the
 stdlib :py:mod:`importlib`.  By taking advantage of all the efficiencies in
 Python's import system, and the fact that it's built into Python, using
 ``importlib_resources`` can be much more performant.  The equivalent code
 using ``importlib_resources`` would look like::
 
-    from importlib_resources import read_text
+    from importlib_resources import files
     # Reads contents with UTF-8 encoding and returns str.
-    eml = read_text('email.tests.data', 'message.eml')
+    eml = files('email.tests.data').joinpath('message.eml').read_text()
 
 
 Packages or package names
@@ -124,7 +136,7 @@ passed in, it must name an importable Python package, and this is first
 imported.  Thus the above example could also be written as::
 
     import email.tests.data
-    eml = read_text(email.tests.data, 'message.eml')
+    eml = files(email.tests.data).joinpath('message.eml').read_text()
 
 
 File system or zip file
@@ -143,8 +155,11 @@ to this temporary file as a :py:class:`pathlib.Path` object.  In order to
 properly clean up this temporary file, what's actually returned is a context
 manager that you can use in a ``with``-statement::
 
-    from importlib_resources import path
-    with path(email.tests.data, 'message.eml') as eml:
+    from importlib_resources import files
+    from importlib_resources.trees import as_file
+
+    source = files(email.tests.data).joinpath('message.eml')
+    with as_file(source) as eml:
         third_party_api_requiring_file_system_path(eml)
 
 You can use all the standard :py:mod:`contextlib` APIs to manage this context
@@ -162,12 +177,12 @@ manager.
    **No**::
 
        sys.path.append('relative/path/to/foo.whl')
-       resource_bytes('foo/data.dat')  # This will fail!
+       files('foo')  # This will fail!
 
    **Yes**::
 
        sys.path.append(os.path.abspath('relative/path/to/foo.whl'))
-       resource_bytes('foo/data.dat')
+       files('foo')
 
 Both relative and absolute paths work for Python 3.7 and newer.