TraversalError in MultiplexedPath.joinpath when parent in compound path is missing

[MattiasDC]

I have a regression between version 5.7.1 and 5.8.0. I think the cause is this commit.

I have a reproduction scenario in the zip.
To reproduce:

• unzip repro.zip
• cd repro
• pip install -e .
• python -m test.test --info

If you change the setup.cfg to use version 5.7.1, no error is raised
repro.zip

[MattiasDC]

The exception callstack that should occur:

Traceback (most recent call last):
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/importlib_resources/abc.py", line 119, in joinpath
    match = next(matches)
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/MDC/git/repro/test/test.py", line 8, in <module>
    test()
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/main.py", line 49, in decorated_main
    _run_hydra(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/utils.py", line 399, in _run_hydra
    hydra.show_info(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 655, in show_info
    options[info](config_name=config_name, overrides=overrides)
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 588, in _print_all_info
    self._print_config_info(config_name, overrides)
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 470, in _print_config_info
    self._print_search_path(config_name=config_name, overrides=overrides)
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 403, in _print_search_path
    cfg = self._get_cfg(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 163, in _get_cfg
    cfg = self.compose_config(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 564, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 146, in load_configuration
    return self._load_configuration_impl(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 239, in _load_configuration_impl
    defaults_list = create_defaults_list(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 719, in create_defaults_list
    defaults, tree = _create_defaults_list(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 689, in _create_defaults_list
    defaults_tree = _create_defaults_tree(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 337, in _create_defaults_tree
    ret = _create_defaults_tree_impl(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 420, in _create_defaults_tree_impl
    return _expand_virtual_root(repo, root, overrides, skip_missing)
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 262, in _expand_virtual_root
    subtree = _create_defaults_tree_impl(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 429, in _create_defaults_tree_impl
    update_package_header(repo=repo, node=parent)
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 244, in update_package_header
    loaded = repo.load_config(config_path=node.get_config_path())
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/config_repository.py", line 337, in load_config
    ret = self.delegate.load_config(config_path=config_path)
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/config_repository.py", line 86, in load_config
    source = self._find_object_source(
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/config_repository.py", line 130, in _find_object_source
    if source.is_config(config_path):
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/hydra/_internal/core_plugins/importlib_resources_config_source.py", line 91, in is_config
    res = files.joinpath(config_path)
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/importlib_resources/readers.py", line 87, in joinpath
    return super().joinpath(*descendants)
  File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/importlib_resources/abc.py", line 121, in joinpath
    raise TraversalError(
importlib_resources.abc.TraversalError: ('Target not found during traversal.', 'hydra', ['config.yaml'])

[MattiasDC]

Notice that I would've expected that this exception would be catched by this frame:
File "/home/MDC/git/repro/.env/lib/python3.8/site-packages/importlib_resources/readers.py", line 87, in joinpath

As per:

Am I missing something obvious here?

[MattiasDC]

Hi @jaraco, is this something you could have a look at as you recently committed in this code?

[jaraco]

Yes. I'll take a look. As described, it's difficult for me to see what's going on, so I'll need to download and unzip the repro to investigate deeper.

[jaraco]

I've pushed the repro to the issue-253 branch for easier testing.

[jaraco]

It works fine for me:

 repro issue-253 $ py -3.10 -m venv .venv
 repro issue-253 $ py -m pip install -q -U pip
 repro issue-253 $ py -m pip install -e .
Obtaining file:///Users/jaraco/draft/repro
  Preparing metadata (setup.py) ... done
Collecting hydra-core==1.1.0
  Using cached hydra_core-1.1.0-py3-none-any.whl (144 kB)
Collecting hydra-joblib-launcher==1.1.5
  Using cached hydra_joblib_launcher-1.1.5-py3-none-any.whl (5.2 kB)
Collecting importlib_resources==5.8.0
  Using cached importlib_resources-5.8.0-py3-none-any.whl (28 kB)
Collecting omegaconf==2.1.*
  Using cached omegaconf-2.1.2-py3-none-any.whl (74 kB)
Collecting antlr4-python3-runtime==4.8
  Using cached antlr4_python3_runtime-4.8-py3-none-any.whl
Collecting joblib>=0.14.0
  Using cached joblib-1.1.0-py2.py3-none-any.whl (306 kB)
Collecting PyYAML>=5.1.0
  Using cached PyYAML-6.0-cp310-cp310-macosx_11_0_arm64.whl (173 kB)
Installing collected packages: antlr4-python3-runtime, PyYAML, joblib, importlib_resources, omegaconf, hydra-core, hydra-joblib-launcher, test
  Running setup.py develop for test
Successfully installed PyYAML-6.0 antlr4-python3-runtime-4.8 hydra-core-1.1.0 hydra-joblib-launcher-1.1.5 importlib_resources-5.8.0 joblib-1.1.0 omegaconf-2.1.2 test-0.0.0
 repro issue-253 $ py -m test.test --info
Hydra 1.1.0
===========
Installed Hydra Plugins
***********************
        ConfigSource:
        -------------
                FileConfigSource
                ImportlibResourcesConfigSource
                StructuredConfigSource
        CompletionPlugin:
        -----------------
                BashCompletion
                FishCompletion
                ZshCompletion
        Launcher:
        ---------
                BasicLauncher
                JoblibLauncher
        Sweeper:
        --------
                BasicSweeper
        SearchPathPlugin:
        -----------------
                Dummy

Config search path
******************
| Provider                 | Search path                           |
--------------------------------------------------------------------
| hydra                    | pkg://hydra.conf                      |
| main                     | file:///Users/jaraco/draft/repro/test |
| dummy-search-path-plugin | pkg://test.config                     |
| schema                   | structured://                         |
--------------------------------------------------------------------

Defaults Tree
*************
<root>:
  hydra/config:
    hydra/output: default
    hydra/launcher: joblib
    hydra/sweeper: basic
    hydra/help: default
    hydra/hydra_help: default
    hydra/hydra_logging: default
    hydra/job_logging: default
    hydra/callbacks: null
    hydra/env: default
    _self_
  run

Defaults List
*************
| Config path                 | Package             | _self_ | Parent       | 
------------------------------------------------------------------------------
| hydra/output/default        | hydra               | False  | hydra/config |
| hydra/launcher/joblib       | hydra.launcher      | False  | hydra/config |
| hydra/sweeper/basic         | hydra.sweeper       | False  | hydra/config |
| hydra/help/default          | hydra.help          | False  | hydra/config |
| hydra/hydra_help/default    | hydra.hydra_help    | False  | hydra/config |
| hydra/hydra_logging/default | hydra.hydra_logging | False  | hydra/config |
| hydra/job_logging/default   | hydra.job_logging   | False  | hydra/config |
| hydra/env/default           | hydra.env           | False  | hydra/config |
| hydra/config                | hydra               | True   | <root>       |
| run                         |                     | False  | <root>       |
------------------------------------------------------------------------------
Config
******
{}

[jaraco]

Aha. So the issue seems only to arise only on older versions of Python.

 repro issue-253 $ py -3.8 -m venv .venv
 repro issue-253 $ py -m pip install -e .
Obtaining file:///Users/jaraco/draft/repro
  Preparing metadata (setup.py) ... done
Collecting hydra-core==1.1.0
  Using cached hydra_core-1.1.0-py3-none-any.whl (144 kB)
Collecting hydra-joblib-launcher==1.1.5
  Using cached hydra_joblib_launcher-1.1.5-py3-none-any.whl (5.2 kB)
Collecting importlib_resources==5.8.0
  Using cached importlib_resources-5.8.0-py3-none-any.whl (28 kB)
Collecting omegaconf==2.1.*
  Using cached omegaconf-2.1.2-py3-none-any.whl (74 kB)
Collecting antlr4-python3-runtime==4.8
  Using cached antlr4-python3-runtime-4.8.tar.gz (112 kB)
  Preparing metadata (setup.py) ... done
Collecting joblib>=0.14.0
  Using cached joblib-1.1.0-py2.py3-none-any.whl (306 kB)
Collecting zipp>=3.1.0
  Using cached zipp-3.8.0-py3-none-any.whl (5.4 kB)
Collecting PyYAML>=5.1.0
  Using cached PyYAML-6.0.tar.gz (124 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Using legacy 'setup.py install' for antlr4-python3-runtime, since package 'wheel' is not installed.
Building wheels for collected packages: PyYAML
  Building wheel for PyYAML (pyproject.toml) ... done
  Created wheel for PyYAML: filename=PyYAML-6.0-cp38-cp38-macosx_12_0_arm64.whl size=45335 sha256=e703e181e9d6429ca0b78de2c2229d95d127275b99c98a5711870bde22f1ac0c
  Stored in directory: /Users/jaraco/Library/Caches/pip/wheels/52/84/66/50912fd7bf1639a31758e40bd4312602e104a8eca1e0da9645
Successfully built PyYAML
Installing collected packages: antlr4-python3-runtime, zipp, PyYAML, joblib, omegaconf, importlib_resources, hydra-core, hydra-joblib-launcher, test
  Running setup.py install for antlr4-python3-runtime ... done
  Running setup.py develop for test
Successfully installed PyYAML-6.0 antlr4-python3-runtime-4.8 hydra-core-1.1.0 hydra-joblib-launcher-1.1.5 importlib_resources-5.8.0 joblib-1.1.0 omegaconf-2.1.2 test-0.0.0 zipp-3.8.0
WARNING: You are using pip version 22.0.4; however, version 22.1.2 is available.
You should consider upgrading via the '/Users/jaraco/draft/repro/.venv/bin/python -m pip install --upgrade pip' command.
 repro issue-253 $ py -m test.test --info
Hydra 1.1.0
===========
Installed Hydra Plugins
***********************
        ConfigSource:
        -------------
                FileConfigSource
                ImportlibResourcesConfigSource
                StructuredConfigSource
        CompletionPlugin:
        -----------------
                BashCompletion
                FishCompletion
                ZshCompletion
        Launcher:
        ---------
                BasicLauncher
                JoblibLauncher
        Sweeper:
        --------
                BasicSweeper
        SearchPathPlugin:
        -----------------
                Dummy

Config search path
******************
Traceback (most recent call last):
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/importlib_resources/abc.py", line 119, in joinpath
    match = next(matches)
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/homebrew/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/jaraco/draft/repro/test/test.py", line 8, in <module>
    test()
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/main.py", line 49, in decorated_main
    _run_hydra(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/utils.py", line 399, in _run_hydra
    hydra.show_info(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 655, in show_info
    options[info](config_name=config_name, overrides=overrides)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 588, in _print_all_info
    self._print_config_info(config_name, overrides)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 470, in _print_config_info
    self._print_search_path(config_name=config_name, overrides=overrides)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 403, in _print_search_path
    cfg = self._get_cfg(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 163, in _get_cfg
    cfg = self.compose_config(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 564, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 146, in load_configuration
    return self._load_configuration_impl(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 239, in _load_configuration_impl
    defaults_list = create_defaults_list(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 719, in create_defaults_list
    defaults, tree = _create_defaults_list(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 689, in _create_defaults_list
    defaults_tree = _create_defaults_tree(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 337, in _create_defaults_tree
    ret = _create_defaults_tree_impl(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 420, in _create_defaults_tree_impl
    return _expand_virtual_root(repo, root, overrides, skip_missing)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 262, in _expand_virtual_root
    subtree = _create_defaults_tree_impl(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 429, in _create_defaults_tree_impl
    update_package_header(repo=repo, node=parent)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 244, in update_package_header
    loaded = repo.load_config(config_path=node.get_config_path())
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/config_repository.py", line 337, in load_config
    ret = self.delegate.load_config(config_path=config_path)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/config_repository.py", line 86, in load_config
    source = self._find_object_source(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/config_repository.py", line 130, in _find_object_source
    if source.is_config(config_path):
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/core_plugins/importlib_resources_config_source.py", line 91, in is_config
    res = files.joinpath(config_path)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/importlib_resources/readers.py", line 87, in joinpath
    return super().joinpath(*descendants)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/importlib_resources/abc.py", line 121, in joinpath
    raise TraversalError(
importlib_resources.abc.TraversalError: ('Target not found during traversal.', 'hydra', ['config.yaml'])

[jaraco]

A bit of debugging:

 repro issue-253 $ py -m pdb -m test.test --info
> /Users/jaraco/draft/repro/test/test.py(1)<module>()
-> import hydra
(Pdb) c
Hydra 1.1.0
===========
Installed Hydra Plugins
***********************
        ConfigSource:
        -------------
                FileConfigSource
                ImportlibResourcesConfigSource
                StructuredConfigSource
        CompletionPlugin:
        -----------------
                BashCompletion
                FishCompletion
                ZshCompletion
        Launcher:
        ---------
                BasicLauncher
                JoblibLauncher
        Sweeper:
        --------
                BasicSweeper
        SearchPathPlugin:
        -----------------
                Dummy

Config search path
******************
Traceback (most recent call last):
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/importlib_resources/abc.py", line 119, in joinpath
    match = next(matches)
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/pdb.py", line 1703, in main
    pdb._runmodule(mainpyfile)
  File "/opt/homebrew/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/pdb.py", line 1547, in _runmodule
    self.run(code)
  File "/opt/homebrew/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/bdb.py", line 580, in run
    exec(cmd, globals, locals)
  File "/Users/jaraco/draft/repro/test/test.py", line 1, in <module>
    import hydra
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/main.py", line 49, in decorated_main
    _run_hydra(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/utils.py", line 399, in _run_hydra
    hydra.show_info(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 655, in show_info
    options[info](config_name=config_name, overrides=overrides)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 588, in _print_all_info
    self._print_config_info(config_name, overrides)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 470, in _print_config_info
    self._print_search_path(config_name=config_name, overrides=overrides)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 403, in _print_search_path
    cfg = self._get_cfg(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 163, in _get_cfg
    cfg = self.compose_config(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/hydra.py", line 564, in compose_config
    cfg = self.config_loader.load_configuration(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 146, in load_configuration
    return self._load_configuration_impl(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/config_loader_impl.py", line 239, in _load_configuration_impl
    defaults_list = create_defaults_list(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 719, in create_defaults_list
    defaults, tree = _create_defaults_list(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 689, in _create_defaults_list
    defaults_tree = _create_defaults_tree(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 337, in _create_defaults_tree
    ret = _create_defaults_tree_impl(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 420, in _create_defaults_tree_impl
    return _expand_virtual_root(repo, root, overrides, skip_missing)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 262, in _expand_virtual_root
    subtree = _create_defaults_tree_impl(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 429, in _create_defaults_tree_impl
    update_package_header(repo=repo, node=parent)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/defaults_list.py", line 244, in update_package_header
    loaded = repo.load_config(config_path=node.get_config_path())
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/config_repository.py", line 337, in load_config
    ret = self.delegate.load_config(config_path=config_path)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/config_repository.py", line 86, in load_config
    source = self._find_object_source(
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/config_repository.py", line 130, in _find_object_source
    if source.is_config(config_path):
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/core_plugins/importlib_resources_config_source.py", line 91, in is_config
    res = files.joinpath(config_path)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/importlib_resources/readers.py", line 87, in joinpath
    return super().joinpath(*descendants)
  File "/Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/importlib_resources/abc.py", line 121, in joinpath
    raise TraversalError(
importlib_resources.abc.TraversalError: ('Target not found during traversal.', 'hydra', ['config.yaml'])
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/importlib_resources/abc.py(121)joinpath()
-> raise TraversalError(
(Pdb) up 2
> /Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/hydra/_internal/core_plugins/importlib_resources_config_source.py(91)is_config()
-> res = files.joinpath(config_path)
(Pdb) files
MultiplexedPath('/Users/jaraco/draft/repro/test/config')
(Pdb) config_path
'hydra/config.yaml'
(Pdb) d
> /Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/importlib_resources/readers.py(87)joinpath()
-> return super().joinpath(*descendants)
(Pdb) descendants
('hydra/config.yaml',)
(Pdb) d
> /Users/jaraco/draft/repro/.venv/lib/python3.8/site-packages/importlib_resources/abc.py(121)joinpath()
-> raise TraversalError(
(Pdb) l
116                 traversable for traversable in self.iterdir() if traversable.name == target
117             )
118             try:
119                 match = next(matches)
120             except StopIteration:
121  ->             raise TraversalError(
122                     "Target not found during traversal.", target, list(names)
123                 )
124             return match.joinpath(*names)
125  
126         def __truediv__(self, child: StrPath) -> "Traversable":
(Pdb) !list(self.iterdir())
[PosixPath('/Users/jaraco/draft/repro/test/config/run.yaml')]
(Pdb) target
'hydra'

It appears from the code that hydra is looking for hydra/config.yaml in /Users/jaraco/draft/repro/test/config and because hydra doesn't exist in test/config, the TraversalError is raised.

[jaraco]

Looking at the calling code, it does appear as if the code is designed to trap exceptions when importlib_resources.files() is called, but not when .joinpath() is called.

Perhaps the refactoring that happened in 5.8.0 changed the behavior around when an exception is raised.

[MattiasDC]

Hi @jaraco, but shouldn´t this code catch already?

[jaraco]

Here's what changed from 5.7.1 to 5.8.0.

[MattiasDC]

Hi @jaraco, but shouldn´t this code catch already?

Ah, but that code can reraise..

[MattiasDC]

Can you confirm that this re-raise, which is new, is as intended?
If that's the case, this issue can be closed and I'll log an issue on hydra instead.

[jaraco]

Aha. I see the issue. Previously, MultiplexedPath.joinpath(), when passed a compound child path (like "hydra/config.yaml"), would incorrectly assume that the child was not a compound path but instead was a simple filename, so would join it to the first path (i.e. test/config/hydra/config.yaml) not raising an Exception and since that doesn't .exist(), the hydra code could handle it fine.

With 5.8 (#250 in particular), however, joinpath is more assertive about resolving parents of any path. If a parent (in this case test/config/hydra doesn't exist, it cannot continue traversal, so raises an exception.

Can you confirm that this re-raise, which is new, is as intended? If that's the case, this issue can be closed and I'll log an issue on hydra instead.

The change was intended, but it wasn't necessarily intended that errors would be raised on joinpath where they weren't before. I need to consider whether it's possible and worthwhile to retain the prior, more lenient behavior.

To be sure Hydra could work around the issue by simply moving the joinpath() up into the try block, but I don't want to recommend that until I'm certain that importlib_resources shouldn't address this issue.