nbdev_test fails with concurrent.futures.process.BrokenProcessPool exception

[bkowshik]

Similar to: #673

---

Error

Was going through the tutorial using nbdev==2.0.4 on macOS.

$ nbdev_test
objc[29851]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[29851]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
Traceback (most recent call last):
  File "/usr/local/bin/nbdev_test", line 8, in <module>
    sys.exit(nbdev_test())
  File "/usr/local/lib/python3.9/site-packages/fastcore/script.py", line 116, in _f
    return tfunc(**merge(args, args_from_prog(func, xtra)))
  File "/usr/local/lib/python3.9/site-packages/nbdev/test.py", line 86, in nbdev_test
    results = parallel(test_nb, files, skip_flags=skip_flags, force_flags=force_flags, n_workers=n_workers, pause=pause, do_print=do_print)
  File "/usr/local/lib/python3.9/site-packages/fastcore/parallel.py", line 117, in parallel
    return L(r)
  File "/usr/local/lib/python3.9/site-packages/fastcore/foundation.py", line 98, in __call__
    return super().__call__(x, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/fastcore/foundation.py", line 106, in __init__
    items = listify(items, *rest, use_list=use_list, match=match)
  File "/usr/local/lib/python3.9/site-packages/fastcore/basics.py", line 63, in listify
    elif is_iter(o): res = list(o)
  File "/usr/local/Cellar/[email protected]/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/process.py", line 559, in _chain_from_iterable_of_lists
    for element in iterable:
  File "/usr/local/Cellar/[email protected]/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 608, in result_iterator
    yield fs.pop().result()
  File "/usr/local/Cellar/[email protected]/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 445, in result
    return self.__get_result()
  File "/usr/local/Cellar/[email protected]/3.9.4/Frameworks/Python.framework/Versions/3.9/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

[jph00]

Try 2.0.5, and also upgrade your fastcore to 1.5.11.

[bkowshik]

I was able to get the tests to run post the upgrades. Thank you @jph00

$ nbdev_test
Success.

---

The one change I had to make to make the tests pass was remove the following two lines which I use to get higher resolution matplotlib charts. The nbdev_test command failed with the following error when I had these two lines in the notebook.

%matplotlib inline
%config InlineBackend.figure_format = 'retina'

Error message of nbdev_test

While Executing Cell #6:
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-0eb3525f3a9a> in <module>
----> 1 get_ipython().run_line_magic('matplotlib', 'inline')
      2 get_ipython().run_line_magic('config', "InlineBackend.figure_format = 'retina'")
      3
      4 import pandas as pd

~/Library/Python/3.7/lib/python/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)
   2334                 kwargs['local_ns'] = self.get_local_scope(stack_depth)
   2335             with self.builtin_trap:
-> 2336                 result = fn(*args, **kwargs)
   2337             return result
   2338

<decorator-gen-101> in matplotlib(self, line)

~/Library/Python/3.7/lib/python/site-packages/IPython/core/magic.py in <lambda>(f, *a, **k)
    185     # but it's overkill for just that one bit of state.
    186     def magic_deco(arg):
--> 187         call = lambda f, *a, **k: f(*a, **k)
    188
    189         if callable(arg):

~/Library/Python/3.7/lib/python/site-packages/IPython/core/magics/pylab.py in matplotlib(self, line)
     97             print("Available matplotlib backends: %s" % backends_list)
     98         else:
---> 99             gui, backend = self.shell.enable_matplotlib(args.gui.lower() if isinstance(args.gui, str) else args.gui)
    100             self._show_matplotlib_backend(args.gui, backend)
    101

/usr/local/lib/python3.7/site-packages/execnb/shell.py in enable_matplotlib(self, gui)
     91     def enable_matplotlib(self, gui=None):
     92         "Enable `matplotlib` in a nested shell"
---> 93         from matplotlib_inline.backend_inline import configure_inline_support
     94         configure_inline_support.current_backend = 'unset'
     95         return super().enable_matplotlib(gui)

ModuleNotFoundError: No module named 'matplotlib_inline'

I looked at example notebooks on fastai to check if something else needs to be done. But, I did not see any.

• Ref: https://github.com/fastai/fastai/blob/master/dev_nbs/course/lesson1-pets.ipynb

[jph00]

OK thanks for letting us know - tracking that in #702

[seeM]

@bkowshik have you tried pip install matplotlib_inline (or the conda equivalent if you're using that)?

[larsyencken]

Following the tutorial. I also get this issue on nbdev==2.1.2 and fastcore==1.5.15 on MacOS (M1) and Python 3.9.7:

objc[12963]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
objc[12963]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
Traceback (most recent call last):
  File "/Users/lars/.pyenv/versions/3.9.7/bin/nbdev_test", line 8, in <module>
    sys.exit(nbdev_test())
  File "/Users/lars/.pyenv/versions/3.9.7/lib/python3.9/site-packages/fastcore/script.py", line 116, in _f
    return tfunc(**merge(args, args_from_prog(func, xtra)))
  File "/Users/lars/.pyenv/versions/3.9.7/lib/python3.9/site-packages/nbdev/test.py", line 88, in nbdev_test
    results = parallel(test_nb, files, skip_flags=skip_flags, force_flags=force_flags, n_workers=n_workers,
  File "/Users/lars/.pyenv/versions/3.9.7/lib/python3.9/site-packages/fastcore/parallel.py", line 117, in parallel
    return L(r)
  File "/Users/lars/.pyenv/versions/3.9.7/lib/python3.9/site-packages/fastcore/foundation.py", line 98, in __call__
    return super().__call__(x, *args, **kwargs)
  File "/Users/lars/.pyenv/versions/3.9.7/lib/python3.9/site-packages/fastcore/foundation.py", line 106, in __init__
    items = listify(items, *rest, use_list=use_list, match=match)
  File "/Users/lars/.pyenv/versions/3.9.7/lib/python3.9/site-packages/fastcore/basics.py", line 66, in listify
    elif is_iter(o): res = list(o)
  File "/Users/lars/.pyenv/versions/3.9.7/lib/python3.9/concurrent/futures/process.py", line 559, in _chain_from_iterable_of_lists
    for element in iterable:
  File "/Users/lars/.pyenv/versions/3.9.7/lib/python3.9/concurrent/futures/_base.py", line 608, in result_iterator
    yield fs.pop().result()
  File "/Users/lars/.pyenv/versions/3.9.7/lib/python3.9/concurrent/futures/_base.py", line 445, in result
    return self.__get_result()
  File "/Users/lars/.pyenv/versions/3.9.7/lib/python3.9/concurrent/futures/_base.py", line 390, in __get_result
    raise self._exception
concurrent.futures.process.BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

I tried downgrading nbdev to 2.0.5 and downgrading fastcore to 1.5.11 as per the suggestion, but it doesn't seem to solve the issue.

[seeM]

@larsyencken could you please try upgrading to the latest Python 3.9 version? There’s a known macOS parallelism bug in earlier versions. Also try using latest versions of execnb, fastcore, and nbdev.

[larsyencken]

Thanks! Upgrading from Python 3.9.7 to 3.9.13 fixed it, works fine now.