Datapipe handling with more than 1 GPU gives torch.multiprocessing.spawn.ProcessExitedException: process 1 terminated with exit code 1

[franchesoni]

🐛 Bug

A dataloader around a datapipe works in cpu and gpu but not with 2 gpus

To Reproduce

Code to reproduce (you need 2 gpus)
https://gist.github.com/franchesoni/7c048c0f0eb56f6aaa2731b0ead78a51#file-issue-py

Output (logged by ussing python issue.py >> issue.out 2>&1
https://gist.github.com/franchesoni/7c048c0f0eb56f6aaa2731b0ead78a51#file-issue-out

Expected behavior

Not break

Environment

• CUDA:
  - GPU:
  - TITAN V
  - GeForce RTX 2080 Ti
  - available: True
  - version: 11.3
• Packages:
  - numpy: 1.22.3
  - pyTorch_debug: False
  - pyTorch_version: 1.11.0
  - pytorch-lightning: 1.6.3
  - tqdm: 4.64.0
• System:
  - OS: Linux
  - architecture:
  - 64bit
  - ELF
  - processor: x86_64
  - python: 3.9.12
  - version: updated support for 1.2.0 #80~18.04.1-Ubuntu SMP Mon Apr 12 23:26:25 UTC 2021

Additional context

cc @otaj @justusschock @awaelchli @ninginthecloud @rohitgr7 @kaushikb11 @akihironitta

[RylanSchaeffer]

I have this problem. Does anyone know the fix, or when a solution will be available?

[otaj]

Hi @franchesoni and @RylanSchaeffer, I just wanted to let you know, that it is being worked on and I have not abandoned this issue. Though I don't think I have an ETA for the fix sadly yet.

[RylanSchaeffer]

@otaj thanks for the update!