TypeError: 'NoneType' object is not callable #3057
Description
when I test for mutil-gpu with llama2-70b, run vllm/examples/offline_inference.py , use params enforce_eager=False, the result can output, but it occur some error
Prompt: 'Hello, my name is', Generated text: ' Dustin Nelson and I’m going to be posting articles and my thoughts'
Prompt: 'The president of the United States is', Generated text: ' one of the most powerful people in the world, as the leader of the only'
Prompt: 'The capital of France is', Generated text: ' one of the world’s leading cities in terms of art, fashion, food'
Prompt: 'The future of AI is', Generated text: ' neither utopian nor apocalyptic—it’s both.\n'
Exception ignored in: <function TCPStore.__del__ at 0x7f930d38e8c0>
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/cupyx/distributed/_store.py", line 59, in __del__
File "/usr/local/lib/python3.10/dist-packages/cupyx/distributed/_store.py", line 109, in stop
File "/usr/local/lib/python3.10/dist-packages/cupyx/distributed/_store.py", line 39, in join
File "/usr/lib/python3.10/multiprocessing/connection.py", line 257, in poll
File "/usr/lib/python3.10/multiprocessing/connection.py", line 424, in _poll
TypeError: 'NoneType' object is not callable
the error in the code https://github.com/vllm-project/vllm/blob/main/vllm/worker/model_runner.py#L750