[Misc] Move find_loaded_library to platform_aware_utils.py

[houseroad]

find_loaded_library is a platform specific implementation, so create "platform_aware_utils.py". For different platform (such as using PAR to pack the binary), we can create our own version of "platform_aware_utils.py" to override such platform aware utility functions.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.
Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can do one of these:

• Add ready label to the PR
• Enable auto-merge.

🚀

[youkaichao]

The change of separating the function makes sense to me.

in vllm, platform mainly indicates accelerators, e.g. cuda, rocm, xpu, etc.

can you provide more intuitive names to them?

maybe you can provide your implementation of find_loaded_library , and we can see how to create proper abstraction for it.

[houseroad]

How about deployment_aware_utils.py instead? Our internal changes can be found in https://github.com/vllm-project/vllm/pull/12129/files. So far two things to change: 1) libcudart path, 2) PYTHONPATH due to PAR packing method.

[youkaichao]

In this case, I think we can have VLLM_CUDART_SO_PATH env var, similar to how we handle the VLLM_NCCL_SO_PATH. see

vllm/vllm/envs.py

Line 191 in 9691255

"VLLM_NCCL_SO_PATH":

[houseroad]

I see, I can have similar way to handle VLLM_CUDART_SO_PATH. How about the solution for subprocess.run? I was about to have a wrapper of subprocess_run to pass in the PYTHONPATH internally. Any suggestion on that?

[youkaichao]

How about the solution for subprocess.run?

commented at #12129 (comment)

[simon-mo]

Hi @houseroad @youkaichao, checking in on this PR, anything blocking landing it?

[houseroad]

Thanks for checking. @youkaichao feels a bit hacky of using file replacement, but it's quite common on the PyTorch internal usage. I am adding another hook in the _version.py internally. Will create some PR to see if it's enough. Let's keep this PR open until the internal hook needs meet our requirement. :-)

[houseroad]

Close this one, since the new proposal got landed.