chore: add pixi for cuda-core and cuda-bindings #1226
Conversation
|
Auto-sync is disabled for ready for review pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
cpcloud
force-pushed
the
pixi-for-cuda-core
branch
from
fb9b67c to
e3f2aa2
Compare
|
/ok to test |
This comment has been minimized.
This comment has been minimized.
|
/ok to test |
1 similar comment
|
/ok to test |
|
/ok to test |
cpcloud
force-pushed
the
pixi-for-cuda-core
branch
from
985a8ab to
3fc0891
Compare
|
/ok to test |
|
Now that we have pixi support, how about adding CI pipelines? (#280) 😉 |
Something I will look into for sure. I would like to get docs building first (or maybe as a parallel work stream). |
leofang
force-pushed
the
pixi-for-cuda-core
branch
from
eda7869 to
cd60970
Compare
| cuda-version = "13.*" | ||
|
|
||
| [environments] | ||
| cu13 = { features = ["cu13", "test", "cython-tests"], solve-group = "cu13" } |
There was a problem hiding this comment.
I am a bit unhappy having to pass -e cu13 explicitly for every pixi operation. In the next PR we should think about if this is avoidable.
| depends-on = [{ task = "build-cython-tests" }] | ||
|
|
||
| [target.linux.tasks.test] | ||
| cmd = ["pytest", "$PIXI_PROJECT_ROOT/tests"] |
There was a problem hiding this comment.
note: I am not sure why, but without adding the tests subdir to the path I hit an error:
(...)
tests/cython/test_cython.py . [100%]
=========================================================================== 1 passed in 0.75s ============================================================================
✨ Pixi task (test in cu13): pytest $PIXI_PROJECT_ROOT import cupy as cp
from cuda.core.experimental import Device
from cuda.core.experimental import LaunchConfig, launch
from cuda.core.experimental import Program, ProgramOptions
dev = Device(0)
dev.set_current()
code = r"""
extern "C" __global__ void arange(int* a, const int N) {
const unsigned int tid = threadIdx.x + blockIdx.x * blockDim.x;
for (auto x=tid; x<N; x+=blockDim.x*gridDim.x) { a[x] = x; }
}
"""
prog = Program(code,
code_type="c++",
options=ProgramOptions(std="c++17"))
mod = prog.compile("cubin")
ker = mod.get_kernel("arange")
s = dev.create_stream()
length = 64
data_type = cp.int32
dtype = cp.dtype(data_type)
buf = dev.memory_resource.allocate(length * dtype.itemsize, s)
config = LaunchConfig(grid=1, block=(32,), stream=s)
launch(ker, config, buf, data_type(length))
s.sync()
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/local/home/leof/dev/cuda-python/cuda_core/.pixi/envs/cu13/lib/python3.14/site-packages/_pytest/main.py", line 314, in wrap_session
INTERNALERROR> config._do_configure()
INTERNALERROR> ~~~~~~~~~~~~~~~~~~~~^^
INTERNALERROR> File "/local/home/leof/dev/cuda-python/cuda_core/.pixi/envs/cu13/lib/python3.14/site-packages/_pytest/config/__init__.py", line 1128, in _do_configure
INTERNALERROR> self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
INTERNALERROR> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File "/local/home/leof/dev/cuda-python/cuda_core/.pixi/envs/cu13/lib/python3.14/site-packages/pluggy/_hooks.py", line 534, in call_historic
INTERNALERROR> res = self._hookexec(self.name, self._hookimpls.copy(), kwargs, False)
INTERNALERROR> File "/local/home/leof/dev/cuda-python/cuda_core/.pixi/envs/cu13/lib/python3.14/site-packages/pluggy/_manager.py", line 120, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR> ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> File "/local/home/leof/dev/cuda-python/cuda_core/.pixi/envs/cu13/lib/python3.14/site-packages/pluggy/_callers.py", line 167, in _multicall
INTERNALERROR> raise exception
INTERNALERROR> File "/local/home/leof/dev/cuda-python/cuda_core/.pixi/envs/cu13/lib/python3.14/site-packages/pluggy/_callers.py", line 121, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/local/home/leof/dev/cuda-python/cuda_core/.pixi/envs/cu13/lib/python3.14/site-packages/_pytest/debugging.py", line 66, in pytest_configure
INTERNALERROR> import pdb
INTERNALERROR> File "/local/home/leof/dev/cuda-python/cuda_core/.pixi/envs/cu13/lib/python3.14/pdb.py", line 299, in <module>
INTERNALERROR> class _PdbInteractiveConsole(code.InteractiveConsole):
INTERNALERROR> ^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR> AttributeError: module 'code' has no attribute 'InteractiveConsole'
|
|
||
| [target.linux.tasks.test] | ||
| cmd = ["pytest", "$PIXI_PROJECT_ROOT/tests"] | ||
| #env = { PYTHONPATH = "$PIXI_PROJECT_ROOT/../cuda_python_test_helpers" } |
There was a problem hiding this comment.
This does not seem necessary...? (I commented it out.)
There was a problem hiding this comment.
Running tests from $PIXI_PROJECT_ROOT doesn't work for me without this. That's where we should run the tests from.
This was an intentional choice that I made so that pixi run -e cu13 pytest "just works".
|
weird, GitHub response is so slow, and the previous CI trigger did nothing |
|
/ok to test b1834b1 |
|
| [target.linux.tasks.build-cython-tests] | ||
| cmd = ["$PIXI_PROJECT_ROOT/tests/cython/build_tests.sh"] | ||
|
|
||
| [target.linux.tasks.run-cython-tests] |
There was a problem hiding this comment.
Is there a point to the complexity of creating a separate task just to run the cython tests and then making that a dependency of the test task?
Why not just run pytest $PIXI_PROJECT_ROOT? If that doesn't work for some reason (e.g., it can't discover the cython tests) then we should pass it on the command line.
There was a problem hiding this comment.
Right, Cython tests aren't auto-discoverable from the project root. We added a rule to exclude them. Feel free to simplify the task dependency. I only wanted to make sure it runs as intended, and thought adding 3 lines of code was not too bad.
3 participants
This PR is a redo of pixi environment setup that combines
cuda-coreandcuda-bindingsinto a single PR.I am not trying to capture every possible environment combination of CUDA and
Python right now, just enough to get a minimal development environment working
for both of these projects on linux-64. What's in these
pixi.tomlfiles isthe scope I intended.
The biggest thing that isn't working yet, but that isn't a blocker is the
ability of
cuda-coreto depend oncuda-bindingsin the repo. During thecuda-corebuild phase,cuda/bindings.pyxisn't found and that causes thebuild to fail. I haven't yet investigated that.