Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

allow using CUDA libraries installed as Python modules #8013

Open
FlorinAndrei opened this issue Dec 2, 2023 · 2 comments
Open

allow using CUDA libraries installed as Python modules #8013

FlorinAndrei opened this issue Dec 2, 2023 · 2 comments
Assignees
@kmaehashi @leofang
Labels
cat:feature New features/APIs prio:high

Comments

@FlorinAndrei
Copy link

FlorinAndrei commented Dec 2, 2023 •
edited

Description

JAX, PyTorch and TensorFlow all can use CUDA libraries installed as Python modules. For example, if you install JAX with the pip version of CUDA...

pip install --upgrade --user "jax[cuda11_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html

...that will also install the relevant CUDA libraries as Python modules:

nvidia-cublas-cu11                11.11.3.6
nvidia-cuda-cupti-cu11            11.8.87
nvidia-cuda-nvcc-cu11             11.8.89
nvidia-cuda-nvrtc-cu11            11.8.89
nvidia-cuda-runtime-cu11          11.8.89
nvidia-cudnn-cu11                 8.9.6.50
nvidia-cufft-cu11                 10.9.0.58
nvidia-cusolver-cu11              11.4.1.48
nvidia-cusparse-cu11              11.7.5.86
nvidia-nccl-cu11                  2.19.3

Then you can install TensorFlow as well...

pip install --upgrade --user tensorflow=2.14.1

...and then PyTorch...

pip install --upgrade --user torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

...and they all work perfectly without needing a separate CUDA install.

However, if you install cupy-cuda11x and try to run it, it claims it cannot find CUDA. It only works if you give it a completely separate installation of CUDA, in /usr/local or something.

Having CUDA installed via pip is easier and more convenient, and may save some storage.

Additional Information

I'm using Ubuntu 22.04 with an RTX 3090.
@FlorinAndrei FlorinAndrei added the cat:feature New features/APIs label Dec 2, 2023
@kmaehashi
Copy link
Member

Hi @FlorinAndrei, thanks for feedback! This is definitely what we want. I'll start investigating changes required to support these PyPI distributed CUDA Toolkit.

This was referenced Feb 5, 2024
Closed
Closed
@HelloCard
Copy link

HelloCard commented Feb 21, 2024 •
edited

Still not working, still relying on NVIDIA CUDA suite instead of Python components.

Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.11/site-packages/cupy/__init__.py", line 17, in <module>
    from cupy import _core  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/lib/python3.11/site-packages/cupy/_core/__init__.py", line 3, in <module>
    from cupy._core import core  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cupy/_core/core.pyx", line 1, in init cupy._core.core
  File "/root/miniconda3/lib/python3.11/site-packages/cupy/cuda/__init__.py", line 8, in <module>
    from cupy.cuda import compiler  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/miniconda3/lib/python3.11/site-packages/cupy/cuda/compiler.py", line 13, in <module>
    from cupy.cuda import device
  File "cupy/cuda/device.pyx", line 9, in init cupy.cuda.device
ImportError: libcusolver.so.11: cannot open shared object file: No such file or directory

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.11/site-packages/vllm/model_executor/parallel_utils/cupy_utils.py", line 16, in <module>
    import cupy
  File "/root/miniconda3/lib/python3.11/site-packages/cupy/__init__.py", line 19, in <module>
    raise ImportError(f'''
ImportError:
================================================================
Failed to import CuPy.

I have already installed it in my Python environment: nvidia-cusolver-cu12 11.4.5.107
\wsl.localhost\Ubuntu-22.04\root\miniconda3\lib\python3.11\site-packages\nvidia\cusolver\lib\libcusolver.so.11
It exists here, but cupy cannot find it.
If I install Nvidia's cuda suite, cupy will work properly, but I will take up an additional 10GB of storage space for these duplicate files.

Adding an LD_LIBRARY_PATH at the end of bashrc in the user directory can temporarily solve this problem(root\.bashrc):
export LD_LIBRARY_PATH=/root/miniconda3/lib/python3.11/site-packages/nvidia/cusolver/lib:$LD_LIBRARY_PATH

@leofang leofang self-assigned this Mar 7, 2024
@leofang leofang mentioned this issue Mar 7, 2024
Closed
@asi1024 asi1024 mentioned this issue Apr 3, 2024
Open
@kmaehashi kmaehashi mentioned this issue Jun 8, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kmaehashi kmaehashi

@leofang leofang

Labels
cat:feature New features/APIs prio:high
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@FlorinAndrei @kmaehashi @leofang @HelloCard