scVI - Help getting GPUs to work! Thanks!

Good morning!

I have very simple configuration problem that I am in the process of googling and troubleshooting. I have installed scVI-tools via conda as shown here Installation — scvi-tools

conda create -n scvi-env python=3.7
source activate scvi-env
conda install scvi-tools -c bioconda -c conda-forge

and scVI works fine. BUT it says there is no GPU available. I am currently going through separate PyTorch documents to see if I need some additional config. The issue is almost certainly my config and not related to scVI directly. However, since everyone here has experience setting this up, I figured it wouldnt hurt to ask in this forum about any troubleshooting steps you might know that I as a new scVI user might be missing.

Thanks in advance for any tips or tricks! I will update this thread as I try different things I find via google.

okay so after reinstalling PyTorch in the scivi-env , i still get

GPU available: False, used: False
TPU available: False, using: 0 TPU cores

I installed PyTorch in the environment with

conda install pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch

I am on Ubuntu 20.04 with a GeForce RTX 2080 Ti

Thank you again in advance for tips and tricks…

Hi, getting a CUDA environment working can be frustrating…

Just a quick check, does the nvidia-smi command in the shell work and return some stats about your GPU?

Thank you Valentine for joining me on my GPU PyTorch adventure!

yes nvidia-smi does indeed work and return some information. I am going to copy and paste the return results as they don’t all mean anything to me per-say

| NVIDIA-SMI 460.80       Driver Version: 460.80       CUDA Version: 11.2     |
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|   0  GeForce RTX 208...  Off  | 00000000:23:00.0  On |                  N/A |
| 33%   53C    P0    66W / 250W |   1168MiB / 11016MiB |      2%      Default |
|                               |                      |                  N/A |
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|    0   N/A  N/A      2754      G   /usr/lib/xorg/Xorg                 90MiB |
|    0   N/A  N/A      3700      G   /usr/lib/xorg/Xorg                633MiB |
|    0   N/A  N/A      3839      G   /usr/bin/gnome-shell              151MiB |
|    0   N/A  N/A      6039      G   ...AAAAAAAA== --shared-files       30MiB |
|    0   N/A  N/A      6704      G   /usr/lib/firefox/firefox          150MiB |
|    0   N/A  N/A      6857      G   /usr/lib/firefox/firefox            3MiB |
|    0   N/A  N/A      6867      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      6888      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      6951      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      6974      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A      7004      G   /usr/lib/firefox/firefox            2MiB |
|    0   N/A  N/A     13544      G   /usr/lib/rstudio/bin/rstudio       51MiB |
|    0   N/A  N/A     14300      G   gnome-control-center                3MiB |

further further update:

I ran the PyTorch commands to ping the GPU availability, and by raw in python it works

(base) UserNameHidden :~/Desktop$ python
Python 3.8.5 (default, Sep  4 2020, 07:30:14) 
[GCC 7.3.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
>>> torch.cuda.device_count()
>>> torch.cuda.get_device_name(0)
'GeForce RTX 2080 Ti'
>>> torch.cuda.get_device_properties(0)
_CudaDeviceProperties(name='GeForce RTX 2080 Ti', major=7, minor=5, total_memory=11016MB, multi_processor_count=68)

EDIT: I just redid it while in scvi-env and not base , just to make sure installation had access… same results… but still when I run model$train() its saying no GPU

I have solved the problem , my answer was here → scvi.model.SCVI.train — scvi-tools

by default in the function use_gpu is set to false , you have to pass TRUE to enable… i feel like an idiot :grimacing:

Well the default is None which uses the gpu if it’s available, so I think something else might have happened?

@LabCoatNomad glad you solved your install problem.

I saw you are using CUDA 11.2, so just wanted to leave a note here in case it helps other people. I actually also had trouble getting the GPU to work torch tensors it, but in my case I found it to be a pytorch issue: Apparently pytorch doesn’t yet support CUDA 11.2 but after trying a bunch of things it magically worked when I tried installing pytorch from the nightly builds.

1 Like

on my machine can confirm model$train() returns GPU available: False, used: False , but when I do model$train(use_gpu = TRUE) i get GPU available: True, used: True , it could be something weird with my config or setup, but all my steps of what I have done are above and this is how its working… * shrug * dont know why , but happy it works =)

Weird… if it works it works…