Multithreading error

Help scvi-tools
1

Hi, I tried to set:
scvi.settings.dl_num_workers = 19

and saw the following warning, accompanied by significant reduction in speed:

os.fork() was called. os.fork() is incompatible with multithreaded code, and JAX is multithreaded, so this will likely lead to a deadlock.

I didn’t see any bugs related to this on scvi github so I am guessing this isn’t a bug with the tool. Does anyone know how to fix it? Is this a pytorch issue?

My pyTorch version: 2.3.1+cu121

p.s. I ran into this while running the new version of MrVI, but I assume this is model agnostic.

Thanks!
Kunal

2

Hi, we don’t recommend settingscvi.settings.dl_num_workers when using JAX models such as MrVI due to the warning that it emits. Dataloading will be a little slower without multiple workers, but in my experience it’s not too bad.

1 Like