# Error when training model on M3 Max MPS

**URL:** <https://discourse.scverse.org/t/error-when-training-model-on-m3-max-mps/1896>\
**Category:** scvi-tools\
**Created:** [November 17, 2023, 7:38pm UTC](https://discourse.scverse.org/t/error-when-training-model-on-m3-max-mps/1896 "2023-11-17T19:38:02Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sshinBRC](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.scverse.org/sshinbrc/32/861_2.png) [@sshinBRC](https://discourse.scverse.org/u/sshinBRC)\
**Post date:** [November 17, 2023, 7:38pm UTC](https://discourse.scverse.org/t/error-when-training-model-on-m3-max-mps/1896/1 "2023-11-17T19:38:03Z")

</div>

Hi there,

I ran into an error when trying to train a model on M3 Max MBP. Python version is 3.10 with nightly Pytorch build:

ValueError: Expected parameter loc (Tensor of shape (128, 10)) of distribution Normal(loc: torch.Size([128, 10]), scale: torch.Size([128, 10])) to satisfy the constraint Real(), but found invalid values:  
tensor([[nan, nan, nan, …, nan, nan, nan],  
[nan, nan, nan, …, nan, nan, nan],  
[nan, nan, nan, …, nan, nan, nan],  
…,  
[nan, nan, nan, …, nan, nan, nan],  
[nan, nan, nan, …, nan, nan, nan],  
[nan, nan, nan, …, nan, nan, nan]], device=‘mps:0’,  
grad\_fn=)

I would like to know if this issue is specifically because I am using a beta version of Pytorch or because I am doing something wrong with the script. I am new to Python sc-seq analysis, so I apologize in advance if this question is trivial.

Thank you for your time!

---

<div class="post-metadata">

**Author:** ![martinkim0](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.scverse.org/martinkim0/32/881_2.png) [@martinkim0](https://discourse.scverse.org/u/martinkim0)\
**Post date:** [November 20, 2023, 8:50pm UTC](https://discourse.scverse.org/t/error-when-training-model-on-m3-max-mps/1896/2 "2023-11-20T20:50:11Z")

</div>

Hi, we’ve had some issues with NaNs in MultiVI and gimVI, are you using either of those models by chance? If so, I’d try downgrading to `scvi-tools==0.20.3` and checking if the issue still persists.

I’ll also note that our models are not fully compatible with PyTorch MPS builds as the [`lgamma`](https://github.com/pytorch/pytorch/issues/77764) kernel is not supported with that backend yet, but based on the traceback, that doesn’t seem to be the source of the error here.

---

<div class="post-metadata">

**Author:** ![sshinBRC](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.scverse.org/sshinbrc/32/861_2.png) [@sshinBRC](https://discourse.scverse.org/u/sshinBRC)\
**Post date:** [November 22, 2023, 4:29pm UTC](https://discourse.scverse.org/t/error-when-training-model-on-m3-max-mps/1896/3 "2023-11-22T16:29:57Z")

</div>

Thank you for the response! I am only using scvi.model.SCVI and I have found that I do not get any errors when I use the CPU instead. Also, scvi-tools==0.20.3 gives the same error as before. I suppose this is a compatibility issue ☹

Thanks again for your time!

---

<div class="post-metadata">

**Author:** ![martinkim0](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.scverse.org/martinkim0/32/881_2.png) [@martinkim0](https://discourse.scverse.org/u/martinkim0)\
**Post date:** [November 22, 2023, 9:59pm UTC](https://discourse.scverse.org/t/error-when-training-model-on-m3-max-mps/1896/4 "2023-11-22T21:59:41Z")

</div>

Sorry about that! We’re also looking forward for PyTorch to add full compatibility with MPS, so we’ll be testing on our end to see when that is available.

---

<div class="post-metadata">

**Author:** ![TaniaThimraj](https://avatars.discourse-cdn.com/v4/letter/t/c2a13f/32.png) [@TaniaThimraj](https://discourse.scverse.org/u/TaniaThimraj)\
**Post date:** [February 12, 2024, 9:18am UTC](https://discourse.scverse.org/t/error-when-training-model-on-m3-max-mps/1896/5 "2024-02-12T09:18:36Z")

</div>

I have the same issue.

---

<div class="post-metadata">

**Author:** ![martinkim0](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.scverse.org/martinkim0/32/881_2.png) [@martinkim0](https://discourse.scverse.org/u/martinkim0)\
**Post date:** [February 16, 2024, 5:45pm UTC](https://discourse.scverse.org/t/error-when-training-model-on-m3-max-mps/1896/6 "2024-02-16T17:45:46Z")

</div>

Please refer to the following thread: [Macbook M1 M2 mps acceleration with scVI - #4 by martinkim0](https://discourse.scverse.org/t/macbook-m1-m2-mps-acceleration-with-scvi/2075/4)