Interpreting model history curves

Help scvi-tools
1

Hello all !

I trained a PeakVI model for 50 epochs. What can you interpret using these curves from model history ?
Also, can please someone explain me:

  • What is the difference between local and global KL ?
  • Should we look for a minimal KL local value ? (i.e, would you train for more epochs to reduce KL local ?)
  • Why are there some peaks in train loss step ?
  • How can be Loss so high (> 1e7) ? Is it important ?

perf
perf1390×989 93.1 KB

Thank you for your help !

Phil

2

I would check the ELBO curves and reconstruction_loss curves. Train loss step is a noisy estimate (it might be that some minibatches have lower loss and I would avoid interpreting it). Elbo is preferable of Loss as we use kl_warmup during model training (the weight of KL local in the total loss increases over time up to a maximum value of 1 - this is tracked in the kl_weight plot and is the reason why the kl_local loss increases and then decreases again). KL global tracks KL losses of parameters that are not computed per cell (e.g. technical variation in DestVI mdel). It depends on your downstream task whether you prefer low reconstruction loss (a good generative model) or low KL local (likely better integration). We use those curves mainly to see that training converged (Elbo reaches a plateau) and we don’t overfit (strong increase in validation losses with descreasing training losses). I hope this gives some overview. There are more general introductions to this topic in most ML introductions.

1 Like