Consistent results when using scvi.model.SCVI

Help scvi-tools
,
1

Hi!
First of all thank you for developing this very useful tool!
I’m currently comparing a model I’ve developed with the original scVI-model but I’m not getting the same results every time I run the scVI-model.

My code for testing the robustness looks something like this:

for _ in range(K):
    _setup_anndata(train_i, labels_key="labels")
    _setup_anndata(test_i, labels_key="labels")
    model_ = model.SCVI(adata=train_i,  n_hidden=128, n_layers=1)
    model_.train(lr=0.0004)
    latent  = model_.get_latent_representation()
    labels_pred = KMeans(9, n_init=200, random_state=settings.seed).fit_predict(latent)

My expectation is to get the same results for each iteration of this for-loop.
What I have checked:

  • Random seed is enabled and the same for each run
  • Data is copied with deepcopy() and not modified during training/testing

Is it expected behavior that the model should produce the exact same results, for the same data/initialization/computer_architecture?

Best regards,
Sarah

2

On setting settings.seed, it re-seeds torch and numpy globally, not per run of SCVI training. Perhaps try this experiment again where you reassign settings.seed to the same value at the beginning of each loop. If it still gives different results, we would definitely want to investigate that.

3

Ah I didn’t think about that! I just tried it and it solved the issue. Thank you so much for your quick reply!
Best regards,
Sarah

4

That’s great (and a relief). Thanks for using scvi-tools!