Computing neighbors from X_scANVI representation

Hi,
in this tutorial: Atlas-level integration of lung data — scvi-tools
a neighborhood graph is computed from the latent representation (here called X_scVI). When I try to reproduce the steps, I get an error:
—> 73 sc.pp.neighbors(self.adata, n_neighbors=10, use_rep=self.rep, method=‘gauss’) #num_pcs=50
74 # The width of the connectivity kernel is implicitly determined by the number of neighbors used to compute the single-cell graph in neighbors()
75 sc.tl.diffmap(self.adata)

File ~/.conda/envs/new_sc/lib/python3.11/site-packages/scanpy/neighbors/init.py:139, in neighbors(adata, n_neighbors, n_pcs, use_rep, knn, random_state, method, metric, metric_kwds, key_added, copy)
137 adata._init_as_actual(adata.copy())
138 neighbors = Neighbors(adata)
→ 139 neighbors.compute_neighbors(
140 n_neighbors=n_neighbors,
141 knn=knn,
142 n_pcs=n_pcs,
143 use_rep=use_rep,
144 method=method,
145 metric=metric,
146 metric_kwds=metric_kwds,
147 random_state=random_state,
148 )
150 if key_added is None:
151 key_added = ‘neighbors’

File ~/.conda/envs/new_sc/lib/python3.11/site-packages/scanpy/neighbors/init.py:820, in Neighbors.compute_neighbors(self, n_neighbors, knn, n_pcs, use_rep, method, random_state, write_knn_indices, metric, metric_kwds)
817 # overwrite the umap connectivities if method is ‘gauss’
818 # self._distances is unaffected by this
819 if method == ‘gauss’:
→ 820 self._compute_connectivities_diffmap()
821 logg.debug(‘computed connectivities’, time=start_connect)
822 self._number_connected_components = 1

File ~/.conda/envs/new_sc/lib/python3.11/site-packages/scanpy/neighbors/init.py:833, in Neighbors._compute_connectivities_diffmap(self, density_normalize)
831 if self.knn:
832 Dsq = self._distances.power(2)
→ 833 indices, distances_sq = _get_indices_distances_from_sparse_matrix(
834 Dsq, self.n_neighbors
835 )
836 else:
837 Dsq = np.power(self._distances, 2)

File ~/.conda/envs/new_sc/lib/python3.11/site-packages/scanpy/neighbors/init.py:452, in _get_indices_distances_from_sparse_matrix(D, n_neighbors)
448 distances[i, 1:] = D[i][
449 neighbors[0][sorted_indices], neighbors[1][sorted_indices]
450 ]
451 else:
→ 452 indices[i, 1:] = neighbors[1]
453 distances[i, 1:] = D[i][neighbors]
454 return indices, distances

ValueError: could not broadcast input array from shape (0,) into shape (9,)

The same happend when I use the laten representation cmputed by scANVI (which I actually want to use, but since it is not shown in the tutorail, I used the X_scVI representation to make sure this is not the source of the problem)

Did someone encounter the problem before and could help me resolve the issue?
Thank you lots in advance.
Best
Anna

Did you ever get any clarity regarding this question? I’m trying to do something similar.

Hi, at some point it just worked, I’m so sorry, I do not remember what I changed… I remember that I tried to print the shapes, but I did not change anything knowingly…

I have never seen this error. Can you tell me the scvi-tools and scanpy version? Is this also happening if not setting method to ‘gauss’. What’s the shape of self.adata.obsm[self.rep]?

Hi, sorry but I do not have access to the workstation which I was running the code on anymore, since this was from a uni project from last semester. The envs do not exist anymore and since the issue resolved, I did not go deeper into this. Sorry!

It was more to get some troubleshooting from @jolespin. I assume some cells are created with only a single neighbor (not an expert for “gauss” nearest neighbor method). In scVI, it’s likely a problem with cells with almost no counts and usual qc (filtering to cells with more than 100 counts or much higher should solve it. Otherwise, my recommendation is using another NN method or if you require “gauss” as a method increase the number of nearest neighbors.