1

Trying to use openTSNE because of the feature it is able to transform embeddings into an existing embeddings space.

I am trying to save the fit/trained embeddings object, so I can use it later but always getting error on pickling.

Here is an example on what I am trying to achieve, I always get PermissionError: WinError 32 The process cannot access the file because it is being used by another process: ...\AppData\Local\Temp\tmp703si9k_\tmp.ann'

    X  = np.load("X.npy")
    X_test = np.load("x_test.npy")
    
    
    affinities = openTSNE.affinity.PerplexityBasedNN(
        X,
        perplexity=500,
        n_jobs=32,
        random_state=0,
    )


    init = openTSNE.initialization.pca(X, random_state=42)


    tsne = openTSNE.TSNE(
        exaggeration=None,
        n_jobs=16,
        verbose=True,
    )
    embeddings = tsne.fit(affinities=affinities, initialization=init)
    
    
    pickle.dump(embeddings,open("embeddings.sav","wb"))


    global_embeddings = pickle.load(open("embeddings.sav","rb"))
    test_embeddings = global_embeddings.transform(x_test)

Or am I doing something wrong? I was also trying something like saving my embeddings as a numpy array and re initialize an openTSNE.TSNEEmbedding class object with the numpy array as embeddigns, but then I will need an affinity object as well what I also can't pickle. What is the problem? or what could be the best solution to solve this?

Tamás Majszlinger
  • 39
  • 6

1 Answers1

0

I ran into the same issue. Running the script on Linux didnt cause any problem. On windows, the only way I found to solve it was to downgrade openTSNE with

pip install openTSNE==0.6.0

There is also an issue open on github at https://github.com/pavlin-policar/openTSNE/issues/210. If we are several to report the issue, the author might fix it.

Aurélien Pélissier
  • 1
  • 2