I get the weirdest error when I use joblib (1.2.0) wit scikit-learn. Here is a MWE:
import numpy as np
from joblib import Parallel, delayed, parallel_backend
from sklearn.neighbors import NearestNeighbors
def it():
df = np.random.randn(1000).reshape(-1,1)
NearestNeighbors(n_neighbors=2, p=2).fit(df).kneighbors(df)
yield 1
f = lambda x: 1
with parallel_backend("loky", inner_max_num_threads=1):
res = Parallel(n_jobs=2)(delayed(f)(p) for p in it())
Running this code, I get AttributeError: 'NoneType' object has no attribute 'submit'
. I cannot get around it. If I comment out the call to NearestNeighbors
the problem disappears. If I let n_jobs=1
the problem disappears. If I skip the context manager - the problem disappears.