Stack Overflow
Stack Overflow

Questions tagged [joblib]

Joblib is a set of tools to provide lightweight pipelining in Python.

Joblib is a set of tools to provide lightweight pipelining in Python.

https://joblib.readthedocs.io/en/latest/

715 questions
0
votes
1 answer

Joblib doesn't see cython modules when n_jobs > 1

I can't get joblib to see a cython function that I am defining, and am unsure why. Here is the minimal code example in a Google Colab notebook. Minimal code example: from joblib import Parallel, delayed %load_ext Cython %%cython…
Tim
  • 1,360
  • 1
  • 12
  • 21
0
votes
2 answers

python 3.x joblib simple save function

I'm trying to create a simple joblib function, which will evaluate the expression and pickle the result, while checking for the existence of the pickle file. But when I put this function in some other file and import the function after adding the…
agarg
  • 318
  • 3
  • 11
0
votes
1 answer

How to save feature engineered model in ML

After building the model we save the model to do live predictions. But saving the model will be simple if there is no feature engineering, for instance say I have done some chisquare, Randomforest to get some features which are contributing on model…
raja mannar
  • 1
  • 3
0
votes
0 answers

How to parallelly process a pandas Dataframe for each unique value in a column?

I'm looking for ideas to optimize my function. I have limited knowledge on multiprocessing so just looking for someone to point me in the right direction! So, I have a pandas DataFrame with the following…
CodeSsscala
  • 729
  • 3
  • 11
  • 23
0
votes
0 answers

Python Parallelize Complex Multi-object for

I've got a multi-object loop like this: a1 = A('path1') # Class from external library a2 = A('path2') # Class from external library b = B('path3', a2) # Class from external library for first, second in foo(a2, a1): # function from external…
Solar
  • 445
  • 1
  • 4
  • 12
0
votes
1 answer

ValueError: negative dimensions are not allowed when loading .pkl file

Although there are many question threads for error ValueError: negative dimensions are not allowed I couldn't find the answer for my problem After training Machine learning model using…
Ranjana Girish
  • 473
  • 7
  • 17
0
votes
0 answers

Loky Initializer (JobLib backend)

The Advanced Executor Setup example from the Loky docs is failing when I try to run it. I get the following: $ python loky_init.py [LokyProcess-1] init Traceback (most recent call last): File "loky_init.py", line 27, in assert…
David
  • 1,391
  • 11
  • 22
0
votes
1 answer

joblib.Parallel is reusing generated numbers instead of redoing for each process

I have many processes to do, each can take up to 20 minutes and uses 100% CPU. I am new to multiprocessing and I decided to use joblib since it seems to let me multiprocess without threading (I have 12 cores and would like to do 12 processes at a…
jpsotka
  • 3
  • 2
0
votes
1 answer

Problem updating joblib library from GitHub repo in IBM Watson Studio

In my program, I need to use some joblib functions. However, when I run the program, I get the error message: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Apparently the library has been updated in this Github repo but…
gerardo monroy
  • 11
  • 1
0
votes
1 answer

Read model as bytes without saving in location in python?

Python, I have saved my model as joblib file in a location, the I open the file in 'rb' read bytes, is it possible to convert straight to bytes instead of saving in a file, import joblib joblib.dump(model, 'model.joblib') #Read as bytes model_bytes…
hanzgs
  • 1,498
  • 17
  • 44
0
votes
1 answer

Predicting not possible due to mismatch of features

I use sklearn to create a logistic regression model based on a xlsx-file. I remove some target and redundant features from the dataset. Now I want to make a prediction and want to get the label based on a new xlsx-file for every row of the file. I'm…
mm1975
  • 1,583
  • 4
  • 30
  • 51
0
votes
0 answers

Reference iterator in parallel loop

I'm trying to reference the iteration value i in a parallel loop using Joblib. I'm getting the error NameError: name 'i' is not defined. Below is my code: def sub5(): curveplots = [] nodes = node_matrix[:,i,:].T curveplots =…
natnay
  • 460
  • 1
  • 5
  • 24
0
votes
1 answer

Can use joblib dump many object?

I have this code. Can I save 'pca' and 'svm_clf' to one file by using joblib.dump? If not, is there any other way to do this? from sklearn.svm import SVC from sklearn.externals import joblib from sklearn import decomposition from sklearn import…
Nguyễn Võ Thành Khang
  • 3
  • 2
0
votes
1 answer

One minimal example of joblib runs on all 4 cores another only on one. Fix?

I am trying to use some parallel computations in my implementation of machine learning algorithms, using joblib, especially the technique used on this page. The following examples are for me to understand parallelism, I have the same problem as the…
Robin Vogel
  • 159
  • 9
0
votes
1 answer

running joblib.Parallel(mlxtend) does not scale in cloud-ml

Im running a job using the mlxtend library. Specifically the sequential_feature_selector that is parallelized using joblib.Parallel source. When I run the package on my local computer it uses all the available CPUs, but when i send the job to…
Pablo
  • 3,135
  • 4
  • 27
  • 43
1 2 3
…
47 48