Joblib is a set of tools to provide lightweight pipelining in Python.
Joblib is a set of tools to provide lightweight pipelining in Python.
Questions tagged [joblib]
715 questions
-1
votes
3 answers
Joblib doesn't run on multiple cores
I need to fit multiple classifiers (from scikit-learn) and score them.
clf1.fit(x_fit,Y_fit)
clf2.fit(x_fit,Y_fit)
clf3.fit(x_fit,Y_fit)
clf4.fit(x_fit,Y_fit)
acc1 = clf1.score(x_score,Y_score)
acc2 = clf2.score(x_score,Y_score)
acc3 =…
Lipa
- 35
- 2
- 10
-1
votes
1 answer
Why do I get Memory Error in stringsource?
I am trying to execute a cython file in parallel. The skeleton of my code is:
def build_DF_single(lo, hi, map[long, set[string]] authors_id_map, map[long, set[string]] authors_org_map,
map[long, set[string]] fos_name_map, map[long, set[string]]…
Caprikuarius
- 176
- 7
-1
votes
1 answer
AttributeError: 'int' object has no attribute 'lower'
I am trying to pass a tweet from a flask UI and be able to make a prediction of the type of the tweet if its a donation, disaster etc.
Here is a working code from Jupyter notebook:
loaded_model = joblib.load('NB_spam_model.pkl')
result =…
Nani3
- 487
- 1
- 6
- 11
-1
votes
1 answer
How to return a generator using joblib.Parallel()?
I have a piece of code below where the joblib.Parallel() returns a list.
import numpy as np
from joblib import Parallel, delayed
lst = [[0.0, 1, 2], [3, 4, 5], [6, 7, 8]]
arr = np.array(lst)
w, v = np.linalg.eigh(arr)
def proj_func(i):
return…
Leockl
- 1,906
- 5
- 18
- 51
-1
votes
1 answer
Loaded model receives different prediction compared to saved model
I am trying to save a model and load it into a different session, but I am having prediction inconsistencies, and I would appreciate any help that can be offered. So here is what I did...
First, after running the model, I used this code to save the…
Nadimprodutions
- 15
- 5
-1
votes
1 answer
Why pycharm having error while loading .joblib file
I have a trained decision tree model file music-recommender.joblib. When I am using Jupyter notebook, I am able to load this trained model successfully and able to do prediction. But the same code I used in pycharm and it is showing error.
In…
Anu
- 1,123
- 2
- 13
- 42
-1
votes
2 answers
How to serialize a large randomforest classifier
I am using sklearn's randomforestclassifier to predict a set of classes. I have over 26000 classes and therefore the size of classifier is exceeding over 30 GBs. I am running it on linux with 64 GB of RAM and 20 GB storage.
I am trying to pickle my…
DumbCoder
- 233
- 2
- 9
-2
votes
1 answer
Best multiprocessing technique to speed up this code
I am trying to learn more about parallelisation to speed up this classification code. I literally started reading about it less than 24 hours ago (to share some context). I am wondering which multiprocessing technique will be the best to tackle this…
Carla
- 1
- 1
- 1
-2
votes
1 answer
Create a parallel 'for' loop?
I have some code that loops through a dictionary where the key is a word and it assigns the term frequency to that key. tok is a list of tokens from some text.
def calculateTF(wordDict, tok):
tfDict = {}
termCount = len(tok)
for word,…
MK de Silva
- 36
- 1
- 3
-3
votes
1 answer
Fastest way of running GridSearchCV with multiple CPUs
I need to perform GridSearchCV with 4 different classifiers. I have at my disposal 128 CPUs which i can use in different sessions (2 sessions with 64CPus, 4 sessions with 32 CPUs, etc).
I'd like to ask if there's any way of knowing which could be…
heresthebuzz
- 678
- 7
- 21