0

Im running a job using the mlxtend library. Specifically the sequential_feature_selector that is parallelized using joblib.Parallel source. When I run the package on my local computer it uses all the available CPUs, but when i send the job to cloud-ml it only uses one core. It doesn't matter what is the number that i put in the n_jobs parameter. I´ve also tried with differents machine types but same thing happen. Does anybody know what the problem might be ?

Pablo
  • 3,135
  • 4
  • 27
  • 43
  • We don't put any restriction on number of cores. Can you share your job id and repro with us via cloudml-feedback@google.com please? – Guoqing Xu Apr 18 '19 at 00:32
  • Hi @GuoqingXu Im trying to reproduce the error with a sample code, I will add it to the question as soon as I have it. Thanks !! – Pablo Apr 18 '19 at 14:44
  • Thanks @GuoqingXu finally it was a problem with the sklearn version, fixing the version in the package setup fixed the problem – Pablo Apr 18 '19 at 18:47
  • Good to know! Thanks! – Guoqing Xu Apr 19 '19 at 16:36

1 Answers1

1

For anyone that might be interested, we solve the problem fixing the sklearn version in the setup.py to the 0.20.2. we had sklearn in the packages before, but without a version. 

#setup.py
from setuptools import find_packages
from setuptools import setup

REQUIRED_PACKAGES = ['joblib==0.13.0',
                     'scikit-learn==0.20.2',
                     'mlxtend']
Pablo
  • 3,135
  • 4
  • 27
  • 43