Python: Alternative to hyperopt that can support multiprocessing?

Question

Is there any other than HyperOpt that can support multiprocessing for a hyper-parameter search? I know that HyperOpt can be configured to use MongoDB but it seems like it is easy to get it wrong and spend a week in the weeds, is there anything that is more popular and effective?

Have you seen [this discussion](https://stackoverflow.com/q/44181511/712995)? — Maxim, Mar 23 '18 at 10:22

score 2 · Answer 1 · answered Apr 01 '18 at 08:29

Check out Ray Tune!

You can use it for multiprocessing and multi-machine executions of random search, grid search, and evolutionary methods. It also has implementations of popular algorithms such as HyperBand.

Here's the docs page - ray.readthedocs.io/en/latest/tune.html

As an example to run 4 parallel experiments at a time:

import ray
import ray.tune as tune


def my_func(config, reporter):  # add the reporter parameter
    import time, numpy as np
    i = 0
    while True:
        reporter(timesteps_total=i, 
                 mean_accuracy=i ** config["alpha"])
        i += 1
        time.sleep(.01)


tune.register_trainable("my_func", my_func)
ray.init(num_cpus=4)
tune.run_experiments({
    "my_experiment": { 
        "run": "my_func", 
        "stop": { "mean_accuracy": 100 }, 
        "config": { 
            "alpha": tune.grid_search([0.2, 0.4, 0.6]), 
            "beta": tune.grid_search([1, 2]) } } })

Disclaimer: I work on this project - let me know if you have any feedback!

I tried running it for 1000 runs, I see overhead is more for the example that was given. Might be useful if you use *distributed training only* — Dileep, Sep 08 '20 at 03:58

score 1 · Answer 2 · answered Sep 17 '20 at 06:54

1

Some models(say RandomForest) have "njobs" parameter for use of number of cores. You can try njobs=-1; thus even if hyperopt uses 1 core, each trial would use all the cores, speeding up the process.

answered Sep 17 '20 at 06:54

Tanveer

51
1
1
7

Python: Alternative to hyperopt that can support multiprocessing?

2 Answers2