Questions tagged [ray]

Ray is a library for writing parallel and distributed Python applications. It scales from your laptop to a large cluster, has a simple yet flexible API, and provides high performance out of the box.

At its core, Ray is a library for writing parallel and distributed Python applications. Its API provides a simple way to take arbitrary Python functions and classes and execute them in the distributed setting.

Learn more about Ray:

Ray also includes a number of powerful libraries:

  • Cluster Autoscaling: Automatically configure, launch, and manage clusters and experiments on AWS or GCP.
  • Hyperparameter Tuning: Automatically run experiments, tune hyperparameters, and visualize results with Ray Tune.
  • Reinforcement Learning: RLlib is a state-of-the-art platform for reinforcement learning research as well as reinforcement learning in practice.
  • Distributed Pandas: Modin provides a faster dataframe library with the same API as Pandas.
702 questions
3
votes
2 answers

Workers not being launched on EC2 by Ray

I am using the Ray module to launch an Ubuntu (16.04) cluster on AWS EC2. In the configuration I specified min_workers, max_workers and initial_workers as 2, because I do not need any auto-sizing. I also want a t2.micro master-node and c4.8xlarge…
3
votes
1 answer

How to evaluate actors trained in custom environments in rllib (Ray)?

Following the SimpleCorridor example I can create my own env and train a model! nice. But when I try to evaluate this trained model, rllib does not recognize my custom env. How can I evaluate a trained model on a custom environment? When I use rllib…
3
votes
0 answers

BayesOptSearch cant be imported even after installation

I wanted to try the new BayesOpt Search from the Ray package. I followed the directions from the Ray documentation and installed the Bayesian Optimization package. I used jupyter notebook to import and start the Tune Trials without problem. But even…
Demi
  • 31
  • 1
3
votes
1 answer

How to define SearchAlgorithm-agnostic, high-dimensional search space in Ray Tune?

I have two questions concerning Ray Tune. First, how can I define a hyperparameter search space independently from the particular SearchAlgorithm used. For instance, HyperOpt uses something like 'height': hp.uniform('height', -100, 100) whereas…
Rylan Schaeffer
  • 1,945
  • 2
  • 28
  • 50
3
votes
1 answer

Standard way to wait for all tasks to finish before exiting

I was wondering - is there a straightforward way to wait for all tasks to finish running before exiting without keeping track of all the ObjectIDs (and get()ing them)? Use case is when I launch @remotes for saving output, for example, where there is…
Zach Barry
  • 33
  • 3
3
votes
1 answer

Run openai-gym environment on parallel

The following code is excerpted from https://bair.berkeley.edu/blog/2018/01/09/ray/. import gym @ray.remote class Simulator(object): def __init__(self): self.env = gym.make("Pong-v0") self.env.reset() def step(self,…
Maybe
  • 2,129
  • 5
  • 25
  • 45
3
votes
1 answer

ap_uniform_sampler() missing 1 required positional argument: 'high' in Ray Tune package for python

I am trying to use the Ray Tune package for hyperparameter tuning of a LSTM implemented using pure Tensorflow. I used the hyperband scheduler and HyperOptSearch algorithms for this and I am also using the trainable class method. When I try to run it…
Suleka_28
  • 2,761
  • 4
  • 27
  • 43
2
votes
1 answer

How to assign a priority to a method in Ray?

I am using Ray to train a distributed RL algorithm. However, the algorithm slows down massively (in terms of how many gradient steps / wall time it does) when using the distributed settings vs. non-distributed. My current theory is that due to the…
direland3
  • 21
  • 3
2
votes
0 answers

Reasons why swifter/dask/ray only use one core for an apply task?

I have this function that I would like to apply to a large dataframe in parallel: from rdkit import Chem from rdkit.Chem.MolStandardize import rdMolStandardize from rdkit import RDLogger RDLogger.DisableLog('rdApp.*') def…
Soerendip
  • 7,684
  • 15
  • 61
  • 128
2
votes
1 answer

Ray Tune for hyper-parameter tuning with specified session directory (!=/tmp/ray/)

I am using Ray Tune to tune the hyper-paramters of a pytorch model. The storage capacity where the default ray session directory is located (/tmp/ray) is limited, thus I want to specify the session directory (i.e. /tmp/ray → /my_directory) when…
stillsen
  • 41
  • 4
2
votes
0 answers

Ray Out of Memory Error - Resources Seemingly not Used

I am new to Ray. I am using it to perform concurrent data transformation on multiple datasets using Python. I am running my head node using a VM with the following specs: 32 cores, 64GB RAM, 256 GB storage I am getting the following error: Traceback…
Irina
  • 1,333
  • 3
  • 17
  • 37
2
votes
0 answers

Installing python packages with extras using Bazel pip_parse

I use Python Rules for Bazel to build my python projects. I use pip_parse to install pip packages, as described in the guide, but that doesn't seem to be working for packages with extras. For example, I have the following dependency in my…
andrii
  • 1,248
  • 1
  • 11
  • 25
2
votes
1 answer

does an instance of a Ray actor run in only one process?

Can the methods of a Ray actor be called in parallel on a single remote instance? it seems i have to create a new instance of the actor, then call the method on that instance in order to achieve parallelization. See the code and out put…
mike01010
  • 5,226
  • 6
  • 44
  • 77
2
votes
1 answer

Ray monitoring fails when binding to empty address

I'm learning to use RLlib. I've been running it in my debugger on an example script, and it works, but for some reason I get an error message about the monitoring service failing. This is the traceback: File…
Ram Rachum
  • 84,019
  • 84
  • 236
  • 374
2
votes
1 answer

RAM Usage Keeps Going Up While Training an RL Network Using RLLib and TensorFlow

I have been using older versions of Ray and TensorFlow, but recently transitioned to the following most up-to-date versions on a Linux Ubuntu 20.04 setup. ray==2.0.0 tensorflow==2.10.0 cuDNN==8.1 CUDA==11.2 While training a single-agent network, I…
troymyname00
  • 670
  • 1
  • 14
  • 32