Questions tagged [ray]

Ray is a library for writing parallel and distributed Python applications. It scales from your laptop to a large cluster, has a simple yet flexible API, and provides high performance out of the box.

At its core, Ray is a library for writing parallel and distributed Python applications. Its API provides a simple way to take arbitrary Python functions and classes and execute them in the distributed setting.

Learn more about Ray:

Ray also includes a number of powerful libraries:

  • Cluster Autoscaling: Automatically configure, launch, and manage clusters and experiments on AWS or GCP.
  • Hyperparameter Tuning: Automatically run experiments, tune hyperparameters, and visualize results with Ray Tune.
  • Reinforcement Learning: RLlib is a state-of-the-art platform for reinforcement learning research as well as reinforcement learning in practice.
  • Distributed Pandas: Modin provides a faster dataframe library with the same API as Pandas.
702 questions
0
votes
1 answer

Error happens when keeping an `ActorHandler` in another actor for later use

I want a remote class to keep another remote class so that it can invoke it later. The following code provide an example import ray ray.init() @ray.remote class Worker: def __init__(self): self.a = 1 self.l = None def…
Maybe
  • 2,129
  • 5
  • 25
  • 45
0
votes
1 answer

Ray remote call executes method, rather than returning ID

I am developing an hyper-parameter optimization function based on this Ray model: https://ray.readthedocs.io/en/latest/auto_examples/plot_hyperparameter.html. The method below, perform_single_rs, is analogous to the evaluate_hyperparameters method…
David Wilt
  • 31
  • 2
0
votes
1 answer

Monitor manual cluster with ray

I don't seem to find in the documentation. Let's say I create my cluster with ray start --head as stated in the documentation. How can I monitor the cluster? When I write the standard command ray monitor, it always throws me an error because I…
silgon
  • 6,890
  • 7
  • 46
  • 67
0
votes
1 answer

Running ray head with external redis

I'm setting up the infrastructure for a Ray project and would like to use an external redis (i.e one not started by ray --head. However that currently does not seem possible, giving me: If --head is passed in, a Redis server will be started, so a…
Niklas B
  • 1,839
  • 18
  • 36
0
votes
2 answers

How to install and use docker in one shell script?

I need to install docker and login to my repository in one script. I'm using ubuntu 16.04 but could also use 18.04. docker_setup.sh: sudo apt-get install -y docker.io sudo service docker start sudo usermod -a -G docker ubuntu exec sudo su -l…
Preethi Vaidyanathan
  • 1,203
  • 1
  • 12
  • 32
0
votes
1 answer

Unable to understand parallelized code output in ray

If I understand correctly the following piece of code should run in parallel @ray.remote class Worker: ... def train(self, item, i): time.sleep(i) logging.info(f'{i} {item}') ... worker = Worker.remote() list = ['a', 'b', 'c'] results…
pleasehalp
  • 136
  • 7
0
votes
1 answer

How to use my own Dockerfile to set up a Ray cluster?

I see how to specify a docker image in ray here: https://ray.readthedocs.io/en/latest/autoscaling.html#common-cluster-configurations But I have my own Dockerfile in my repository. Is it possible to specify that that Dockerfile get spun up on every…
Preethi Vaidyanathan
  • 1,203
  • 1
  • 12
  • 32
0
votes
0 answers

Should I reset running averages of Ray actor in reset_config when reuse_actors=True?

Looking at the pbt_example, I see that the actor's accumulated accuracy is not reset to zero in reset_config, only the new hyperparameters are set (including changing lr in particular). While implementing my own actor, which as state has a running…
Andreas Schuh
  • 1,147
  • 9
  • 4
0
votes
0 answers

Is it possible to disable / reconfigure ray's logging format?

I've written a small class to override default logging and output my logs as JSON. One of the steps is to do the following: logging.basicConfig(format='%(message)s') Works great as resultant logs omit timestamp, etc. However, any logging events…
0
votes
1 answer

Using a class whose methods are decorated by @ray.remote in another directory

I'm learning how to use the python package Ray to parallel my code. I'm facing a problem with a class whose methods are decorated with @ray.remote. It is okay for me to import the class in the same folder and execute the method in the class.…
Quan Hoang
  • 93
  • 7
0
votes
1 answer

ray up local cluster errors - cannot set terminal process group

Using Ubuntu 18.04 with Ray. Trying to start a local cluster (currently 1 server but planning to add more) using the following command (running from terminal on the local server): ray up my_cluster.yaml With the following yaml file: # An unique…
Amit_JCI
  • 159
  • 1
  • 14
0
votes
1 answer

Using track.log in ray

In ray, track.init has to be done before calling track.log. However, if i do track.log outside the function that defines the actor it gives me error complaining that no track session found. But if i do the following, i am creating a tracking session…
Captain Jack sparrow
  • 959
  • 1
  • 13
  • 28
0
votes
1 answer

How to create a new worker(by running a new python script) and connect it to an existed learner

I've created a learner(a remote object decorated by @ray.remote) in one python process, and now I want to create a worker in a new process(run by python new_file.py either in the same machine or in a different machine) and connect it to that…
Maybe
  • 2,129
  • 5
  • 25
  • 45
0
votes
1 answer

Problem setting up ray using google cloud

I try to set up Ray cluster using Kubernetes according to https://ray.readthedocs.io/en/latest/autoscaling.html#kubernetes. Here are my steps: Create a Kubernetes cluster in google cloud platform Connect to the cluster through cloud shell run the…
0
votes
0 answers

Troubleshooting worker usage with Ray

New to Ray, trying to do some troubleshooting on distribution of tasks across the cluster. Right now we have the head node doing all of the work, and not delegating anything at all to workers. Hoping to get some suggestions on logs to investigate…
jsleeuw
  • 283
  • 5
  • 13