Questions tagged [ray]

Ray is a library for writing parallel and distributed Python applications. It scales from your laptop to a large cluster, has a simple yet flexible API, and provides high performance out of the box.

At its core, Ray is a library for writing parallel and distributed Python applications. Its API provides a simple way to take arbitrary Python functions and classes and execute them in the distributed setting.

Learn more about Ray:

Ray also includes a number of powerful libraries:

  • Cluster Autoscaling: Automatically configure, launch, and manage clusters and experiments on AWS or GCP.
  • Hyperparameter Tuning: Automatically run experiments, tune hyperparameters, and visualize results with Ray Tune.
  • Reinforcement Learning: RLlib is a state-of-the-art platform for reinforcement learning research as well as reinforcement learning in practice.
  • Distributed Pandas: Modin provides a faster dataframe library with the same API as Pandas.
702 questions
0
votes
0 answers

UnreconstructableError: Object is lost (either LRU evicted or deleted by user) and cannot be reconstructed

My code looks like that: # Features' construction - Multiprocessing # import pandas as pd import time import ray import multiprocessing ray.shutdown() num_cpus =…
Outcast
  • 4,967
  • 5
  • 44
  • 99
0
votes
1 answer

Dataset enumeration (epoch and batchSize) when implementing Ray.Tune PBT hyperparameter optimization

This is my first time trying to use Ray.Tune for hyperparameter optimization. I am confused as to where in the Ray code I should initialize the dataset as well as where to put the for-loops for defining the epoch and enumerating the dataset…
LaMaster90
  • 11
  • 1
0
votes
1 answer

How and when is ray.method(num_return_values) used?

The ray tutorial explains, that for having a method on an object, which returns multiple object_ids one can use the @ray.method() decorator see here. But in the example 'Learning to play Pong' the method compute gradient actually has two return…
LJKS
  • 899
  • 2
  • 9
  • 17
0
votes
0 answers

Python Parallelize Complex Multi-object for

I've got a multi-object loop like this: a1 = A('path1') # Class from external library a2 = A('path2') # Class from external library b = B('path3', a2) # Class from external library for first, second in foo(a2, a1): # function from external…
Solar
  • 445
  • 1
  • 4
  • 12
0
votes
0 answers

How can I ensure that one worker runs on a CPU?

I create several workers and a learner using ray. How can I ensure that each actor runs on a CPU without contention?
Maybe
  • 2,129
  • 5
  • 25
  • 45
0
votes
0 answers

Workers on AWS EC2 cluster raise RayOutOfMemory error after some iterations?

I spinned up a ray cluster on AWS to perform some grid search for my tf.keras model. And for almost all jobs (around 500) the workers throw an RayOutOfMemory exception after some iterations (between 6 and 53) and I can not figure out where the issue…
Denis
  • 13
  • 2
0
votes
1 answer

what are the options to implement random search?

So i want to implement random search but there is no clear cut example as to how to do this. I am confused between the following methods: tune.randint() ray.tune.suggest.BasicVariantGenerator() tune.sample_from(lambda spec: blah blah…
shreyagu
  • 1
  • 1
0
votes
0 answers

`ray up` only launches Docker container on head

I am trying to use the Ray library to launch runs on multiple remote machines with Docker. Per the docs, I use ray up CONFIG_YAML to set up my cluster and ray submit [OPTIONS] CLUSTER_CONFIG_FILE SCRIPT to run a script on them. The problem is that…
ethanabrooks
  • 747
  • 8
  • 19
0
votes
1 answer

How to change the experiment file path generated when running Ray's run_experiments()?

I'm using the following spec on my code to generate experiments: experiment_spec = { "test_experiment": { "run": "PPO", "env": "MultiTradingEnv-v1", "stop": { "timesteps_total": 1e6 }, …
0
votes
1 answer

How to log more frequently than evaluating with `ray.tune.Trainable`

I am interested in using the tune library for reinforcement learning and I would like to use the in-built tensorboard capability. However, the metric that I am using to tune my hyperparameters is based on a time-consuming evaluation procedure that…
ethanabrooks
  • 747
  • 8
  • 19
0
votes
1 answer

Ray local cluster web-ui shows 0 workers

Launching the ray head node with web-ui: ray start --head --redis-port=6379 --include-webui --num-cpus=0 when I navigate to the UI (after I start training) I don't see any workers being used. Can someone confirm that this works for them? Thanks.
Andriy Drozdyuk
  • 58,435
  • 50
  • 171
  • 272
0
votes
0 answers

handling masked numpy array with ray

Is it possible for ray (python) to handle operations/stats over masked np.arrays? [np.nansum(...) in this case]. For non-masked arrays works perfectly; but it crashes for masked arrays (see message error below). import numpy as np import numpy.ma as…
Manuel F
  • 145
  • 2
  • 7
0
votes
1 answer

Class membership of Ray-AABB intersection function

I'm writing code for mouse picking in 3D space. So far I've made Ray and AABB classes. What I need is a function for Ray-AABB intersecting. I know how to write it and make it work, my question is which class should define said funcion? Should it be…
NRUB
  • 404
  • 4
  • 17
0
votes
1 answer

How is Ray's RLlib making use of LazyFrames?

I'm trying to find out how RLLib makes efficient use of the frames, i.e. how it avoids saving repetitive frames into memory, which in the OpenAI baselines is done via LazyFrames. In Ray's RLLib atari_wrapper.py it seems that all observations are…
Muppet
  • 5,767
  • 6
  • 29
  • 39
0
votes
1 answer

The custom model weights are not changing (tensorflow)

I am trying to train an agent in a racing car simulator. I have built a custom model. I found that the weights are not changing during the training. I am trying to train an agent in a racing car simulator. I have built a custom model following the…
iceman126
  • 45
  • 2
  • 8