Stack Overflow
Stack Overflow

Questions tagged [rllib]

Ray RLlib is an open-source Python library for Reinforcement Learning. Use with applicable framework tags, such as TensorFlow or PyTorch.

105 questions
0
votes
0 answers

How to read RLLib results? It seem they indicate that training does not happen within each training iteration but it does train

I am a looking at the example code for RLLib from https://docs.ray.io/en/latest/rllib/rllib-training.html#rllib-config-framework with modified line .rollouts(num_rollout_workers=10, horizon = 50000) to use as many workers as I have CPU cores and…
sebtac
  • 538
  • 5
  • 8
0
votes
0 answers

What is the cause of the low CPU utilization in rllib PPO? What does 'cpu_util_percent' measure?

I implement multiagent ppo in rllib with a custom environment, it learns and works well except for the speed performance. I wonder if an underutilized CPU may cause the issue, so I want to know what ray/tune/perf/cpu_util_percent measures. Does it…
Kuan-Ho Lao
  • 1
  • 1
0
votes
0 answers

How to set up `timesteps_total` in the config in Python API? (RLlib)

I just started playing with RLlib and I wanted to test offline DQN training on a CartPole. Thus, I generated the data as in the tutorial: rllib train --run=PG --env=CartPole-v1 --config='{"output": "/tmp/cartpole-out", "output_max_file_size":…
Fallen Apart
  • 723
  • 8
  • 20
0
votes
1 answer

RolloutWorker problem when try to execute PPOConfig: Exception raised in creation task: The actor died because of an error raised in its creation task

I am trying to follow the steps mentioned on "Getting Started with RLlib" (https://docs.ray.io/en/latest/rllib/rllib-training.html) along with my custom environment. However my run doesn't work in the first code block show in the guide. This is…
Jason_mils
  • 97
  • 1
0
votes
0 answers

Access the action distribution network in Ray's RLlib library

I would like to train a Proximal Policy Optimization (PPO) type model using RLlib and then serve the action distribution model using Tensorflow Lite or the equivalent PyTorch technology. I am interested in determining the ranking of actions and not…
Setjmp
  • 27,279
  • 27
  • 74
  • 92
0
votes
0 answers

how do i change the framestack from 3 to 4 in rllib

I am exploring rllib, training atari ‘breakout’. The difficulty I face is that the making the trained agent to play the game. The First Problem is that the Agent.compute_single_action(obs) doesn’t automatically preprocess the (1, 210, 160, 3)…
A Rajaguhan
  • 1
  • 2
0
votes
0 answers

Rllib. PPO. No trial resources are available for launching the actor

Im trying to tune hyperparameters with optuna. But it constantly gives me an error. It looks like to solve i need to some how register gpus in optuna. I failed to do so. I do not understand how registrate gpus for optuna Tuner. Error: …
Alexander Alexandrov
  • 101
  • 3
0
votes
0 answers

Defining a 4D observation space in gym

I am new to reinforcement learning and I wanna create a gym 4D environment, where it is a 468x225x182x54 space. And every location in this space has a unique reward (or penalty). And my agent (e.g. rabbit) can jump anywhere in this space and will be…
Mshz
  • 17
  • 4
0
votes
0 answers

Custom gym environnement does not converge well. What hyperparameters should I use?

I created a custom gym environment representing ships which have to intercept asteroids. I used the ray RLLib to train a PPO algorithm on my environment My issue is that using ray RLLib (a PPOTrainer with a simple config), the environment training…
Clm28
  • 9
  • 2
0
votes
1 answer

Attribute Error: 'RandomNumberGenerator._generator_ctor' in gym.utils.seeding

I´m trying to run a reinforcement learning algorithm for a production line optimization. As an engineering student I am not very familiar with coding so I´m looking for help from you guys. I get this error when trying to train the agent:…
Benedict Janssen
  • 1
0
votes
1 answer

Migration tips for ray.rllib.models: DeprecationWarning: `Model` has been deprecated. Use `ModelV2` instead

For migration from ray 0.x to ray 1.x, ray.rllib.models.Model must be replaced with ray.rllib.models.ModelV2. Are there any migration guidelines describing how to do this properly? Thank you.
Anton
  • 494
  • 5
  • 19
0
votes
0 answers

Episode Length and train_batch_size compatibility with RLLib PPO

I have created a custom single agent Gym environment which I am trying to train using a quite a simple action space and reward function. self.action_space = spaces.MultiDiscrete([3, 3]) Each gym step is a single second in my custom simulator and…
deeplearningDamien
  • 1
  • 2
0
votes
1 answer

How to input images in rllib

last time I saw library rllib: https://docs.ray.io/en/latest/rllib/index.html. It has amazing features for reinforcement learning, but unfortunately, I couldn't find a way to input images as an observation without flattening them (I basically want…
dato nefaridze
  • 127
  • 1
  • 6
0
votes
1 answer

Using ray with custom environment created with gym.make()

I would like to run the following code but instead of Cartpole use a custom environment: import ray import ray.rllib.agents.dqn.apex as apex from ray.tune.logger import pretty_print def train_cartpole() -> None: ray.init() config =…
blindeyes
  • 409
  • 3
  • 13
0
votes
1 answer

Keep Training in Ray RLLib even if environment throws exception

Sometimes I have environments, which can run into unhandled situations after choosing several bad actions, which yields exceptions. Instead of crashing the agent, I'd like to just retry and ignore that episode. How to achieve this?
Dominik
  • 116
  • 10
1 2 3 4 5
6
7