Ray RLlib is an open-source Python library for Reinforcement Learning. Use with applicable framework tags, such as TensorFlow or PyTorch.
Questions tagged [rllib]
105 questions
1
vote
0 answers
ValueError: RolloutWorker has no `input_reader` object! Cannot call `sample()` occurs when try to start training in RLlib
I just started to use Ray-RLlib and I try to restructure my VSL algorithm (using SUMO) to be able to work with RLlib. This is a MARL (4 agents) DQN approach. I restructured the environment based on the documentation and created the training…
komate1995
- 11
- 2
1
vote
0 answers
No module named 'ray.rllib.agents.registry'
When running the example (examples/train.py singleagent_ring) to test my rllib installation to use flow with SUMO, I am getting the following error:
from ray.rllib.agents.registry import get_agent_class
ModuleNotFoundError: No module named…
user21753265
- 11
- 1
1
vote
0 answers
i run a RL on SUMO using traci and rllib but when the sumo-gui lanches nothing happens
i am Getting Started with RLlib and sumo i am tring a rl algorithm to manage traffic lights here is the code
description of the code :
This code defines a reinforcement learning environment for traffic signal control using SUMO (Simulation of Urban…
shi ft
- 19
- 1
1
vote
0 answers
Ray tune samples more than one value for the same model in multi-agent environment
So I have this weird behavior of ray tune that I can't make sense of.
What I'm trying to do:
I have setup a custom rllib multi-agent env with two agents
Both agents have different observation and action spaces
Both should be trained with PPO but…
Pat396
- 21
- 3
1
vote
1 answer
Tensor shape seems to disappear when indexing result of tf.shape(tensor)
When I try to index the result of tf.shape(tensor) where tensor is some tensor, the result seems to turn into None unexpectedly. For example, I ran this code:
>>> from ray.rllib.models.utils import try_import_tf
>>> tf1, tf, tfv = try_import_tf()…
Andi Qu
- 186
- 1
- 10
1
vote
1 answer
How can I import a ray rllib pytorch whole model into next round training and subsquent inference using torch save load method other than checkpoints
In ray rllib, I usually apply ray.tune.run a ppo trainning like this:
ray.init(log_to_driver=False, num_cpus=3,
local_mode=args.local_mode, num_gpus=1)
env_config={"code":"codeA"}
config={
env_config={
"code":"codeA"},
…
Dr. GUO
- 11
- 1
1
vote
2 answers
RLlib PPO continuous actions seem to become nan after total_loss = inf?
After some amount of training on a custom Multi-agent environment using RLlib's (1.4.0) PPO network, I found that my continuous actions turn into nan (explodes?) which is probably caused by a bad gradient update which in turn depends on the…
hridayns
- 697
- 8
- 16
1
vote
2 answers
How to prevent my reward sum received during evaluation runs repeating in intervals when using RLlib?
I am using Ray 1.3.0 (for RLlib) with a combination of SUMO version 1.9.2 for the simulation of a multi-agent scenario. I have configured RLlib to use a single PPO network that is commonly updated/used by all N agents. My evaluation settings look…
hridayns
- 697
- 8
- 16
1
vote
1 answer
save the model and checkpointing for algorithm-Trainers in ray-rllib
Does anyone know how can I do checkpointing and saving the model for algorithm-Trainer models in ray-rllib?
I know that that is available for ray.tune, but it seems that it is not directly possible to do so for the rllib algorithms.
Afshin Oroojlooy
- 1,326
- 3
- 21
- 43
1
vote
1 answer
Passing optional arguments when registering gym environment in rllib
I've been following the helpful example here to create a custom environment in gym, which I then want to train in rllib.
My environment has some optional parameters which I would like to select when training. I think the relevant part of code is in…
hughes
- 27
- 3
1
vote
1 answer
Complex Action Mask in rllib
A parametric/variable-length action model is provided in rllib examples. The example assumes the outputs are logits for a single Categorical action dist. How to getting this work with a more complex output?
For example, there are 200 different balls…
Gulololo
- 11
- 1
1
vote
0 answers
Correct use of a2c.A2cTrainer Rllib with gym and pettingzoo
I'm building a speaker listener training environment with rllib from this article.
Using pettingzoo and supersuit.
I've encountered the following error:
NotImplementedError: Cannot convert a symbolic Tensor (default_policy/cond/strided_slice:0) to a…
user13399343
- 11
- 2
1
vote
1 answer
Rllib OfflineData preparation for SAC
I have some offline experiences: (s, a, r, s') that were generated with a heuristic. And I want to use these when training SAC agents. Using the example saving_experiences to prepare my data gives me an error when using with SAC. Here is a colab…
Anas BELFADIL
- 106
- 9
1
vote
1 answer
RLLib tunes PPOTrainer but not A2CTrainer
I am making a comparison between both kind of algorithms against the CartPole environment. Having the imports as:
import ray
from ray import tune
from ray.rllib import agents
ray.init() # Skip or set to ignore if already called
Running this works…
Angelo
- 575
- 3
- 18
1
vote
2 answers
Using Ray RLlib with custom simulator
I'm very new to Ray RLlib and have an issue with using a custom simulator my team made.
We're trying to integrate a custom Python-based simulator into Ray RLlib to do a single-agent DQN training. However, I'm uncertain about how to integrate the…
Kai Yun
- 97
- 8