0

I have been trying to figure out a way to Pre-Train a model using Stable-baselines3.

In the original documentation for Stable-baseline (the version which runs on Tensorflow 1.X), this seems to be an easy task:

    from stable_baselines import PPO2
    
    from stable_baselines.gail import ExpertDataset
    
    dataset = ExpertDataset(expert_path='expert_cartpole.npz', traj_limitation=1, batch_size=128)
    
    model = PPO2('MlpPolicy', 'CartPole-v1', verbose=1)
    
    \# Pretrain the PPO2 model
    
    model.pretrain(dataset, n_epochs=1000)

The problem is, there is no "from stable_baselines3.gail import ExpertDataset"

basically what I want to do is I want to create a .npz file using a specific algorithm to generate the observation, rewards, action and then pass that to an RL agent.

I found the original code from this document:

https://readthedocs.org/projects/stable-baselines/downloads/pdf/master/

update 4 March 2023: I found this link that explains how this was done on Stable Baseline: https://stable-baselines.readthedocs.io/en/master/guide/pretrain.html And I want to do the exact same thing on SB3.

Lord-Goku
  • 1
  • 4

1 Answers1

0

You can use imitation, which is built on top of SB3.

pip install imitation

Then you can use:

from imitation.algorithms.adversarial.gail import GAIL

Here is their docs

Minh-Long Luu
  • 2,393
  • 1
  • 17
  • 39
  • Thanks Minh-Long, I have seen that API but the issue is still the same. the imitation API does not provide `ExpertDataset(expert_path='expert_cartpole.npz', traj_limitation=1, batch_size=128)` With the imitation API, I'd need to save my expert data myself and then load it as numpy arrays and then pass it to `train_disc(*, expert_samples=None, gen_samples=None)` – Lord-Goku Mar 06 '23 at 00:09