5

Ok so there must be some option in OpenAI gym that allows it to run as fast as possible? I have a linux environment that does exactly this(run as fast as possible), but when I run the exact setup on Windows, it instead runs it only in real-time.

The specific environment I'm working on is in Montezuma's Revenge Atari game. I run the exact same code but on my linux setup, it was able to run the game much faster. Just so you know my linux computer has worse specs than my windows.

here is some code for those who want it:

for i in range(episode_count):
    ob = env.reset()
    ob = np.expand_dims(ob, axis=0)
    time = 0
    while True:
        time += 1
        action = agent.act(ob, reward, done)
        new_ob, reward, done, _ = env.step(action)
        new_ob = np.expand_dims(new_ob, axis=0)
        agent.remember(ob, action, reward, new_ob, done)

        ob = new_ob
        env.render()
        if done or time >= 1000:
            print("episode: {}/{}, time: {}, e: {:.3}"
                  .format(i, episode_count, time, agent.epsilon))
            if len(agent.memory) > batch_size:
                agent.replay(batch_size)
            # agent.save("./save/montazuma-dqn.h5")
            break

The same thing runs on two setups, get different results in run speed.

stoplime
  • 81
  • 1
  • 5
  • The speed with which an environment runs really should just depend on how quickly you call `step()` in your own code outside the environment. Could you post that code in your question? – Dennis Soemers Apr 12 '18 at 09:55
  • I posted some code, but it is running the same code in both setups, so I don't think it's the code. I suspect there is a setting in the openai gym that changes how fast it runs the game. – stoplime Apr 12 '18 at 20:24
  • Try replacing your agent temporarily with one that definitely doesn't take a lot of processing time (like an agent that always selects random actions), see if that changes things. I don't think there should be any difference in performance from `gym`'s side really, I suspect the difference in performance may be in your agent's code. Is it possible that, for example, on Windows you're running on CPU only with `tensorflow` for example, and on Linux have it set up to use the GPU? – Dennis Soemers Apr 13 '18 at 07:43

1 Answers1

2

For anyone looking at this in the future, it's because of self.env.render(). Took me some time to figure out what was slowing my code down. It ends up rendering each action takes time, which will slow your code down

Gabe
  • 21
  • 3