2

When training the "CartPole" environment with Stable Baselines 3 using PPO, I get that training the model using cuda GPU is almost twice as slow as training the model with just the cpu (both in google colab and in local).

I thought using cuda for this kind of task would make it faster, am I doing something wrong?

My code is:

import gym
import time
from stable_baselines3 import PPO

env = gym.make("CartPole-v1")

t1 = time.time()
model = PPO("MlpPolicy", env, verbose=0, device="cuda")
model.learn(total_timesteps=10_000)
print(f"Time with cuda : {time.time()-t1:.2f}s")

t1 = time.time()
model = PPO("MlpPolicy", env, verbose=0, device="cpu")
model.learn(total_timesteps=10_000)
print(f"Time with cpu : {time.time()-t1:.2f}s")

env.close()

The output is:

Time with cuda : 21.76s
Time with cpu : 13.33s
talonmies
  • 70,661
  • 34
  • 192
  • 269
Joel
  • 21
  • 2
  • 2
    Relative performance CPU/GPU depends on the complexity of the calculation. For simple computations the time it takes to copy the data from system memory to CUDA device and then back far outweighs any advantage that GPU has over the CPU. So my guess will be that the data transfer is the bottleneck here. – NotAName Feb 13 '23 at 05:07

0 Answers0