OpenAI gym: How to get pixels in CartPole-v0

Question

I would like to access the raw pixels in the OpenAI gym CartPole-v0 environment without opening a render window. How do I do this?

Example code:

import gym
env = gym.make("CartPole-v0")
env.reset()
img = env.render(mode='rgb_array', close=True) # Returns None
print(img)
img = env.render(mode='rgb_array', close=False) 
          # Opens annoying window, but gives me the array that I want
print(img.shape)

PS. I am having a hard time finding good documentation for OpenAI gym. Is it just me, or does it simply not exist?

Edit: I don't need to ever open the render video.

Answer 1

I was curious about same so I started looking into the source code and this is what I found.

Open AI uses pyglet for displaying the window and animations.

For showing the animation everything is drawn on to window and then rendered.

And then pyglet stores what is being displayed on to a buffer.

Dummy version of how code is written in open AI

import pyglet
from pyglet.gl import *
import numpy as np

display = pyglet.canvas.get_display()
screen = display.get_screens()
config = screen[0].get_best_config()

pyglet.window.Window(width=500, height=500, display=display, config=config)

# draw what ever you want

#get image from the buffer

buffer = pyglet.image.get_buffer_manager().get_color_buffer()

image_data=buffer.get_image_data()

arr = np.frombuffer(image_data.get_data(),dtype=np.uint8)

print(arr)
print(arr.shape)

output: [0 0 0 ... 0 0 0]
(1000000,)

so basically every image we get is from buffer of what is being displayed on the window. So if we don't draw anything on window we get no image so that window is required to get the image. so you need to find a way such that windows is not displayed but its values are stored in buffer. I know its not what you wanted but I hope it might lead you to a solution.

Answer 2

I've just gone through half of the gym source code line by line, and I can tell you that 1, the observation space of cartpole is digits to the ai, not pixels. eg, from their cartpole env py file...

Observation:
        Type: Box(4)
        Num     Observation               Min                     Max
        0       Cart Position             -2.4                    2.4
        1       Cart Velocity             -Inf                    Inf
        2       Pole Angle                -0.209 rad (-12 deg)    0.209 rad (12 deg)
        3       Pole Angular Velocity     -Inf                    Inf

So, the pixels are for you at this point. And 2, if your goal is to teach the ai on pixels, you will need to render images from your data-in array, then pass them THROUGH the observation space as a pixel array, like Maunish Dave shows. OpenAI's Atari version does this.

If you want a better guide, don't read the OpenAI Docs, read the Stable Baseline docs here: https://stable-baselines.readthedocs.io/

Answer 3

Anyone tried the PixelObservationWrapper? Seems like that should yield the pixel values into the observation space, but I haven't tried it yet.

Name: PixelObservationWrapper

Type: gym.ObservationWrapper

Arguments: env, pixels_only=True, render_kwargs=None, pixel_keys=("pixels",)

Description: Augment observations by pixel values obtained via render. You can specify whether the original observations should be discarded entirely or be augmented by setting pixels_only. Also, you can provide keyword arguments for render.

Answer 4

Someone offers an answer here:

https://github.com/openai/gym/issues/374

"The atari and doom environments give pixels in their observations (ie, the return value from step). I don't think any other ones do.

render produces different results on different OSes, so they're not part of any official environment for benchmarking purposes. But if you want to create a new environment where the observation is in pixels, you could implement it by wrapping an existing environment and calling render."

I'm also working on getting raw pixels as well and I'm trying to find a way to see if what has been returned is what I expect it is.

The documentation can be found: https://gym.openai.com/docs

And a forum for discussing OpenAI: discuss.openai.com

Although its not very lively.

Answer 5

I have faced the similar problem:

This is how fixed it, in rendering.py file at /gym/envs/classic_control find the following line in the Viewer class:

self.window = pyglet.window.Window(width=width, height=height, display=display)

Change this line to:

self.window = pyglet.window.Window(width=width, height=height, display=display, visible=False)

Hope it helps!!