Panda3d rendered window to numpy array

Question

I was following this topic to try to get the buffer data from a camera and save it as a numpy array. Following the topic I reused this code:

base.graphicsEngine.renderFrame()
dr = base.camNode.getDisplayRegion(0)
tex = dr.getScreenshot()
data = tex.getRamImage()
image = np.frombuffer(data,np.uint8)
image.shape = (tex.getYSize(),tex.getXSize(),tex.getNumComponents())
print(image)

However, I get this error:

  File "main.py", line 137, in __init__
    image = np.frombuffer(data,np.uint8)
AttributeError: 'panda3d.core.ConstPointerToArray_unsigned_char' object has no attribute '__buffer__'

Any advice?

Answer 1

Solved it by changing the original code to the following:

base.graphicsEngine.renderFrame()
dr = base.camNode.getDisplayRegion(0)
tex = dr.getScreenshot()
data = tex.getRamImage()
v = memoryview(data).tolist()
img = np.array(v,dtype=np.uint8)
img = img.reshape((tex.getYSize(),tex.getXSize(),4))
img = img[::-1]
cv2.imshow('img',img)
cv2.waitKey(0)

The image ends up being flipped for whatever reason after reshaping the numpy array hence the 3rd line from the bottom. You should see an identical image of whatever your camera sees when you run this snippet as an accept key or something. Hope this helps someone having the same issues.

Answer 2

I was having trouble with a few bugs on pyrender, so i wanted to use panda3d as a offscreen render, and this was very usefull, expanding on what Masa Hu posted, here is the full example on how to do that:

from direct.showbase.ShowBase import ShowBase
import cv2
import numpy as np


base = ShowBase( windowType='offscreen')
box = base.loader.loadModel("meshes/box.egg")
box.setPos(0,10,0)
box.reparentTo(base.render)
base.graphicsEngine.renderFrame()
dr = base.camNode.getDisplayRegion(0)
tex = dr.getScreenshot()
data = tex.getRamImage()
v = memoryview(data).tolist()
img = np.array(v,dtype=np.uint8)
img = img.reshape((tex.getYSize(),tex.getXSize(),4))
img = img[::-1]
cv2.imshow('img',img)
cv2.waitKey(0)