Open3D scene renderer render_to_image returns black screen

Question

I am trying to render a scene that simply contains a mesh of obj file and the material file. It looks okay when I try to view with

o3d.visualization.draw([{
    "name": "Model",
    "geometry": model,
    "material": mat
}])

But when I try to render the same scene with the following code, the output image is just black. I have tried everything but nothing seems to be working at the moment

import numpy as np
import open3d as o3d
import cv2

def main():
    render = o3d.visualization.rendering.OffscreenRenderer(640, 480)
    model, mat=getModel()
    render.scene.set_background([0, 0, 0, 0])
    render.scene.add_geometry("model", model, mat)
    render.scene.set_lighting(render.scene.LightingProfile.NO_SHADOWS, (0, 0, 0))
    render.scene.camera.look_at([0, 0, 0], [0, 10, 0], [0, 0, 1])
    img_o3d = render.render_to_image()
    o3d.io.write_image("mtest2.jpeg", img_o3d, 9)
    img = np.array(img_o3d)
    cv2.imshow("model", img)
    cv2.waitKey(1)

def getModel():

    model_name = "mouse.obj"
    model = o3d.io.read_triangle_mesh(model_name)
    material = o3d.visualization.rendering.MaterialRecord()
    material.shader = "defaultLit"
    albedo_name = "albedo.jpeg"
    material.albedo_img = o3d.io.read_image(albedo_name)
    return (model, material)

main()

@J_H the question was fine imo. I would try to move away from being unhelpful and try to answer the question if possible and then provide support for new members. Thanks! — smerkd, Jan 22 '23 at 04:41

score 1 · Answer 1 · answered Apr 28 '22 at 18:43

I'm also certain it's because your camera is not looking at the object.

First method

In your render try changing the position. Like so

render.scene.camera.look_at([0, 0, 0], [0, 0, -800], [0, 1, 0])

Remember the first argument is where to look, the second argument is the position of the camera (which has to be far enough to see the object), and the third is the camera orientation.

The rest of your code is fine, and I've tested it to render some images.

Second method

Also, big help to find the camera position, for any orientation, is using the visualizer. Use the mouse to navigate and find the position which you want to render and then hit the button p. This will save a JSON file in the same directory called something like ScreenCamera_<somedate>.json.

Example of using interactive visualizer for you

render = o3d.visualization.Visualizer()
render.create_window()
model, mat = getModel()
render.add_geometry(model)
render.run()

Then if you want to reuse that pinhole camera configuration later on in your offscreen render do this.

parameters = o3d.io.read_pinhole_camera_parameters("ScreenCamera_<somedate>.json")
render.setup_camera(parameters.intrinsic, parameters.extrinsic)

Open3D scene renderer render_to_image returns black screen

1 Answers1