Display to Raspberry Pi HDMI From Raspberry Pi Camera using C++

Question

Question: How do I write video stream from Raspberry Pi camera to HDMI connected display?

Answer: see below

Answer 1

My system: Raspberry Pi3, Raspbian Jessie distribution VisualGDB and VisualStudio 2015

Spending a few hours putting together various test code from different posts, I'm posting this compiled test code for the community to use. It works ~ 15fps with 1200x720p color video output.

Sources: https://raspberrypi.stackexchange.com/questions/7092/display-a-test-pattern-by-writing-the-frame-buffer-and-then-clear-it

https://visualgdb.com/tutorials/raspberry/camera/

NOTES: VisualGDB has some issues with Sysroot syncing, when I ran the built-in operation from Visual Studio, it copied a bunch of headers with size 0. Instead, I manually copied the /opt/vc folder to C:\SysGCC\raspberry\arm-linux-gnueabihf\sysroot directly

I also had to copy libbcm_host.so (along with libraspicam.so.0.1 as in the linked tutorial) from /opt/vc/lib to local build/Debug/src folder,

And add =/opt/vc/include to includes (= indicates local sysroom directory) And bcm_host to library names.

NOTE2: Either Raspbian distro or the compiler do not like while(1) loops, so if you replace the for(...) loop for drawing frames with a while(1) without any exit conditions, you will get a black screen for output. Probable cause is the optimizer, good practice is to just avoid having infinite loops without an exit condition altogether.

NOTE3: If using HDMI output to monitor with less than 1280x720 resolution (less than camera video stream), some issues may occur, looking for someone to edit the code for smaller screens. Older versions of RPi with Composite video output are also not tested.

Thanks,

#include <stdio.h>
#include <syslog.h>
#include <fcntl.h>
#include <linux/fb.h>
#include <sys/mman.h>
#include <sys/ioctl.h>
#include "interface\vmcs_host\vc_dispmanx_types.h"
#include <bcm_host.h>
#include "raspicam.h"
#include <iostream>


typedef struct
{
    DISPMANX_DISPLAY_HANDLE_T   display;
    DISPMANX_MODEINFO_T         info;
    void                        *image;
    DISPMANX_UPDATE_HANDLE_T    update;
    DISPMANX_RESOURCE_HANDLE_T  resource;
    DISPMANX_ELEMENT_HANDLE_T   element;
    uint32_t                    vc_image_ptr;

} RECT_VARS_T;

int main(int argc, char **argv) 
{
    RECT_VARS_T vars;
    VC_RECT_T src_rect;
    VC_RECT_T dst_rect;
    VC_DISPMANX_ALPHA_T alpha = { 
        static_cast<DISPMANX_FLAGS_ALPHA_T>(DISPMANX_FLAGS_ALPHA_FROM_SOURCE | DISPMANX_FLAGS_ALPHA_FIXED_ALL_PIXELS),
        255, /*alpha 0->255*/
        0
    };

    bcm_host_init();

    vars.display = vc_dispmanx_display_open(0);
    vc_dispmanx_display_get_info(vars.display, &vars.info);

    //Grab Camera feed
    raspicam::RaspiCam v_camera; //camera object
    //Open camera 
    std::cout << "Opening Camera..." << std::endl;
    if (!v_camera.open()) 
    {
        std::cerr << "Error opening camera" << std::endl; 
        return -1;
    }
    //may need to wait a while until camera stabilizes

    int cam_width = v_camera.getWidth();
    int cam_height = v_camera.getHeight();

    vars.image = calloc(1, cam_width * cam_height * 3);
    vars.resource = vc_dispmanx_resource_create( VC_IMAGE_RGB888,
        cam_width,
        cam_height,
        &vars.vc_image_ptr);

    vc_dispmanx_rect_set(&dst_rect, 0, 0, cam_width, cam_height);
    vars.update = vc_dispmanx_update_start(10);
    vars.element = vc_dispmanx_element_add( vars.update,
        vars.display,
        2000, // layer
        &dst_rect,
        vars.resource,
        &src_rect, //may not need this
        DISPMANX_PROTECTION_NONE,
        &alpha,
        NULL, // clamp
        static_cast<DISPMANX_TRANSFORM_T>(0));

    //Draw 50 frames to screen
    for (int i = 0; i < 50; i++)
    {    
        vc_dispmanx_resource_write_data( vars.resource,
            VC_IMAGE_RGB888,
            cam_width * 3,
            vars.image,
            &dst_rect);

        unsigned char* fbp = static_cast<unsigned char*>(vars.image);   
        v_camera.grab();
        v_camera.retrieve(fbp, raspicam::RASPICAM_FORMAT_RGB);//get camera image
        vc_dispmanx_update_submit_sync(vars.update); 
    }

    int ret = vc_dispmanx_resource_delete(vars.resource);
    vc_dispmanx_display_close(vars.display);

    return true;
}

If you want to perform any image processing, just write directly using fbp pointer inside the loop, for example (not the most efficient method)

int location_cam = 0;
for (int x = 200; x < 300; x++)
{       
    for (int y = 200; y < 300; y++)
    {
        location_cam = (x) * (3) + (y) * cam_width * 3;
        *(fbp + location_cam) = 255;        //red
        *(fbp + location_cam + 1) = 0;     //green
        *(fbp + location_cam + 2) = 0;    //blue
    }
}