Weird lines on camera feed with Opencv + Irrlicht on Texture Mapping

Question

I've Been trying to texturemap Irrlicht 3d model with Opencv Camera for an augmented reality project, Im using irrlicht 1.8.1 and opencv 2.4.9, and i almost got the textureMapping done, but the problem is there are weird lines coming in the camera feed and its below 4fps and the output appears as in this screenshot (Below) :

enter image description here

I Have the Following code that does this:

 #include <irrlicht.h>
 #include "driverChoice.h"
 #include <opencv/cv.h>
 #include <opencv/cxcore.h>
 #include <opencv/highgui.h>
 #include <time.h>
 #include <sys/timeb.h> 
 #include <opencv2\highgui\highgui.hpp>
 #include <IGUIEnvironment.h>

 using namespace cv;
 using namespace irr;
 using namespace core;
 using namespace scene;
 using namespace video;
 using namespace io;
 using namespace gui;

 #ifdef _MSC_VER
 #pragma comment(lib, "Irrlicht.lib")
 #endif

 #ifdef _IRR_WINDOWS_
 #pragma comment(lib, "Irrlicht.lib")
 #pragma comment(linker, "/subsystem:windows /ENTRY:mainCRTStartup")
 #endif


 int main()
 {
 IrrlichtDevice *device = createDevice(video::EDT_OPENGL, dimension2d<s32>     (640 , 480), 16, false, true /* shadow */, false);
if (!device)
    return 1;
device->setWindowCaption(L"Hello World! - Irrlicht Engine Demo");
IVideoDriver* driver = device->getVideoDriver();
ISceneManager* smgr = device->getSceneManager();
IGUIEnvironment* guienv = device->getGUIEnvironment();
ITexture* frame_tex = driver->addTexture(vector2d<s32>(640, 480), "video_stream");
guienv->addStaticText(L"Hello World! This is the Irrlicht Software renderer!",
    rect<s32>(10,10,260,22), true);
IAnimatedMesh* mesh = smgr->getMesh("E:/Akshay/VS2012Projects/IrrlichtDemoApp/IrrlichtDemoApp/ratamahatta.md2");
if (!mesh)
{
    device->drop();
    return 1;
}
IAnimatedMeshSceneNode* node = smgr->addAnimatedMeshSceneNode( mesh );
if (node)
{
    node->setMaterialFlag(EMF_LIGHTING, false);
    node->setMD2Animation(scene::EMAT_STAND);
    node->setMaterialTexture( 0, driver->getTexture("E:/Akshay/VS2012Projects/IrrlichtDemoApp/IrrlichtDemoApp/ctf_r.png") );
}   
smgr->addCameraSceneNode(0, vector3df(20,30,-50), vector3df(0,5,0));
while(device->run())
{
VideoCapture capture(0);
Mat camera_frame;


if( cv::waitKey(50) >= 0 ) break;
if( !capture.grab() )
{
    std::cout << "Unable to grab camera frame\n";
}
capture >> camera_frame;

if( ! camera_frame.empty() )
{
    //exception here
    unsigned char *tex_buf = (unsigned char*)frame_tex->lock();
    unsigned char *frame_buf = camera_frame.data;
    // Convert from RGB to RGBA
    for(int y=0; y < camera_frame.rows; y++) {
        for(int x=0; x < camera_frame.cols; x++) {
            *(tex_buf++) = *(frame_buf++);
            *(tex_buf++) = *(frame_buf++);
            *(tex_buf++) = *(frame_buf++);
            *(tex_buf++) = 255;
        }
    }
    frame_tex->unlock();

    driver->beginScene(true, true, SColor(255,100,101,140));

    driver->draw2DImage(frame_tex, core::rect<s32>(0,0,640,480),
        core::rect<s32>(0,0,480,640));

    smgr->drawAll();
    guienv->drawAll();

    driver->endScene();
}
}

device->drop();
}

Any help would be greatly appreciated!

Thanks in advance!

It looks like your stride is incorrect, is the video coming in as YUV or RGB, also what is the bit-depth and packing? — EdChum, May 08 '15 at 10:04
unrelated, but you should not open a new VideoCapture for each frrame retrieved. (keep one around for the whole lifetime of your prog) — berak, May 08 '15 at 10:12
@EdChum Im not much aware of the bit-depth and packing, and video is coming in as RGB, and im retrieving frames from webcam — Akshay Pramoth, May 08 '15 at 10:16
@berak Could you help me, what to move up and where ? please! — Akshay Pramoth, May 08 '15 at 10:44
`VideoCaptue capture(0);` just goes 2 lines up, before `while(device->run())` — berak, May 08 '15 at 10:47
@berak Thanks a Lot Berak! It solved the fps issue! but the weird lines still appears the same! — Akshay Pramoth, May 08 '15 at 10:51

score 0 · Accepted Answer · answered May 12 '15 at 06:52

0

I Solved the issue, The problem was with my laptop, i tried doing this in another pc and its working fine. Its because of the s32 , i changed it to u32 and it works fine now. thanks all for your help!

answered May 12 '15 at 06:52

Akshay Pramoth

11
3

Weird lines on camera feed with Opencv + Irrlicht on Texture Mapping

1 Answers1