how to use Tango3DR_updateTexture?

Question

I am trying to make 3DR texturing but it always use only vertex colors in texture.

On every frame I store frame as PNG:

    RGBImage frame(t3dr_image, 4);
    std::ostringstream ss;
    ss << dataset_.c_str();
    ss << "/";
    ss << poses_.size();
    ss << ".png";
    frame.Write(ss.str().c_str());
    poses_.push_back(t3dr_image_pose);
    timestamps_.push_back(t3dr_image.timestamp);

In the method Save I am trying to process texturing:

1) I extract full mesh from context

    Tango3DR_Mesh* mesh = 0;
    Tango3DR_Status ret;
    ret = Tango3DR_extractFullMesh(t3dr_context_, &mesh);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);

2) Create texturing context using extracted mesh

    Tango3DR_ConfigH textureConfig;
    textureConfig = Tango3DR_Config_create(TANGO_3DR_CONFIG_TEXTURING);
    ret = Tango3DR_Config_setDouble(textureConfig, "min_resolution", 0.01);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);
    Tango3DR_TexturingContext context;
    context = Tango3DR_createTexturingContext(textureConfig, dataset.c_str(), mesh);
    if (context == nullptr)
        std::exit(EXIT_SUCCESS);
    Tango3DR_Config_destroy(textureConfig);

3) Call Tango3DR_updateTexture with data I stored before (this does not work)

    for (unsigned int i = 0; i < poses_.size(); i++) {
        std::ostringstream ss;
        ss << dataset_.c_str();
        ss << "/";
        ss << i;
        ss << ".png";
        RGBImage frame(ss.str());
        Tango3DR_ImageBuffer image;
        image.width = frame.GetWidth();
        image.height = frame.GetHeight();
        image.stride = frame.GetWidth() * 3;
        image.timestamp = timestamps_[i];
        //data are for sure in this format
        image.format = TANGO_3DR_HAL_PIXEL_FORMAT_RGB_888;
        image.data = frame.GetData();
        ret = Tango3DR_updateTexture(context, &image, &poses_[i]);
        if (ret != TANGO_3DR_SUCCESS)
            std::exit(EXIT_SUCCESS);
    }

4) Texturize mesh

    ret = Tango3DR_Mesh_destroy(mesh);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);
    mesh = 0;
    ret = Tango3DR_getTexturedMesh(context, &mesh);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);

5) Save it as OBJ (in the result texture are only data from vertex colors, why?)

    ret = Tango3DR_Mesh_saveToObj(mesh, filename.c_str());
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);
    ret = Tango3DR_destroyTexturingContext(context);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);
    ret = Tango3DR_Mesh_destroy(mesh);
    if (ret != TANGO_3DR_SUCCESS)
        std::exit(EXIT_SUCCESS);

All methods returned TANGO_3DR_SUCCESS.

Full code here: https://github.com/lvonasek/tango

Ivan Dryanovski · Accepted Answer · 2017-04-08T00:25:41.537

2

Thanks for reaching out and providing the detailed code breakdown.

The error is on our end - the library currently doesn't support RGB texture inputs. It assumes YUV for all input images. I've opened a ticket to track this bug and we'll fix it for the next release, by allowing RGB input and providing better return values for invalid image formats.

Edit: Found another bug on our end. The API states image_pose should be the pose of the image, but our implementation actually expects the pose of the device. I've opened a bug, and this will be fixed in next release (release-H).

You can try working around this for now by passing in the device pose without multiplying the device-to-camera extrinsic calibration, although of course that's just a temp bandaid.

edited Apr 08 '17 at 00:25

answered Mar 31 '17 at 06:04

Ivan Dryanovski

61
3

even with image.stride = frame.GetWidth() * 2; image.format = TANGO_3DR_HAL_PIXEL_FORMAT_YCrCb_420_SP; image.data = frame.ExtractYUV(); it does not work :( – Luboš V. Mar 31 '17 at 06:32
I found I/tango: mesh_painter.cc:124 Initialized mesh renderer with resolution 1920 x 1080... so I set resolution to fullHD and it still does not work – Luboš V. Mar 31 '17 at 06:48
stride = 2* image_width is not a correct assumption in this case. I think for YUV images stride = width. Could you post your logcat output? – Ivan Dryanovski Mar 31 '17 at 07:02
also - what is the type for "frame"? It could be that it uses a different YUV format than the one the library is using. – Ivan Dryanovski Mar 31 '17 at 07:04
And final comment - the API currently does not support images with a different resolution than the original camera image. Another thing we should patch / make more explicit, apologies. – Ivan Dryanovski Mar 31 '17 at 07:05
frame is using unsigned char* – Luboš V. Mar 31 '17 at 07:17
you can see in the logcat that updateTexture does not log anything (it should be between lines tango_app: Reading /storage/emulated/0/Models/temp/xxx.png) – Luboš V. Mar 31 '17 at 07:23
Thank you for the log. We'll try to reproduce on our end. Most likely, as noted above, it's format inconsistencies. We'll provide an update with the next release of the library. – Ivan Dryanovski Mar 31 '17 at 16:05
I found this problem but solution did not help: - Tango3DR_Context context; + Tango3DR_TexturingContext context; – Luboš V. Apr 04 '17 at 19:04

how to use Tango3DR_updateTexture?

1 Answers1