Returning Image Pointer to Erlang

Question

I am trying to use openCV with the Erlang NIF. So I want to do a basic thing and that's just to read a picture and send back the pointer to erlang. and be able to again send back the pointer received to C and just show the pic

so the niftest.cpp look like this:

/* niftest.cpp */

#include "erl_nif.h"
#include <opencv/highgui.h>
#include <opencv/cv.h>
using namespace cv;
using namespace std;


static ErlNifResourceType* frame_res = NULL;


typedef struct _frame_t {
IplImage* _frame;
} frame_t;

//------------------------------------------------------------------------------
// NIF callbacks
//------------------------------------------------------------------------------

static void frame_cleanup(ErlNifEnv* env, void* arg) {
enif_free(arg);
}

static int load(ErlNifEnv* env, void** priv, ERL_NIF_TERM load_info)
{

ErlNifResourceFlags flags = (ErlNifResourceFlags) (ERL_NIF_RT_CREATE | ERL_NIF_RT_TAKEOVER);
frame_res = enif_open_resource_type(env, "niftest", "ocv_frame",
                      &frame_cleanup,
                      flags, 0);
return 0;
}


static ERL_NIF_TERM get_pic(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[])
{

IplImage* src = cvLoadImage("/home/khashayar/Downloads/pic.png");

cout << src->width << endl;

IplImage* gray = cvCreateImage(cvGetSize(src), IPL_DEPTH_8U, 1);
cvCvtColor(src, gray, CV_RGB2GRAY);

frame_t* frame = (frame_t*)enif_alloc_resource(frame_res, sizeof(frame_t));
frame->_frame = gray ;

ERL_NIF_TERM term = enif_make_resource(env, frame);
enif_release_resource(frame);
return enif_make_tuple2(env, enif_make_atom(env, "ok"), term); 

}


static ERL_NIF_TERM show_pic(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[]){

frame_t* frame;
 if (!enif_get_resource(env, argv[0], frame_res, (void**) &frame)) {
   return enif_make_badarg(env);
 }

 cvShowImage("YOOHOO", frame->_frame);

 cvWaitKey(30);

 return enif_make_atom(env, "ok");
}

static ErlNifFunc nif_funcs[] =
  {
    {"show_pic", 1, show_pic},
    {"get_pic", 0, get_pic}
  };

ERL_NIF_INIT(niftest,nif_funcs,load,NULL,NULL,NULL)

and my niftest.erl looks like this :

-module(niftest).

-compile(export_all).

init() ->
      erlang:load_nif("./niftest", 0).

get_pic() ->
      "NIF library not loaded".

show_pic(F) ->
      "NIF library not loaded".

So now the problem is when I call the get_pic what I get in return is {ok, <<>>} and the Pointer is not valid at all.

when I cout the frame before making the enif_make_resource it has a value and I can see it but it returns empty to me!

What I am doing wrong? I have read all the documentation and I really can't figure this out.

NOTE: you can compile the code with this command:

g++ -fPIC -shared -o niftest.so niftest.cpp -lopencv_core -lopencv_imgproc -lopencv_highgui -I /usr/lib64/erlang/usr/include/

and then run the erlang shell and call the init and get_pic function

Answer 1

NIF is the wrong solution to port OpenCV's High-level GUI.

Yet, to answer your question: the apparently empty binary in the {ok, <<>>} tuple you get is opaque from Erlang. This is a resource object as documented in erl_nif manual page.

Resource objects are garbage collector friendly. If no process refers to a given resource, the cleanup function will be called. They are typically the proper structure for embedding C or C++ pointers in your NIF.

IplImage* pointer is a perfect candidate for a resource object. You probably do not need the frame_t type as you can simply cast the pointer to IplImage**. The cleanup function should release memory and, in your example, call cvReleaseImage.

Since the pointer is opaque, you need to port accessor functions to provide data to Erlang. This really depends on the kind of data you would like to extract from the image. For example, you could port the cvEncodeImage function and convert the data from CvMat* to an erlang binary using enif_make_binary.

Also, as a side note, instead of returning the list "NIF library not loaded", you should call erlang:nif_error/1,2 in stub functions.

The proper approach to port an API such as OpenCV's High GUI would be an external driver (or a C-node).

There are several reasons, including:

• NIF calls should return quickly (calling cvWaitKey is a poor candidate for a NIF, as well as computations taking too long), as they would otherwise confuse the scheduler ;
• with a NIF or a linked-in driver, memory management directly impacts Erlang's virtual machine, and any crash will take the whole Erlang node down.

An external driver is a process that gets data from stdin (typically) and replies on stdout. This is very simple to design in C or C++. You could either port OpenCV's API or more complex functions depending on your needs. In this case, pointers such as IplImage* could be transferred as an opaque series of 4 or 8 bytes, or as a reference number provided you maintain a list of all IplImage* pointers Erlang has allocated. Unlike NIFs, however, there is no resource object and you will have to design your Erlang-side code to ensure memory is properly released.

You will find more information and sample code in the Interoperability Tutorial User's Guide.

See also this question: OpenCV on Erlang