create lightuserdata in lua for luaglut glReadPixels

Question

I'm using luaglut and when I try to use the glReadPixels to capture a frame, I cannot prepare the last input argument for it successfully.

This is how I call the function:

glReadPixels(0, 0, 250, 250, GL_RGB, GL_UNSIGNED_BYTE, img)

The img is where I want to store the frame, but no matter in which type I define img, there's always error. The compiler always wants lightuserdata, but I've searching for several days and seems there is no way to create this type of data within lua.

How should I deal with this problem? Thank you in advance!

[`lua_pushlightuserdata`](http://www.lua.org/manual/5.1/manual.html#lua_pushlightuserdata) though it would probably be more useful to create a full userdata, which can have metatables and garbage collection. — Colonel Thirty Two, Nov 10 '15 at 02:02
@EtanReisner Hi, maybe it should be exactly what I need. So I looked into memarray.c, it is defined as "typedef struct memarray { size_t size; size_t length; unsigned int flags; void *data; } memarray_t;" So I did the following: local img = memarray('uchar', 250 * 250 * 3); glReadPixels(0, 0, 250, 250, GL_RGB, GL_UNSIGNED_BYTE, img.data); But now I get the error: bad argument #7 to 'glReadPixels' (lightuserdata expected, got nil) Any idea how to fix it? — user1514071, Nov 10 '15 at 09:53
@ColonelThirtyTwo Hi, thanks for the reply, but lua_pushlightuserdata is on the C side right? I tried creating a function in C which push a lightuserdata as a return value, then I call this function in lua. But when I print out the type of the return value, it shows it's userdata again ... Do you know how to fix it? Thanks! — user1514071, Nov 10 '15 at 09:59
lightuserdata and userdata both report "userdata" when run through `tostring`. The distinction is meaningless to the lua side of things so it doesn't show it to you. — Etan Reisner, Nov 10 '15 at 13:44
@ColonelThirtyTwo Hi, I'm really new to lua and only now I see what you mean... although I get lightuserdata in lua, I can do nothing with it... can you give me any suggestions on how should I implement this full userdata for my purpose? I have no clue... thanks! — user1514071, Nov 10 '15 at 16:46

user1514071 · Answer 1 · 2015-11-10T15:09:31.010

1st Update: So I managed to get lightuserdata into lua, but now I get Segmentation fault (core dumped) problem. Below is how my code looks right now:

//"cpp_test.cpp"

extern "C" {
    #include <lua5.1/lua.h>
    #include <lua5.1/lualib.h>
    #include <lua5.1/lauxlib.h>
}

int getLightUserData_func_cpp(lua_State* L){
    int nargc = lua_gettop(L);
    if (0 != nargc){
        luaL_error(L, "Wrong number of arguments.");
        return 0;
    }
    static char var_name = 'a';
    lua_pushlightuserdata(L, (void*)&var_name);
    return 1;
}

static const struct luaL_reg lib_cpp[] = {
    {"getLightUserData_func_lua", getLightUserData_func_cpp},
    {NULL, NULL}
};

extern "C"
int luaopen_test(lua_State *L){
    luaL_register(L, "lib_lua", lib_cpp);
    return 1;
}

Makefile:

g++ -o test.so ./cpp_test.cpp  -I/usr/include/lua5.1 -llua5.1 -shared -fpic

Then when I use test.so in lua:

require "luagl"
require "luaglut"
require "memarray"

require "test"

print(lib_lua)
a = lib_lua.getLightUserData_func_lua()
glReadPixels(0, 0, 250, 250, GL_RGB, GL_UNSIGNED_BYTE, a)
print(type(a))
print(a)

This is the output I get:

{
  getLightUserData_func_lua : function: 0x41ecbf10
}
userdata    
userdata: 0x7fe73297f050

So it seems to work. But when I require("test") in my other code, like (I only put the relevant part here):

-- "main.lua"
require "luagl"
require "luaglut"
require "memarray"

require "test"

HEIGHT = 250
WIDTH = 250

function captureFrame()
    color = lib_lua.getLightUserData_func_lua()
    glReadPixels(0, 0, HEIGHT, WIDTH, GL_RGB, GL_UNSIGNED_BYTE, color)
end

-- main loop
glutInit(arg)
glutInitDisplayMode(GLUT_RGB + GLUT_DOUBLE + GLUT_DEPTH)
if arg then title = arg[0] else title = "glut" end
glutInitWindowPosition(710, 0)
glutInitWindowSize(HEIGHT, WIDTH)
window = glutCreateWindow(title)
glutDisplayFunc(display_func)
glutReshapeFunc(reshape_func)

local terminal = false
while not terminal do
    io.read()
    glutMainLoopEvent()
    captureFrame()
end

I get Segmentation fault (core dumped) when captureFrame() is called. Can anyone help me with it? Thanks a lot!

2nd Update: OK, I found the bug, the only think to change is in cpp_test.cpp. NOTE that the old line is commented off:

int getLightUserData_func_cpp(lua_State* L){
    int nargc = lua_gettop(L);
    if (0 != nargc){
        luaL_error(L, "Wrong number of arguments.");
        return 0;
    }
    // static char var_name = 'a';
    static unsigned char var_name[3][255][255] = {{{0}}};
    lua_pushlightuserdata(L, (void*)&var_name);
    return 1;
}

This way the pointer type matches the image data that it will point to. Thanks a lot guys for all the suggestions :D

@EtanReisner And do you think it is caused by the method I created the `lightuserdata` or anything? Or is it better to use `memarray`? I didn't manage to use `memarray` successfully with `glReadPixels`, can you tell me more specifically how to use it? — user1514071, Nov 10 '15 at 14:29

create lightuserdata in lua for luaglut glReadPixels

1 Answers1

Linked