C++ Lua access violation when table size is above treshold

Question

Recently I became Lua expert in my team due to an issue occurring when we want to send a large table to the following function:

  int native_sl_shootlaserpulse(lua_State* L)
  {
    int iRetVal = 0;

    // L1 is class instance
    luaL_checktype(L, 2, LUA_TTABLE);
    double jumpSpeed = luaL_checknumber(L, 3);
    double settleTime = luaL_checknumber(L, 4);
    unsigned int numberOfPulses = luaL_checknumber(L, 5);
    bool simulateLaserPulse = lua_toboolean(L, 6);

    size_t tableSize = lua_objlen(L, 2);

    std::vector<Scriptey::OffsetXY> scanOffsets(tableSize);
    Scriptey::OffsetXY offset;
    bool scanOffsetsValid(true);

    for (size_t i(1); i < tableSize; ++i)
    {
      lua_rawgeti(L, 2, i + 1); // push the value found in the table at L2, index i+1 on the stack
  
      if (lua_istable(L, -1) && lua_objlen(L, -1) == 2)
      {
        lua_pushnil(L);
  
        // Get x
        lua_next(L, -2);
        offset.x = lua_tonumber(L, -1);
        lua_pop(L, 1);
  
        // Get y
        lua_next(L, -2);
        offset.y =lua_tonumber(L, -1);
        lua_pop(L, 1);
  
        scanOffsets[i] = offset;
      }

      else
      {
        printf(
        "Coordinate in table should have exact two values (x and y), actual size: %d\n",
        lua_objlen(L, -1));
        scanOffsetsValid = false;
      }

       lua_pop(L, 1);
    }

    return iRetVal;
  }

This function is meant to convert the Lua table to a C++ vector of a certain type.

In the Lua script the following works fine where X=50:

scanOffsets = {}

for i=1,X do
    scanOffsets[i] = {0.5, 0.6}
end

sl_shootlaserpulse(scanOffsets, 1.0, 1000, 1, 0)

However, when the size of X is increased to 200, the program gives an access violation error. Of course something goes wrong with the memory management in Lua. But I cannot seem to find the actual cause of the crash. The table gets converted correctly. Only when garbage collection is triggered things seem to fall apart.

I tried increasing the stack size in VS linker but that did not work.

Does anyone had similar experiences using Lua with C++?

Lua seems to run from 1 to X, C++ indexing is 0-based (0 to X-1). Maybe that's why it throws access violation. — kiner_shah, Apr 29 '22 at 07:20
That is true, the 1 in the c++ loop was a typo but is not the source of the issue. Thanks for pointing that out. The actual cause remains a mystery when it is 0. — Bas Brussen, Apr 29 '22 at 07:24
You probably need to do debugging especially w.r.t. vector index. Access violation can happen when you try to access vector out of bounds. — kiner_shah, Apr 29 '22 at 07:26
Probably not the cause of the access violation, but using `lua_next` for the "list" is perhaps undefined behavior (as in, it might return the values in YX order). — Luatic, Apr 29 '22 at 08:31
You should try to run your code under *Address Sanitizer*, or maybe *Valgrind*. — prapin, Apr 29 '22 at 08:36

score 1 · Answer 1 · answered Apr 29 '22 at 11:23

You have Lua C API stack overflow: each iteration of the loop for (size_t i(1); i < tableSize; ++i) pushes one more value to the API stack.

You can make the loop balanced by inserting lua_pop(L, 1); before scanOffsets[i] = offset; to pop the key pushed by the last call to lua_next(L, -2);.

The better approach is to use lua_rawgeti instead of lua_next to get x and y.

There are two other mistakes in the code:

In the loop for (size_t i(1); i < tableSize; ++i) the first element of the array is not being traversed, set initial value of i to 0 to fix it.
The Lua number 0 is treated as true by the function lua_toboolean(L, 6);, probably it is not what you want here.

Thanks for you answer. I did add the rawgeti call instead of next, reads more logical imo. The boolean comment unfortunately is something that is known and is part of the work flow with the Lua interface, so people know to parse 0 in case of true. Tough what fixed the issue was adding ```luaL_checkstack(L, 3, nullptr)``` at the top of the for loop. — Bas Brussen, Apr 29 '22 at 12:28

Bas Brussen · Answer 2 · 2022-05-03T06:23:05.677

Adding the checkstack call at the top of the loop resolved the read access violations:

for (size_t i(0); i < tableSize; ++i)
{
  luaL_checkstack(L, 3, nullptr); // <-- fix
  lua_rawgeti(L, 2, i + 1); // push the value found in the table at L2, index i+1 on the stack
  
  if (lua_istable(L, -1) && lua_objlen(L, -1) == 2)
  {
    lua_pushnil(L);
  
    // Get x
    lua_rawgeti(L, -2, 2);
    scanOffsets[i].x = lua_tonumber(L, -1);
    lua_pop(L, 1);
  
    // Get y
    lua_rawgeti(L, -2, 1);
    scanOffsets[i].y = lua_tonumber(L, -1);
    lua_pop(L, 1);
  }

  lua_pop(L, 1);
}

Edit: the pushnil call was removed from the loop as rawgeti is being used here. This was the reason a stack overflow occurred. Now the checkstack is also not required and the loop is balanced.

C++ Lua access violation when table size is above treshold

2 Answers2