Lua - How to log nested table access/writes without creating a "proxy" index outside of the metatable?

Question

There's a method where you'd be doing something like this to the original table.

function setproxy(t)
  new_t = {proxy = t}
  setmetatable(new_t, mt)
  return new_t
end

However, if you'd be printing out t, you'd get this: t["proxy"].

So I was wondering how to do the nested logging, in this fashion:

local Settings = {}
Settings.example = "Value"

local proxyTable = setmetatable({}, {
    __index = Settings,
    
    __newindex = function(t, k, v)
        print(t, k, v)
        
        Settings[k] = v
    end,
})

proxyTable.example = "value"

With the exception that it supports nesting as well.

It can be a pain to figure out what is going on with this complexity.

So far I had this approach, but I am not sure if it's correct:

local table1 = {
    table2 = {}
}

local function newProxy(tbl)
    local mt = {}
    mt.__index = tbl
    
    mt.__newindex = function(t, k, v)
        print(t, k, v)

        if (type(v) == "table") then
            v = newProxy(v)
        end

        rawset(t,k,v)
    end
    
    local newProxy = setmetatable(tbl, mt)
    return newProxy
end



local proxyTable = setmetatable({},
{
    __index = table1,

    __newindex = function(t, k, v)
        print(t, k, v)

        if (type(v) == "table") then
            v = newProxy(v)
        end

        rawset(t,k,v)
    end,
})


for k,v in pairs(table1) do
    if (type(v) == "table") then
        v = newProxy(v)
    end
    
    rawset(proxyTable, k, v)
end




proxyTable.table2.a = {val = "value"}
proxyTable.table2.a.b = {a="a"}
proxyTable.table2.a.b.c = "hi"

for k,v in pairs(proxyTable) do
    print(k,v)
end

point of the a.val

is to test cases, because when I tried to do this, a would have reset back to {} if I created a table inside of it.

And it was all depending on local newProxy = setmetatable(tbl, mt). If I'd set it to local newProxy = setmetatable({}, mt), I'd reset proxyTable.table2.a = {val = "value"} if I created proxyTable.table2.a.b = {a="a"}

That's my current approach. Maybe it's even the correct way, but I don't know.

What I need to know is, whether my approach is still a proxy, or whether by doing this newProxy = setmetatable(tbl, mt) if I already broke the purpose.

Because I believe I only need one proxy. And seen that proxyTable, is a proxy for table1. I believe it's already done.

The correct way is `local newProxy = setmetatable({}, mt)`. I didn't understand what problem you have (what does "resetting" of `table2.a` mean?). — ESkri, Aug 19 '23 at 07:03
since I wanna keep track of everything within that table, including tables. The only way by doing that, is by creating proxies for the nested table as well. but by doing setmetatable({}, mt) on the nested tables, it would reset whatever they contained before, because it gets turned back into {} — karl-police, Aug 19 '23 at 21:08
`setmetatable({}, mt)` resets nothing: empty table `{}` does not have a metatable. — ESkri, Aug 19 '23 at 23:08
@ESkri If I change ``setmetatable(tbl, mt)`` to ``setmetatable({}, mt)``, the nested table's original value becomes ``{}``, if it contained ``{test="value"}`` before, it would be gone. The first table can use ``setmetatable({}, mt)``, but I am unsure about tables within that table. I believe only the first table needs to act as a proxy. — karl-police, Aug 20 '23 at 17:25
The table get shifted into .__index, using direct print() doesn't return the table. Unless you index in it. — karl-police, Aug 20 '23 at 18:27
In your code, you're calling `newProxy` as a function, but later, you're defining `newProxy` as a table. What is `newProxy` really supposed to be? — luther, Aug 20 '23 at 19:31
@luther That's a shadowed variable. It may be poor semantics, but its technically fine, and does not influence the design (could easily be `return setmetatable(tbl, mt)` instead, however). — Oka, Aug 20 '23 at 19:48
The fact that `newProxy` is being called means we need to see some kind of implementation to make a [Minimal, Reproducible Example](https://stackoverflow.com/help/minimal-reproducible-example) and thus understand the problem. From what I can see `newProxy` looks pretty central to the algorithm, so it must be influencing the design somehow. — luther, Aug 20 '23 at 20:09
@luther The *function* `local function newProxy(tbl)` is clearly defined in the third example, which constitutes the main [mre]. — Oka, Aug 20 '23 at 20:13
Ok. the function didn't show up for me until I did a Ctrl+F. I tried scrolling through everything, but it didn't work. — luther, Aug 20 '23 at 20:20

Oka · Answer 1 · 2023-08-21T00:21:19.843

A proxy needs to be empty, so that all forms of access are intercepted.

In newProxy, setmetatable(tbl, mt) reuses the existing table tbl as its own proxy. This means __newindex will not fire for existing keys in tbl.

You can test this by adding the line

-- ...
proxyTable.table2.a.b.c = "hi"
proxyTable.table2.a.b.c = "will not be logged" -- this does not invoke __newindex

and observing no additional logging.

Using closures is one way to achieve a proxy that does not contain any metadata.

A few changes to your code yields the following. Note that this function does not handle nested tables in its argument (tbl), so the proxy must be built up one level at a time.

local function newProxy(tbl)
    local mt = {}
    mt.__index = tbl

    mt.__newindex = function(t, k, v)
        print(t, k, v)

        if type(v) == "table" then
            v = newProxy(v)
        end

        rawset(tbl, k, v)
    end

    return setmetatable({}, mt)
end

local proxyTable = newProxy {}
proxyTable.table2 = {}
proxyTable.table2.a = { val = "value" }
proxyTable.table2.a.b = { a ="a" }
proxyTable.table2.a.b.c = "hi"
proxyTable.table2.a.b.c = "this will be logged"

-- note that this no longer works, because it iterates on the (empty) proxy
for k,v in pairs(proxyTable) do
    print(k, v)
end

Example output:

table: 0x55b9fc7d3f10   table2  table: 0x55b9fc7d3f50
table: 0x55b9fc7d6240   a   table: 0x55b9fc7d6280
table: 0x55b9fc7d6430   b   table: 0x55b9fc7d6470
table: 0x55b9fc7d6620   c   hi
table: 0x55b9fc7d6620   c   this will be logged

Here is a function that recursively clones tables, and gives each a proxy:

-- Lua 5.4.6, LuaJIT 2.1.0-beta3
-- This function returns a proxy to a clone of its argument
-- It does not handle circular references in its argument
local function Proxy(t)
    local actual = {}
    t = t or {}

    local proxy = setmetatable({}, {
        __metatable = false,
        __name = "Proxy",
        __tostring = function (self)
            return string.format("Proxy: %p -> %p(%p)", self, actual, t)
        end,
        __eq = function (self, other)
            -- create a closure around `t` to emulate shallow equality
            return rawequal(t, other) or rawequal(self, other)
        end,
        __pairs = function (self)
            -- wrap `next` to enable proxy hits during traversal
            return function (tab, key)
                local index, value = next(actual, key)

                return index, value ~= nil and self[index]
            end, self, nil
        end,
        -- these metamethods create closures around `actual`
        __len = function (self)
            return rawlen(actual)
        end,
        __index = function (self, key)
            print("LOG: index", self, key)

            return rawget(actual, key)
        end,
        __newindex = function (self, key, value)
            print("LOG: newindex", self, key, "=", value)

            -- Any new table-values are proxied ...
            rawset(actual, key, type(value) == "table" and Proxy(value) or value)
        end
    })

    -- ... recursively
    for key, value in pairs(t) do
        proxy[key] = value
    end

    return proxy
end

--[[ simple usage --]]
-- create a proxy, add values, and access them
-- every read and write is logged
local p = Proxy()
p.foo = { bar = { 'hello world' } }
print(p.foo.bar[1])
print("proxies are immutable: ", not getmetatable(p))
print(select(2, pcall(setmetatable, p, {})))
--]]

--[[ more info:
-- ... or clone existing tables
local foo = Proxy { bar = { qux = { zun = "hello" } } }

-- proxies of tables
local thing = {}
foo.bar.qux.zarb = thing

-- proxies of proxies
foo.bar.qux.plaz = foo.bar.qux.zarb

-- shallow equality holds true ...
print("shallow equality should be true:", foo.bar.qux.zarb == thing)
-- ... but only through one level of proxy
print("transitive equality should be false:", foo.bar.qux.plaz == thing)

-- ... deep equality does not persist (proxies are clones)
thing.x = 51
print("deep equality should be false:", foo.bar.qux.zarb.x == thing.x)

-- length operators and iterators work, `ipairs` triggers `__index` for `#t + 1`
local abc = Proxy { 'a', 'b', 'c' }
print(abc, "has a length of", #abc)

for k, v in ipairs(abc) do
    print(string.format("abc[%s] = %q", k, v))
end
--]]

Example output:

LOG: newindex   Proxy: 0x556701bbde90 -> 0x556701bbde10(0x556701bbde50) foo =   table: 0x556701bbe0c0
LOG: newindex   Proxy: 0x556701bbe7d0 -> 0x556701bbe790(0x556701bbe0c0) bar =   table: 0x556701bbe100
LOG: newindex   Proxy: 0x556701bbebb0 -> 0x556701bbeb70(0x556701bbe100) 1   =   hello world
LOG: index  Proxy: 0x556701bbde90 -> 0x556701bbde10(0x556701bbde50) foo
LOG: index  Proxy: 0x556701bbe7d0 -> 0x556701bbe790(0x556701bbe0c0) bar
LOG: index  Proxy: 0x556701bbebb0 -> 0x556701bbeb70(0x556701bbe100) 1
hello world
proxies are immutable:  true
cannot change a protected metatable

Lua - How to log nested table access/writes without creating a "proxy" index outside of the metatable?

1 Answers1