Pandoc Lua Filters: pairs does not show all keys in table

Question

Since Pandoc version 2.0, there has been the ability to write Lua Filters. However, in Pandoc 2.0, I find that using Lua's pairs on an element table does not show all keys in the table.

Here is a minimal example to illustrate the point. In filter.lua I have:

function Para(elem)
  io.stderr:write("A: " .. type(elem) .. "\n")
  for k, v in pairs(elem) do
    io.stderr:write("B: " .. k .. "\n")
  end
  io.stderr:write("C: " .. elem["t"] .. "\n")
  io.stderr:write("D: " .. tostring(elem["c"]) .. "\n")
  -- Return elem unchanged
  return nil
end

Now from the command-line, I run:

echo "Hello." | pandoc -f markdown -t native --lua-filter filter.lua

This produces the output:

A: table
B: c
C: Para
D: table: 0x53adb40
[Para [Str "Hello."]]

I can change the -t native to -t json so that the last line becomes:

{"blocks":[{"t":"Para","c":[{"t":"Str","c":"Hello."}]}],"pandoc-api-version":[1,17,2],"meta":{}}

So from the output at (B), it looks like c is the only key in elem, but from (C) it is clear that t is also a key as I can access it to obtain Para. What is happening here and why is the t key hidden from the loop done with pairs?

score 3 · Accepted Answer · answered Oct 31 '17 at 11:16

The t value is hidden in the element's metatable: instead of assigning a t value to each element, pandoc sets a metatable for each element. The reason for this is to find a balance between usability and performance.

It is quite a bit faster to get numeric table indices back into Haskell than it is to access string-indexed values. However, users should be able to use the element in a straight-forward object oriented way, accessing element components via readable string attributes. This is why we assign metatables to each element. The metatable contains info on the element type (e.g., Plain vs. Para etc.) and defines accessors (e.g., content is aliases to index 0 in Plain and Para elements).

So one can get the type of an element elem by calling elem.t, but the element itself doesn't have that key, the metatable does. That's why t does not show up when iterating over the element using pairs. One can use the getmetatable function to receive the metatable.

You might appreciate the following method to get the name of accessors (undocumented and subject to change).

for k, _ in pairs(getmetatable(elem).getters) do
    print k
end

Alternatively, the section on "Module pandoc" in the lua filters documentation lists the accessors for each element type.

Pandoc Lua Filters: pairs does not show all keys in table

1 Answers1