Variadic macros to create multiple constructs with delimiters

Question

EDIT2: Figured it out. See my answer below.

Some background:
I am using an SQL library that returns query results as tuples. For each DB statement I write:

An SQL query that includes a list of field names
A tuple to receive the results of each row
an enum with symbolic constants to use with std::get<> to access the individual fields.

This is an error prone process (especially if that I'd like to automate using a macro, such that for the following table and statement:

CREATE TABLE data (
  id integer NOT NULL GENERATED ALWAYS AS IDENTITY,
  field1 integer NOT NULL,
  field2 bigint NOT NULL,
  field3 varchar(100) NOT NULL
  field4 smallint);

SELECT field1, field2, field3, field4
FROM data WHERE id = $1

I could write something like:

GENERATE_QUERY(
  (FIELD1, int32_t),
  (FIELD2, int64_t),
  (FIELD3, std::string),
  (FIELD4, std::optional<int16_t>)
)

To have the preprocessor generate something like this:

using RowType = std::tuple<int32_t, int64_t, std::string, std::optional<int16_t>>;
enum FIELDS { FIELD1, FIELD2, FIELD3, FIELD4 };
static constexpr auto fieldList = "FIELD1, FIELD2, FIELD3, FIELD4";
// or the equivalent "FIELD1" "," "FIELD2" "," "FIELD3" "," "FIELD4"

And then, this will work:

auto query = "select "s + fieldList + " from data where id = $1";
auto row = exec(query, i).asTuple<RowType>()
do_something_with(std::get<FILED3>(row));

I tried to use the FOR_EACH macro from David Mazières's blog:

#define PARENS ()

#define EXPAND(arg) EXPAND1(EXPAND1(EXPAND1(EXPAND1(arg))))
#define EXPAND1(arg) EXPAND2(EXPAND2(EXPAND2(EXPAND2(arg))))
#define EXPAND2(arg) EXPAND3(EXPAND3(EXPAND3(EXPAND3(arg))))
#define EXPAND3(arg) EXPAND4(EXPAND4(EXPAND4(EXPAND4(arg))))
#define EXPAND4(arg) arg

#define FOR_EACH(macro, ...) \
  __VA_OPT__(EXPAND(FOR_EACH_HELPER(macro, __VA_ARGS__)))
  
#define FOR_EACH_HELPER(macro, arg, ...) \
  macro(arg) __VA_OPT__(FOR_EACH_AGAIN PARENS (macro, __VA_ARGS__))

#define FOR_EACH_AGAIN() FOR_EACH_HELPER

and the following got me close:

#define ARG1_(arg, ...) arg
#define ARG1(arg) ARG1_ arg

#define ARG2_(_1, arg, ...) arg
#define ARG2(arg) ARG2_ arg

#define ARG_1_STR(arg, ...) #arg
#define ARG1_STR(arg) ARG_1_STR arg

#define GENERATE_QUERY(...) \
    enum Columns { FOR_EACH(ARG1, __VA_ARGS__) }; \
    using Tuple = std::tuple<FOR_EACH(ARG2, __VA_ARGS__)>; \
    static constexpr auto query = FOR_EACH(ARG1_STR, __VA_ARGS__);

The result this gets me is:

enum Columns { FIELD1 FIELD2 FIELD3 FIELD4 };
using Tuple = std::tuple<int32_t int64_t std::string std::optional<int16_t> >;
static constexpr auto query = "FIELD1" "FIELD2" "FIELD3" "FIELD4";

The only thing missing is the commas. Unfortunately I was not able to modify the macros to accept a delimiter to be emitted between the elements.

So my question is:
How do I modify the FOR_EACH macros to add a user-specified delimiter (, or "," in this case) to insert between the elements? Or failing that, what construct should I use instead?

(Generalizing the ARGn macros would be nice but it's secondary).

EDIT: A failed attempt:
The following modifications did not work:

#define FOR_EACH(macro, delim, ...) \
    __VA_OPT__(EXPAND(FOR_EACH_HELPER(macro, delim, __VA_ARGS__)))
  
#define FOR_EACH_HELPER(macro, delim, arg, ...) \
    macro(arg) __VA_OPT__( delim FOR_EACH_AGAIN PARENS (macro, delim, __VA_ARGS__))

#define COMMA() ,
#define COMMA_STR() ","

#define GENERATE_QUERY(...) \
    enum Columns { FOR_EACH(ARG1, COMMA, __VA_ARGS__) }; \
    using Tuple = std::tuple<FOR_EACH(ARG2, COMMA, __VA_ARGS__)>; \
    static constexpr auto query = FOR_EACH(ARG1_STR, COMMA_STR, __VA_ARGS__);

It inserted the unexpanded COMMA and COMMA_STR between the arguments. Almost...

RandomBits · Answer 1 · 2023-05-24T03:40:23.883

I would take a look at this GitHub repo and the corresponding documentation. It provides functional oriented style macro programming.

A snippet from the documentation intro:

If you find youself using the preprocessor to generate code, just don’t. Take a walk in the woods and clear you head. Re-evaluate the problem and come up with a better option. Maybe use a code-generation tool designed for the task at hand. Using the preprocessor should be a last resort. But, for those times when using the preprocessor is the lessor of the evils, can we all agree to use it in a structured fashion

Here are some examples from tests:

#define CORE_PP_TEST_ITEM(a) Tensor ## a ## n<uint,a>;
TEST(PP, MAP)
{
    EXPECT_EQ(CORE_PP_STRINGIZE(CORE_PP_EVAL_MAP(CORE_PP_TEST_ITEM, 1, 2, 3)),
              "Tensor1n<uint,1>; Tensor2n<uint,2>; Tensor3n<uint,3>;");
}
#undef CORE_PP_TEST_ITEM


TEST(PP, MAP_SEQ)
{
    EXPECT_EQ(CORE_PP_STRINGIZE(CORE_PP_EVAL_MAP_SEQ(CORE_PP_HEAD_SEQ, ((1,2), (2,3)))), "1 2");
}

#define CORE_PP_TEST_ITEM(a,b) Tensor ## b ## n<a,b>;
TEST(PP, MAPN)
{
    EXPECT_EQ(CORE_PP_STRINGIZE(CORE_PP_EVAL_MAPN(CORE_PP_TEST_ITEM, (uint,1), (uint,2), (int,1))),
              "Tensor1n<uint,1>; Tensor2n<uint,2>; Tensor1n<int,1>;");
}
#undef CORE_PP_TEST_ITEM

TEST(PP, MAPN_SEQ)
{
    EXPECT_EQ(CORE_PP_STRINGIZE(CORE_PP_EVAL_MAPN_SEQ(int, (a;, b;, c;))), "int a; int b; int c;");
}

#define CORE_PP_TEST_ITEM(x) x x
TEST(PP, MAP_INFIX)
{
    EXPECT_EQ(CORE_PP_STRINGIZE(CORE_PP_EVAL_MAP_INFIX(CORE_PP_TEST_ITEM, CORE_PP_COMMA, a, b, c)),
              "a a , b b , c c");
}
#undef CORE_PP_TEST_ITEM

TEST(PP, MAP_INFIX_SEQ)
{
    EXPECT_EQ(CORE_PP_STRINGIZE(CORE_PP_EVAL_MAP_INFIX_SEQ(CORE_PP_SECOND_SEQ, CORE_PP_COMMA,
                                                           ((a, 1), (b, 2), (c, 3)))),
              "1 , 2 , 3");
}

#define CORE_PP_TEST_ITEM(name, value) name::value,
TEST(PP, MAP_WITH)
{
    EXPECT_EQ(CORE_PP_STRINGIZE(CORE_PP_EVAL_MAP_WITH(CORE_PP_TEST_ITEM, Foo, a, b, c)),
              "Foo::a, Foo::b, Foo::c,");
}
#undef CORE_PP_TEST_ITEM

#define CORE_PP_TEST_ITEM(name, seq) name::CORE_PP_SECOND_SEQ(seq),
TEST(PP, MAP_WITH_SEQ)
{
    EXPECT_EQ(CORE_PP_STRINGIZE(CORE_PP_EVAL_MAP_WITH_SEQ(CORE_PP_TEST_ITEM, Foo,
                                                          ((1, a), (2, b), (3, c)))),
              "Foo::a, Foo::b, Foo::c,");
}
#undef CORE_PP_TEST_ITEM

TEST(PP, CARTESIAN_PRODUCT)
{
    EXPECT_EQ(CORE_PP_STRINGIZE(CORE_PP_EVAL_CARTESIAN_PRODUCT_SEQ((a,b,c), (1,2,3))),
              "( (a,1), (a,2), (a,3), (b,1), (b,2), (b,3), (c,1), (c,2), (c,3), )");

}

Update

I believe the following produces the output requested in the question. The code is even readable considering it is macro programming.

Source Code

#include "core/pp/pp.h"

#define FIELD_LIST ((FIELD1, int32_t),                  \
                    (FIELD2, int64_t),                  \
                    (FIELD3, std::string),              \
                    (FIELD4, std::optional<int16_t>))

#define SELECT_FIELD(x) CORE_PP_HEAD_SEQ(x)

#define SELECT_TYPE(x) CORE_PP_SECOND_SEQ(x)

#define TUPLE(N,S) using N = std::tuple<CORE_PP_EVAL_MAP_INFIX_SEQ(SELECT_TYPE, CORE_PP_COMMA, S)>

#define ENUM(N,S) enum N { CORE_PP_EVAL_MAP_INFIX_SEQ(SELECT_FIELD, CORE_PP_COMMA, S) }

#define LIST(N,S) static constexpr auto N =                             \
        CORE_PP_STRINGIZE(CORE_PP_EVAL_MAP_INFIX_SEQ(SELECT_FIELD, CORE_PP_COMMA, S))

TUPLE(RowType, FIELD_LIST);
ENUM(Fields, FIELD_LIST);
LIST(FieldList, FIELD_LIST);

Preprocessed Code

// Lots of prior output...

using RowType = std::tuple<int32_t , int64_t , std::string , std::optional<int16_t> >;
enum Fields { FIELD1 , FIELD2 , FIELD3 , FIELD4 };
static constexpr auto FieldList = "FIELD1 , FIELD2 , FIELD3 , FIELD4";

int tool_main(int argc, const char *argv[]) {
    return 0;
}

Good answer. I ended up not using it, but the library looks very helpful. — Alex O, May 26 '23 at 01:10

score 0 · Answer 2 · answered May 24 '23 at 03:45

Managed to get it working.

The problem turned out to be specific to my case: it worked with every delimiter except a comma, because a comma has a special meaning in macros. Changing the EXPAND macro to be variadic took care of it.

Here's the final version:

#define PARENS ()

#define EVAL(...)  EVAL1(EVAL1(EVAL1(EVAL1(__VA_ARGS__))))
#define EVAL1(...) EVAL2(EVAL2(EVAL2(EVAL2(__VA_ARGS__))))
#define EVAL2(...) EVAL3(EVAL3(EVAL3(EVAL3(__VA_ARGS__))))
#define EVAL3(...) EVAL4(EVAL4(EVAL4(EVAL4(__VA_ARGS__))))
#define EVAL4(...) __VA_ARGS__

#define FOR_EACH(macro, delim, ...) \
    __VA_OPT__(EVAL(FOR_EACH_IMPL(macro, delim, __VA_ARGS__)))

#define FOR_EACH_IMPL(macro, delim, arg, ...) \
    macro(arg)__VA_OPT__( delim PARENS FOR_EACH_AGAIN PARENS (macro, delim, __VA_ARGS__))

#define FOR_EACH_AGAIN() FOR_EACH_IMPL

#define ARG1_(arg, ...) arg
#define ARG1(arg) ARG1_ arg

#define ARG2_(_1, arg, ...) arg
#define ARG2(arg) ARG2_ arg

#define ARG1_STR_(arg, ...) #arg
#define ARG1_STR(arg) ARG1_STR_ arg

#define COMMA() ,
#define COMMA_STR() ", "

#define GENERATE_QUERY(query, prefix, suffix, names, types, ...) \
    static constexpr auto query = prefix " " FOR_EACH(ARG1_STR, COMMA_STR, __VA_ARGS__) " " suffix; \
    using types = std::tuple<FOR_EACH(ARG2, COMMA, __VA_ARGS__)>; \
    enum names { FOR_EACH(ARG1, COMMA, __VA_ARGS__) };

The ARG* macros can probably be improved, but that's a different story.

Variadic macros to create multiple constructs with delimiters

2 Answers2

Source Code

Preprocessed Code