Avoiding an extra space when detokenizing a non-range

Question

I have a structure with many "options", each of which, depending on its value, translates into a command-line option - either with or without a value. The order of the command-line options is of no consequence.

The detokenization is not always into a string, and the delimiter may not always be a character (it might actually be some functor object which manipulates the marshalled structure), so this needs to be templated code.

Right now, I have the following function:

template <typename MarshalTarget, typename Delimiter>
void process(const my_options_t& opts, MarshalTarget& marshalled, Delimiter optend)
{
    if (opts.generate_relocatable_code)         { marshalled << "--relocatable-device-code=true" << optend;      }
    if (opts.compile_extensible_whole_program)  { marshalled << "--extensible-whole-program=true" << optend;     }
    if (opts.debug)                             { marshalled << "--device-debug" << optend;                      }
    if (opts.generate_line_info)                { marshalled << "--generate-line-info" << optend;                }
    if (opts.support_128bit_integers)           { marshalled << "--device-int128" << optend;                     }
    if (opts.indicate_function_inlining)        { marshalled << "--optimization-info=inline" << optend;          }
    if (opts.compiler_self_identification)      { marshalled << "--version-ident=true" << optend;                }
    if (not opts.builtin_initializer_list)      { marshalled << "--builtin-initializer-list=false" << optend;    }
    if (opts.specify_language_dialect) {
        marshalled << "--std=" << detail_::cpp_dialect_names[(unsigned) opts.language_dialect] << optend;
    }
    // etc. etc.
}

... but this has a problem: When MarshalTarget is, say, an std::ostream, and Delimiter is a char, I get a command-line fragment which ends in an extra space character. This isn't terrible, but I would rather avoid that extra space.

Would would be a good way to do so?

einpoklum · Answer 1 · 2022-07-22T20:02:53.583

Currently, my thinking is to make two changes:

Use an optstart rather than an optend.
Replace char with a stateful proxy object, which has a boolean. The first time it is streamed to marshalled, don't stream any characters, but set the boolean; and subsequently, do stream a single space character.

template <typename Delimiter>
struct opt_start_t {
    bool      ever_used;
    Delimiter delimiter;

    opt_start_t(Delimiter delimiter) : ever_used(false), delimiter(delimiter){ }
};

template <typename MarshalTarget, typename Delimiter>
MarshalTarget& operator<<(MarshalTarget& mt, opt_start_t<Delimiter>& opt_start)
{
    if (not opt_start.ever_used) {
        opt_start.ever_used = true;
    }
    else {
        mt << opt_start.delimiter;
    }
    return mt;
}

This will work, and will not lengthen the code of the actual marshalling/detokenizing function, but I feel a bit like I'm reinventing the wheel.

score 0 · Answer 2 · answered Jul 22 '22 at 20:14

You could have:

A generic function that builds a vector of items (are these always strings? otherwise, could they be templatized?) from opts,

then does all the marshalled << opt_string << optend stuff without printing the last delimiter, and
finally prints the last delimiter or not based on an input parameter.

Different template specializations that would decide if they want to print the last delimiter or not.

template <typename MarshalTarget, typename Delimiter>
void process(const my_options_t& opts, MarshalTarget& marshalled, Delimiter optend,
    bool print_last_delim = true) {

    std::vector<std::string> opt_strings{};
    if (opts.generate_relocatable_code) {
        opt_strings.push_back("--relocatable-device-code=true");
    }
    // ...

    std::ranges::for_each(opt_strings, [first=true](auto&& opt_string) mutable {
        if (first) {
            first = false;
        } else {
            marshalled << optend;
        }
        marshalled << opt_string;
    });

    if (print_last_delim) {
        marshalled << optend;
    }
}

template <typename MarshalTarget, typename Delimiter>
void process(const my_options_t& opts, MarshalTarget& marshalled, Delimiter optend) {
    process(opts, marshalled, optend, true);
}

template <>
void process(const my_options_t& opts, std::ostream& marshalled, char optend) {
    process(opts, marshalled, optend, false);
}

1. Your solution multiples the length of my code by a factor of 3 at least 2. This would mean an much extra space as the overall length of the final string, plus multiple heap allocations, while my solution is O(1) extra space, and no heap allocation. — einpoklum, Jul 22 '22 at 20:20
Fair enough. I guess the vector of strings could be avoided (maybe with a static map of plus a local vector of indices; but that would add even more complexity to this solution. — rturrado, Jul 22 '22 at 20:30

Avoiding an extra space when detokenizing a non-range

2 Answers2