how to pass command-line arguments as a string to an embedded Python script executed from C++?

Question

I have a C++ program which exposes a Python interface to execute users' embedded Python scripts.

The user inserts the path of the Python script to run and the command-line arguments. Then the script is executed through

boost::python::exec_file(filename, main_globals, main_globals)

To pass the command-line arguments to the Python script we have to set them through the Python C-API function

PySys_SetArgv(int args, char** argv)

before calling exec_file().

But this requires to tokenize the user's string containing the command-line arguments to get the list of arguments, and then to pass them back to the Python interpreter through PySys_SetArgv. And that's more than a mere waste of time, because in this way the main C++ program has to take the responsibility of tokenizing the command-line string without knowing the logics behind, which is only defined in the custom user's script.

A much nicer and cleaner approach would be something like this in metacode:

string command_line_args = '-v -p "filename" -t="anotherfile" --list="["a", "b"]" --myFunnyOpt'
exec_file( filename, command_line_args, ...)

I spent hours looking at the Boost and Python C-API documentation but I did not find anything useful. Do you know if there is a way to achieve this, i.e. passing a whole string of command line arguments to an embedded Python script from C++?

Update:

As Steve suggested in the comments here below, I solved my problem tokenizing the input string, following https://stackoverflow.com/a/8965249/320369.

In my case I used:

// defining the separators
std::string escape_char = "\\"; // the escape character
std::string sep_char = " "; // empty space as separator
std::string quote_char = ""; // empty string --> we don't want a quote char'
boost::escaped_list_separator<char> sep( escape_char, sep_char, quote_char );

because I wanted to be able to parse tuples containing strings as well, like:

'--option-two=("A", "B")'

and if you use:

escaped_list_separator<char> sep('\\', ' ', '"');

as in the original post, you don't get the quoted strings tokenized correctly.

"knowing the logics behind, which is only defined in the custom user's script." - this seems false to me. The Python script *doesn't* contain the logic to tokenize the command line, that's generally done by the shell (or equivalent) that python is launched from. Then the python interpreter will construct `sys.argv` based on what it gets from its own `main` arguments or from what's passed to `PySys_SetArgv`. In this case, you're the equivalent of the shell. Congratulations! — Steve Jessop, May 24 '12 at 15:34
This is probably relevant: http://stackoverflow.com/questions/8964307/c-how-to-break-not-parse-a-string-into-command-line-arguments. Not the Boost.Program_Options part of it, just the bit about splitting a single string prior to processing as options. — Steve Jessop, May 24 '12 at 15:39
I agree with @Steve, you need to token-ize the commands your self and pass them in much the same way the python interpreter does with sys.argv. — secumind, Jun 01 '12 at 02:58
Thanks @Steve, I solved my problem following your suggestion. +1. — rmbianchi, Jun 07 '12 at 17:35

score 1 · Answer 1 · answered Jun 03 '12 at 21:30

Since you are not adverse to executing an external file, you can use a helper program to make your shell command do the parsing for you. Your helper program could be:

#include <stdio.h>
int main (int argc, char *argv[])
{
    for (int i = 1; i < argc; ++i) printf("%s\n", argv[i]);
    return 0;
}

And then you could have code that sends your single string of arguments to the helper program (perhaps using popen) and read back the parsed arguments, each arg on a separate line.

unparsed_line.insert(0, "./parser_helper ");
FILE *helper = popen(unparsed_line.c_str(), "r");
std::vector<std::string> args;
std::vector<const char *> argv;
std::string arg;
while (fgetstring(arg, helper)) {
    args.push_back(arg);
    argv.push_back(args.rbegin()->c_str());
}
pclose(helper);

The fgetstring routine is something I wrote that is like a cross between fgets and std::getline. It reads from the FILE * one line at a time, populating a std:string argument.

static bool
fgetstring (std::string &s, FILE *in)
{
    bool ok = false;
    std::string r;
    char buf[512];
    while (fgets(buf, sizeof(buf), in) != 0) {
        ++ok;
        r += buf;
        if (*r.rbegin() == '\n') {
            r.resize(r.size()-1);
            break;
        }
    }
    if (ok) s = r;
    return ok;
}

I seem to remember a post on SO that had a routine similar to this, but I couldn't find it. I'll update my post if I find it later.

Thanks for your suggestion, and +1 for that. But in the end I solved my problem with this http://stackoverflow.com/a/8965249/320369, as was suggested by @SteveJessop above. — rmbianchi, Jun 07 '12 at 17:34

how to pass command-line arguments as a string to an embedded Python script executed from C++?

1 Answers1