Dynamic arg types for a python function when embedding

Question

I am adding to Exim an embedded python interpreter. I have copied the embedded perl interface and expect python to work the same as the long-since-coded embedded perl interpreter. The goal is to allow the sysadmin to do complex functions in a powerful scripting language (i.e. python) instead of trying to use exim's standard ACL commands because it can get quite complex to do relatively simple things using the exim ACL language.

My current code as of the time of this writing is located at http://git.exim.org/users/tlyons/exim.git/blob/9b2c5e1427d3861a2154bba04ac9b1f2420908f7:/src/src/python.c . It is working properly in that it can import the sysadmin's custom python code, call functions in it, and handle the returned values (simple return types only: int, float, or string). However, it does not yet handle values that are passed to a python function, which is where my question begins.

Python seems to require that any args I pass to the embedded python function be explicitly cast to one of int,long,double,float or string using the c api. The problem is the sysadmin can put anything in that embedded python code and in the c side of things in exim, I won't know what those variable types are. I know that python is dynamically typed so I was hoping to maintain that compliance when passing values to the embedded code. But it's not working that way in my testing.

Using the following basic super-simple python code:

def dumb_add(a,b):
        return a+b

...and the calling code from my exim ACL language is:

${python {dumb_add}{800}{100}}

In my c code below, reference counting is omitted for brevity. count is the number of args I'm passing:

pArgs = PyTuple_New(count);
for (i=0; i<count; ++i)
{
  pValue = PyString_FromString((const char *)arg[i]);
  PyTuple_SetItem(pArgs, i, pValue);
}
pReturn = PyObject_CallObject(pFunc, pArgs);

Yes, **arg is a pointer to an array of strings (two strings in this simple case). The problem is that the two values are treated as strings in the python code, so the result of that c code executing the embedded python is:

${python {dumb_add}{800}{100}}

800100

If I change the python to be:

def dumb_add(a,b):
        return int(a)+int(b)

Then the result of that c code executing the python code is as expected:

${python {dumb_add}{800}{100}}

900

My goal is that I don't want to force a python user to manually cast all of the numeric parameters they pass to an embedded python function. Instead of PyString_FromString(), if there was a PyDynamicType_FromString(), I would be ecstatic. Exim's embedded perl parses the args and does the casting automatically, I was hoping for the same from the embedded python. Can anybody suggest if python can do this arg parsing to provide the dynamic typing I was expecting?

Or if I want to maintain that dynamic typing, is my only option going to be for me to parse each arg and guess at the type to cast it to? I was really really REALLY hoping to avoid that approach. If it comes to that, I may just document "All parameters passed are strings, so if you are actually trying to pass numbers, you must cast all parameters with int(), float(), double(), or long()". However, and there is always a comma after however, I feel that approach will sour strong python coders on my implementation. I want to avoid that too.

Any and all suggestions are appreciated, aside from "make your app into a python module".

Have you tried using [PyRun_String](http://docs.python.org/2/c-api/veryhigh.html#PyRun_StringFlags) instead of PyString_FromString? — Markku K., May 21 '13 at 19:25
PyRun_String might be sufficient if it was only ever one function being called. But the expectation is that the sysadmin can define 1, 5, 20 functions, however many (s)he needs, and call each of them for various purposes at any time from the Exim ACLs. — Todd Lyons, May 21 '13 at 19:41
My comment was only meant to address dynamic typing. So, in your `dumb_add` example, 800 and 100 would be converted to PyObjects using something like `pValue = PyRun_String((const char *)arg[i],...);`. In this case, arg[i] would be "800", or "600". So, you are "running" the string "800", and it should return a PyObject of integer type. Does that make sense? — Markku K., May 21 '13 at 20:28

score 0 · Answer 1 · answered Sep 11 '13 at 13:49

The way I ended up solving this was by finding out how many args the function expected, and exit with an error if the number of args passed to the function didn't match. Rather than try and synthesize missing args or to simply omit extra args, for my use case I felt it was best to enforce matching arg counts.

The args are passed to this function as an unsigned char ** arg:

  int count = 0;
  /* Identify and call appropriate function */
  pFunc = PyObject_GetAttrString(pModule, (const char *) name);
  if (pFunc && PyCallable_Check(pFunc))
  {
    PyCodeObject *pFuncCode = (PyCodeObject *)PyFunction_GET_CODE(pFunc);
    /* Should not fail if pFunc succeeded, but check to be thorough */
    if (!pFuncCode)
    {
      *errstrp = string_sprintf("Can't check function arg count for %s",
                                name);
      return NULL;
    }
    while(arg[count])
      count++;
    /* Sanity checking: Calling a python object requires to state number of
       vars being passed, bail if it doesn't match function declaration. */
    if (count != pFuncCode->co_argcount)
    {
      *errstrp = string_sprintf("Expected %d args to %s, was passed %d",
                                pFuncCode->co_argcount, name, count);
      return NULL;
    }

The string_sprintf is a function within the Exim source code which also handles memory allocation, making life easy for me.

Dynamic arg types for a python function when embedding

1 Answers1