SWIG interfacing C library to Python (SWIG generated classes are cumbersome to use)

Question

I am using SWIG to generate Python language bindings to my C library. I have managed to build the bindings and exported data structures, but I'm having to jump through some hoops when using the library.

For example the C header has data types and function prototypes as follows:

struct MyStruct
{
    /* fields */
} 

struct MyStruct * MYSTRUCT_Alloc(void);
void MYSTRUCT_Free(struct MyStruct *);
struct MyStruct  * MYSTRUCT_Clone(const struct MyStruct *);
int MYSTRUCT_Func1(const struct MyStruct *, const int);

/* and so on */

In my SWIG interface file, I am exporting both the functions and the MyStruct data type. Assuming my python extension module is called foobar, I can then write Python script like this:

#import foobar as fb

# The line below creates a Python class which is a wrapper to MyStruct. HOWEVER I cannot pass this class to a function like MYSTRUCT_Func1 until I have initialized it by calling MYSTRUCT_Alloc ...

ms = fb.MyStruct  

# This will fail (throws a Python exception)
# ret =  fb.MYSTRUCT_Func1(ms, 123)

# However this works
ms = fb.MYSTRUCT_Alloc()
ret =  fb.MYSTRUCT_Func1(ms, 123)

It is very cumbersome (and error prone) to declare an object and then assign a pointer to it before using it. Is there a better way of using the SWIG generated classes?. I was thinking of wrapping higher level classes (or subclassing the SWIG generated classes) to automagically take care of object creation and destruction (as well as providing some OBVIOUS member functions like MYSTRUCT_Func1().

HOWEVER, if I do wrap/subclass the SWIG generated classes, then I am not sure that I can pass the new classes to the C API functions that expect a pointer to a C struct. I can't modify the SWIG generated classes directly (or atleast I shouldn't) - for OBVIOUS reasons.

What is the best way to solve this problem? A more pythonic way of creating/destroying objects, whilst at the same time being able to pass pointers directly to the exposed C functions?

Consider using ctypes to access your library instead of SWIG. — ThiefMaster, Nov 08 '11 at 09:23

Miguel Grinberg · Answer 1 · 2011-11-08T17:10:44.537

2

Writing a wrapper on the Python side seems like a good idea to me, not sure why you think that is not going to work.

class MyStructWrapper:
    def __init__(self):
        self.ms = fb.MYSTRUCT_Alloc()

    def __del__(self):
        fb.MYSTRUCT_Free(self.ms)

    def func1(self, arg):
        return fb.MYSTRUCT_Func1(self.ms, arg)

And if you need to access the members of the struct, then you can do so with self.ms.member or by writing getters and setters.

You can also fit your clone function into this design.

Edit: Regarding your comment, let's say you have a global function that takes a pointer to MyStruct:

int gFunc(MyStruct* ms);

In the Python side, you can write a wrapper as follows:

def gFuncWrapper(mystruct):
    return fb.gFunc(mystruct.ms)

I hope this helps.

edited Nov 08 '11 at 17:10

answered Nov 08 '11 at 02:20

Miguel Grinberg

65,299
14
133
152

I also have free (non-member) functions which expect C struct ptrs as arguments. Currently, I can pass the SWIG generated class to those exposed C functions. Can I similarly pass MyStructWrapper to those functions?. If no, is there a "special method" that I can e.g. __value__ that I can implement on the wrapper class so that it returns the raw pointer when being passed to a function that expects the C struct pointers? – Homunculus Reticulli Nov 08 '11 at 08:18
Sure, you need to create Python wrapper functions for the global functions. Those will take the Python wrapper class, but send down the ms member down to the C functions. I'll update my answer with an example. – Miguel Grinberg Nov 08 '11 at 17:07

score 2 · Accepted Answer · answered Nov 08 '11 at 09:20

2

Your code:

ms = fb.MyStruct

# This will fail (throws a Python exception) 
# ret =  fb.MYSTRUCT_Func1(ms, 123)

Only assigns the class to ms, not an instance of a class, which is why the commented line fails. The following should work:

ms = fb.MyStruct()
ret =  fb.MYSTRUCT_Func1(ms, 123)

answered Nov 08 '11 at 09:20

Mark Tolonen

166,664
26
169
251

Ah, so that it where I am going wrong (I'm a python newbie). It works correctly now. One final thing though. I would like to add new behaviours and methods to the MyStruct() class - BUT crucially, I still want to be able to pass the "enhanced class" to functions that expect a raw pointer (currently, I CAN pass the auto generated MyStruct class to my exported C funcs that expect a raw pointer). Obviously, I can't directly modify the generated code, so I tried to subclass MyStruct, but when I pass the new class to my C function, I get a TypeError exception argument 1 of type 'struct MyStruct *' – Homunculus Reticulli Nov 08 '11 at 10:11
Another reason I would like to provide a wrapper around MyStruct is so that I can destroy it correctly. It is a nested struct and some functions actually allocate memory from the heap - so it neeads to be destroyed properly, to avoid mem leakage – Homunculus Reticulli Nov 08 '11 at 10:18
2

Have you considered writing a C++ class and using SWIG to expose that instead? Then you can define proper constructors and a destructor and add additional methods. SWIG will do the work flattening the class into C methods and provide a Python wrapper class so it acts like a class again in Python. – Mark Tolonen Nov 08 '11 at 10:31
What @Mark said is a good idea. My answer to this question proposes doing something like this but on the Python side. Doing it on the C++ side would be better if you feel more comfortable working on C++ than on Python. – Miguel Grinberg Nov 08 '11 at 17:13
Mark & Miguel. I think you are both right. I have written C++ wrappers around the C library to ease use from Python. I'll accept Mark's answer because he pointed out how to use the library from Python as well as suggesting to use a C++ wrapper to simplify things. – Homunculus Reticulli Nov 12 '11 at 14:34
@Homunculus Reticulli: Or you could write the extension type in Cython as [suggested in my answer](http://stackoverflow.com/questions/8044870/swig-interfacing-c-library-to-python-swig-generated-classes-are-cumbersome-to-u/8045288#8045288) instead of C++. It allows you to call Python transparently as well as to invoke C efficiently. As a bonus you use Python-like syntax instead of C++ in this case, after all you've chosen to write a Python wrapper for C library so you might've seen some advantages in using Python language. – jfs Nov 13 '11 at 07:32

jfs · Answer 3 · 2011-11-08T02:55:57.667

You could name your SWIG-generated module _foobar and write a pure Python module foobar that defines the necessary pythonic interface e.g., M2Crypto an openssl wrapper follows that approach.

Another option is to use Cython to create the interface straight in C:

cdef class MyStruct:
    cdef MyStruct_ptr this

    def __cinit__(self):
        self.this = MYSTRUCT_Alloc();
        if self.this is NULL:
           raise MemoryError

    def __dealloc__(self):
        if self.this is not NULL:
            MYSTRUCT_Free(self.this)

    def func1(self, n):
        return MYSTRUCT_Func1(self.this, n)

This creates Python C extension type MyStruct that can be used from Python as:

ms = Mystruct()
print(ms.func1(123))

See wrap Person.h for a complete example.

SWIG interfacing C library to Python (SWIG generated classes are cumbersome to use)

3 Answers3