0

So what I want to do is fairly straight forward, I want to access a C++ class in Python using swig. I've managed to do this for stand alone applications but that's not enough.

What I have: a fairly huge C++ library which is compiled using SCons. This generates a static (lib---.a) library. I also have a small C++ class using some of the library functionality.

What I tried to do was letting SCons compile everything, including my custom class, and then do the swig magic. I defined an swig interface file including the structure of my header file which I also included in the swig interface file.

In the header file there are dependencies to the library, but since these already have been compiled (into the static lib) swig can't find it. Replacing SCons is not an option. However, I can make a dynamic library instead of a static one.

So my question boils down to this: is there a proper way of using Swig inside SCons, or, can you somehow include the library dependency as an existing dynamic library? Furthermore, I can (using SCons) either compile my custom class into an object (.o) file or a dynamic library (.so) as well. Regarding the dynamic library, this is where things get messy since this is also what swig generates (?), or at least generated by someone.

Does this even make sense? I'm clearly on thin ice and don't necessarily know what I'm talking about.

Below are the c++ header and swig interface, I excluded c++ source since it felt redundant.

My Header File:

#include "../include/LogCabin/Client.h"
#include "../include/LogCabin/Debug.h"
#include "../include/LogCabin/Util.h"

class Ops{
private:
    void dumpTree(std::string);
    void dumpTreeLocal(std::string);
public:
    Ops(std::string, uint64_t, std::string);
   ~Ops();
    ...
    void makeLeader();
    void getLeader();
    int reconfigure(std::vector<std::string>);
};

My corresponding swig interface file:

%module Ops

%{
#include "Ops.h"
%}
%include "Ops.h"

class Ops{
public:
    Ops(std::string, uint64_t, std::string);
    ~Ops();
    ...
    void makeLeader();
    void getLeader();
};

Commands used:

swig -c++ -python Ops.i;
g++ -std=c++11 -c -fPIC Ops_wrap.cxx  -I/usr/include/python2.7 -I/usr/lib/python2.7;
g++ -std=c++11 -shared -Wl,-soname,_Ops.so -o _Ops.so Ops.o Ops_wrap.o;

Thanks. #swig

EDIT:

I include the SConscript called by SConstruct.

Import('env', 'object_files')

libs = [ "pthread", "protobuf", "rt", "cryptopp" ]

src = [
    "Ops.cc",
    "Ops.i"
]
/*object_files['Impl'] = env.StaticObject(src)*/

env.Default([
    env.SharedLibrary("Ops", src,
            LIBS = libs)
])

Edit2:

I updated the SContruct env declaration to set some parameters for swig.

env = Environment(options = opts,
              SWIGFLAGS=['-c++','-python'],
              CPPPATH=["/usr/include/python2.7"]),
              LIBPATH=["/usr/lib/python2.7"]),
              SHLIBPREFIX="",
              tools = ['default', 'protoc', 'packaging'],
              ENV = os.environ)

When compiling the generated Ops_wrap.cc file it gives me a lot of type cast warnings but still finish the compilation, e.g.

build/Impl/Ops_wrap.cc:654:23: warning: conversion to ‘unsigned char’ from ‘int’ may alter its value [-Wconversion]
       uu = ((d - '0') << 4);

When trying to access the Ops class via the generated Ops.py file it gives me error when importing Ops:

Traceback (most recent call last):
  File "test.py", line 3, in <module>
    import Ops
ImportError: /home/erneborg/Applications/coherent/logcabin/build/Impl/Ops.so: undefined symbol: _ZNK8LogCabin6Client4Tree6readExERKSs

The undefined symbol corresponds to a function in the library "LogCabin::Client::Tree::readEx()". Thoughts?

bdbaddog
  • 3,357
  • 2
  • 22
  • 33
Robert Erneborg
  • 69
  • 2
  • 12
  • Yes there is a proper way of using swig inside of SCons. It should be fairly straightforward. What version of SCons are you using? 2.5.0 has some changes to swig and multi-language scanning. Can you post a full (small) example? (.i, .cxx, .h, and SConstruct) Ignore the fact that there's a library. It shouldn't matter. – bdbaddog May 23 '16 at 20:58
  • the .i and .h files are just like the ones in the post, but i scipped a few functions to reduce the length of the post. They are also void functions (for the time being) with a std::string as arg. The cxx do you mean the .._wrap.cxx? if so, this file became about 5k lines long. As for the Sconstruct, it is almost 500 lines (I didn't write it...). For both of these are there any particular parts that are relevant (that I should post)? I will post the SConscript called by SConstruct. – Robert Erneborg May 24 '16 at 08:36
  • to clarify: I havent written the _wrap.cxx, it was generated when I ran "swig -c++ -python Ops.i" post Scons. I realize that it doesn't make sense to run swig after running scons (?). – Robert Erneborg May 24 '16 at 08:42
  • Normally, you just have to add the `*.i` file to your library sources. SCons will detect that it has to run `swig` for creating the CPP files, and will compile them to objects automagically. What's making things harder here is that you seem to fidlle around with single objects (see `object_files` in your example), so you're breaking up the "SCons flow". This may make an answer to your question harder as you think it should be. To put it another way: you seem to have additional constraints to your problem that aren't fully described yet (this is why @bdbaddog requests an MWE). ;) – dirkbaechle May 24 '16 at 09:13
  • Actually I don't need the StaticObject part (i think). I added Ops.i to src but now scons complaines about not finding the Python.h header. More details in Edit2 in the post. – Robert Erneborg May 24 '16 at 10:44
  • Just a small correction here: it's not SCons that complains about the missing header, but your compiler right? So, you just have to set the CPPPATH variable correctly, such that the required include paths appear on the command line. Get the command right in a terminal/shell first, and then try to setup SCons such that the arguments are the same... – dirkbaechle May 24 '16 at 11:35
  • yes you're correct. I did set the CPPATH and now everything compiles. However it gives me a load of warnings for weird type casts when it compiles the *_wrap.cc file. still finishes the compilation though. When I later test the generated python interface (dont know the correct word for it), i.e. Ops.py it gives me an importerror - described in the post – Robert Erneborg May 24 '16 at 15:52
  • Then you may want to open a separate question for this new problem...from my angle your original question (see title) is answered. – dirkbaechle May 25 '16 at 09:30

0 Answers0