Using a struct from a different library in Cython

Question

I am trying to call a C function in Cython and the header looks like this:

#include <stdio.h>
#include <stdint.h>
#include <inttypes.h>
#include <ctype.h>
#include <unistd.h>
#include <math.h>

#include <apriltag.h>
#include <tag36h11.h>

#include <common/getopt.h>
#include <common/image_u8.h>
#include <common/image_u8x4.h>
#include <common/pjpeg.h>
#include <common/zarray.h>

apriltag_detection_t* scan_frame(int width, int height, uint8_t* data);

As you can see, I want to return an array of structs, which are type defined to be apriltag_detection_t. According to the documentation, in order to be able to use this in Cython, I must define some sort of pxd file which is essentially a copy of header.

However, apriltag_detection_t is a type that is already defined in apriltag.h. Furthermore, apriltag_detection_t has members that are already defined in apriltag.h. Do I have to recursively redefine all of these types (by hand) in the Cython file before being able to use this library? Where would I write them?

Thanks!

UPDATE 6

Finally at the step of wrapping a function!

from libc.stdint cimport uint8_t

cdef extern from "<apriltag.h>":
    cdef struct apriltag_detection:
        int id
        double c[2]
        double p[4][2]

    ctypedef apriltag_detection apriltag_detection_t

cdef extern from "tag36h11_detector/tag36h11_detector.h":
    apriltag_detection_t* scan_frame(int width, int height, uint8_t* data);

def detect(width, height, frame):
    return scan_frame(width, height, frame)

tag36h11_detector.pyx:15:21: Cannot convert 'apriltag_detection_t *' to Python object

apriltag_detection_t* is meant to be an array of structs

UPDATE 5 This seems to have worked.

from libc.stdint cimport uint8_t

cdef extern from "<apriltag.h>":
    cdef struct apriltag_detection:
        int id
        double c[2]
        double p[4][2]

    ctypedef apriltag_detection apriltag_detection_t

cdef extern from "tag36h11_detector/tag36h11_detector.h":
    apriltag_detection_t* scan_frame(int width, int height, uint8_t* data);

UPDATE 4 Solved the previous issues by importing the necessary types.

from libc.stdint cimport uint8_t

cdef extern from "apriltag.h":
    cdef struct apriltag_detection:
        int id
        double c[2]
        double p[4][2]

    ctypedef apriltag_detection apriltag_detection_t

cdef extern from "tag36h11_detector.h":
    apriltag_detection_t* scan_frame(int width, int height, uint8_t* data);

tag36h11_detector.c:533:10: fatal error: 'apriltag.h' file not found

I'm not sure where this is coming from because my header file, as provided in the original post, required <apriltag.h> and not "apriltag.h". This is what my setup.py looks like.

from distutils.core import setup, Extension
from Cython.Build import cythonize

setup(ext_modules=cythonize(Extension(\
    name='tag36h11_detector', \
    sources=["tag36h11_detector.pyx", \
    "tag36h11_detector/tag36h11_detector.c"], \
    include_path=["/usr/local/include/apriltag"], \
    libraries=["apriltag"])))

UPDATE 3

cdef extern from "apriltag.h":
    cdef struct apriltag_detection:
        int id
        double c[2]
        double p[4][2]

    ctypedef apriltag_detection apriltag_detection_t

cdef extern from "tag36h11_detector.h":
    apriltag_detection_t* scan_frame(int width, int height, uint8_t* data);

tag36h11_detector.pyx:10:60: 'uint8_t' is not a type identifier

UPDATE 2

This is my current code and the following is the compilation error

// tag36h11_detector.pyx
cdef extern from "apriltag.h":
    ctypedef apriltag_detection_t:
        int id
        double c[2]
        double p[4][2]

cdef extern from "tag36h11_detector.h":
    apriltag_detection_t* scan_frame(int width, int height, uint8_t* data);

// apriltag.h
    ...
    typedef struct apriltag_detector apriltag_detector_t;
    ...

tag36h11_detector.pyx:2:33: Syntax error in ctypedef statement

UPDATE 1

So I'm trying to interface with the above header file with Python (which I wrote and implemented) with types defined in apriltag.h (from a library).

cdef extern from "apriltag.h":
    struct apriltag_detection:
        int id
        double c[2]
        double p[4][2]

cdef extern from "tag36h11_detector.h":
    struct apriltag_detection* scan_frame(int width, int height, uint8_t* data);

When I try compiling the above, I get

tag36h11_detector.pyx:8:29: Syntax error in struct or union definition

For your updated question, have you tried defining the structs as `ctypedef struct apriltag_detection` instead in your pxd file? — CodeSurgeon, Oct 01 '17 at 16:58
@CodeSurgeon same error. Also, I wrote all of the above on a `.pyx` file, as the answer below said. — Carpetfizz, Oct 01 '17 at 17:00
I think you don't need "struct" prior to 'apriltag_detection" in the last line. See also the way the function is imported in the example — ead, Oct 01 '17 at 17:07
Interesting. Usually, I actually write my c-library wrappers in a separate pxd file. When writing them, I found some online examples to be helpful. These ones are for [OpenGL](https://gist.github.com/krischer/4219796) and [SDL2](https://gist.github.com/krischer/4219808) respectively that gave me some assistance. And [here](https://gist.github.com/AnishN/860776e75f58dc8595f7955198af42a8) is an incomplete one that I am currently in the process of writing for the chipmunk physics library. Hope these examples help to give you a different approach! — CodeSurgeon, Oct 01 '17 at 17:08
@ead The `struct` in the original header, `apriltag.h` is actually `typedef struct apriltag_detection apriltag_detection_t`, so I tried accounting for that but I'm getting a different error — Carpetfizz, Oct 01 '17 at 17:32
You are fast! If path to header is relative to the pyx-file (e.g. In the same folder) then " " should be used, if from include-path, then "<>" — ead, Oct 01 '17 at 18:15
Thanks for all your help! I think I'm close.. I'm having the error "Cannot convert `apriltag_detection_t*` to Python object". This is an array of structs, so I'm not sure how to write the appropriate Cython code — Carpetfizz, Oct 01 '17 at 18:17
There are no pointers in python. If you want to use this function and its result in python, you have to covert data to a python object(dictionary or a handmade class) or declare this function 'cdef' otherwise — ead, Oct 01 '17 at 19:13
@ead thanks. I have opened a new question here with some updated code: https://stackoverflow.com/questions/46515782/returning-an-array-of-structs-in-cython?noredirect=1#comment79985875_46515782 If I declare the function as `cdef`, how will I interpret the return type in my Python code? — Carpetfizz, Oct 01 '17 at 19:14

ead · Accepted Answer · 2017-10-02T19:09:10.477

This is basically covered in this part of cython documentation, which says that you only need to import parts, which you will be using in your cython code.

For example let's take a look at the following C interface:

#struct.h
struct Needed{
  int a;
};

struct NotNeeded{
  int b;
};

struct Combined{
  struct Needed needed;
  struct NotNeeded notneeded;
};

struct Combined create(void);

You would like to call the function create and use the value a from the Needed struct, that means you have to import struct Needed and parts of struct Combined but not NotNeeded in your cython code:

#struct_import.pyx
cdef extern from "struct.h":
     struct Needed:   # use "ctypedef struct Needed" if defined with typedef in h-file! 
            int a  
     struct Combined:  #NotNeeded is missing!
          Needed needed 
     Combined create()  

def get_needed():
    return create().needed.a #should be 42!

Now, using setup.py (its content can be seen further below, the same goes for the content of struct.cto) we get the expected result:

[] python setup.py build_ext --inplace
[] python -c "python -c "import struct_import as si; print si.get_needed()"
    42

If you use cython to glue some C code together, it is possible, even less is necessary. In oour example, if we had a C-function which would extract the needed value from the Combined struct:

#struct.h 
...
int extract(struct Combined combined);//returns combined.needed.a

We could use it as follows in the pyx-file:

#struct_import.pyx
cdef extern from "struct.h":
     struct Combined: 
          pass  #nothing imported!
     Combined create()  
     int extract(Combined combined)

def get_needed():
    return extract(create()) #should be 42!

And despite we didn't import the Needed struct at all, it works as good as the first version.

So if it becomes a chore to import all those structs, one could extend the c interface to make it unnecessary.

To make the example complete, here the missing setup.py and struct.c files:

#setup.py:
from distutils.core import setup, Extension
from Cython.Build import cythonize

setup(ext_modules=cythonize(Extension(
            name='struct_import',
            sources = ["struct_import.pyx", "struct.c"]
    )))

and

//struct.c
#include "struct.h"

struct Combined create(){
  struct Combined res;
  res.needed.a=42;
  res.notneeded.b=21;
  return res;
}

int extract(struct Combined combined){
   return combined.needed.a;
}

Using a struct from a different library in Cython

1 Answers1