Read swig python 'double *' object into numpy (maybe through ctypes?)

Question

I have swig python library that I can't modify.

It returns a <Swig object of type 'double *'> which I know is a pointer to an array of doubles. Through a separate function, I get a python int for the length.

My question is, how can I read this data into numpy?

I've found the numpy.ndarray.ctypes module, and another stackoverflow answer that hints that a conversion from SWIG to ctypes is possible (https://stackoverflow.com/a/41212424/654602) but makes no mention how.

Any help is appreciated.

Mark Tolonen · Accepted Answer · 2018-08-10T04:44:05.810

I created a sample SWIG wrapper to test this:

%module test

%{
#include <stdlib.h>
#include <stdio.h>

double* get(void)
{
    double* p = malloc(sizeof(double) * 10);
    for(int i = 0; i < 10; ++i)
        p[i] = 1.1 * i;
    printf("%p\n",p);
    return p;
}
%}

double* get(void);

The following retrieves the data via ctypes:

>>> import test
>>> a = test.get()
000001A3D05ED890        # From the printf...
>>> a
<Swig Object of type 'double *' at 0x000001A3D27D6030>
>>> hex(int(a))
'0x1a3d05ed890'         # int() of the object is the same address
>>> from ctypes import *
>>> p = (c_double * 10).from_address(int(a))
>>> list(p)
[0.0, 1.1, 2.2, 3.3000000000000003, 4.4, 5.5, 6.6000000000000005, 7.700000000000001, 8.8, 9.9]

Now for numpy. There may be a better way, but I found __array_interface__ (link). An "array-like" object has this interface and another array can be created from it:

>>> class Tmp: pass
...
>>> Tmp.__array_interface__ = {'shape':(10,),'typestr':'<f8','data':(int(a),False),'version':3}
>>> import numpy as np
>>> np.array(Tmp,copy=False)   # Create array that shares the same interface
array([0. , 1.1, 2.2, 3.3, 4.4, 5.5, 6.6, 7.7, 8.8, 9.9])

Maybe not the best way, but I'm not a heavy user of numpy.

I didn't even think of casting to int to get the address. Wonderful! For the numpy conversion, I'm using `np.ctypeslib.as_array(p)`, which I found in the answers from here: https://stackoverflow.com/questions/4355524 — Tyler Fox, Aug 10 '18 at 18:03
@Tyler I'll have to try that one out. I found that `__int__` was a supported method on the Swig Object by doing a `dir()` on it and found it was the actual buffer address. — Mark Tolonen, Aug 10 '18 at 18:08

score 0 · Answer 2 · answered Aug 28 '18 at 06:03

If you want to receive an array from C, I recommend you use NumPy SWIG bindings numpy.i.

Here I am using ARGOUTVIEWM_ARRAY1 such that the memory allocated in C will henceforth be managed by the NumPy array. If this does not suit your needs, you can choose a different typemap instead. There is good documentation.

I also rename the the library_function so I can override it with my own function which has the correct signature for the NumPy typemap.

test.t

%module example
%{
#define SWIG_FILE_WITH_INIT

#include <stdlib.h>
#include <stdio.h>

double* library_function(void)
{
    double* p = malloc(sizeof(double) * 10);
    for(int i = 0; i < 10; ++i)
        p[i] = 1.1 * i;
    printf("%p\n",p);
    return p;
}
%}

%include "numpy.i"

%init %{
import_array();
%}

%apply (double** ARGOUTVIEWM_ARRAY1, int* DIM1) {(double** data, int* length)};
%rename (library_function) encapsulate_library_function;
%inline %{
void encapsulate_library_function(double** data, int* length) {
    *data = library_function();
    *length = 10;
}
%}

test.py

from example import library_function

a = library_function()
print(type(a))
print(a)

Example invocation:

$ swig -python test.i
$ clang -Wall -Wextra -Wpedantic -I /usr/include/python3.6m/ -fPIC -shared -o _example.so test_wrap.c -lpython3.6m
$ python3 test.py
0x2b9d4a0
<class 'numpy.ndarray'>
[ 0.   1.1  2.2  3.3  4.4  5.5  6.6  7.7  8.8  9.9]

Read swig python 'double *' object into numpy (maybe through ctypes?)

2 Answers2

Linked