What ctypes argument type to use for variably modified multidimensional arrays

Question

Using ctypes in python I can freely pass an array to a C function using a pointer to its first element since an array decays to a pointer to its first element in C. So for the C function signature

void insert_sort(int A[], size_t length);

I can implement the following python wrapper.

import ctypes
from typing import List, Callable

def wrap_function(lib: ctypes.CDLL, name: str, restype: object, argtypes: List[object]) -> Callable:
    """Simplify wrapping ctypes functions"""
    func = lib.__getattr__(name)
    func.restype = restype
    func.argtypes = argtypes
    return func


lib = ctypes.CDLL("path/to/library.so")
insert_sort = wrap_function(lib, 'insert_sort', None, [ctypes.POINTER(ctypes.c_int), ctypes.c_size_t])

But C also allows one to have arrays of array and whats more variable sized arrays of arrays. For example the C function signature below demonstrates this.

void matrix_multiply(size_t n, int A[n][n], int B[n][n], int C[n][n]);

How would one wrap this function? To codify my question what goes in the blank below?

matrix_multiply = wrap_function(lib, 'matrix_multiply', None, [...What goes here...])

If the size of the arrays were known we could simply write

matrix_multiply = wrap_function(lib, 'matrix_multiply', None, [
    ctypes.c_size_t, ((ctypes.c_int * 4)*4), ((ctypes.c_int * 4)*4), ((ctypes.c_int * 4)*4)
])

for arrays of size 4. But what can we do when this number is unknown?

Show an *matrix\_multiply* example and how do you compile it. — CristiFati, Jun 20 '20 at 18:05
@CristiFati I'm not sure what that has to do with the question. It could just be an empty function ```void matrix_multiply(size_t n, int A[n][n], int B[n][n], int C[n][n]){}```. The point is getting python to run it for arbitrary input arrays. — Ivor Denham-Dyson, Jun 20 '20 at 18:30
Good question indeed, I haven't ever thought about using vla with ctypes — Antti Haapala -- Слава Україні, Jun 20 '20 at 18:34
@IvorDenham-Dyson: it does because I'm not sure what compiler would tolerate that (`void matrix_multiply(size_t n, int A[n][n], int B[n][n], int C[n][n]);`). Meaning that the presumption that you start from is invalid. You can pass multidimensional arrays, it's just a bit more complicated. However you can't pass *VLA*s. The arrays would have to be dimensioned. — CristiFati, Jun 20 '20 at 18:57
@CristiFati Here's an online example https://onlinegdb.com/SkxXkTRipL of such a function. — Ivor Denham-Dyson, Jun 20 '20 at 19:01
*gcc* and *cl* don't like it. Anyhow, some examples for *2D* arrays: https://stackoverflow.com/questions/58727931/how-to-pass-a-2d-array-from-python-to-c/58735157#58735157, https://stackoverflow.com/questions/58781199/how-can-i-cast-a-double-pointer-ctype-to-numpy-array/58784385#58784385 — CristiFati, Jun 20 '20 at 19:06
@CristiFati Looking into it a bit it seems it to be fine with gcc and clang here https://godbolt.org/z/77wc3Z. There is also reference to this exact type of declaration here https://gcc.gnu.org/onlinedocs/gcc/Variable-Length.html, it appears to be a gnu extension. Thanks for the links — Ivor Denham-Dyson, Jun 20 '20 at 19:59
To clarify the above the gnu extension is that it allows this in C90 mode and C++. VLA is part of the C99 standard as far as I understand it. — Ivor Denham-Dyson, Jun 20 '20 at 20:57

score 0 · Answer 1 · answered Jun 22 '20 at 13:42

This answer is based on reverse engineering the ABI. I compiled this test program to assembly...

#include <stddef.h>
extern void matrix_multiply(size_t n, int A[n][n], int B[n][n], int C[n][n]);

extern int X[64][64];
extern int Y[64][64];
extern int Z[64][64];

void test(void)
{
  matrix_multiply(64, X, Y, Z);
}

... using GCC 9 on x86_64-linux, and I got this assembly language:

test:
    leaq    Z(%rip), %rcx
    leaq    Y(%rip), %rdx
    leaq    X(%rip), %rsi
    movl    $64, %edi
    jmp matrix_multiply@PLT

Based on this I conclude that a variably modified array parameter is passed as a pointer to its first element, so the appropriate ctypes declaration for matrix_multiply is

c_int_p = ctypes.POINTER(ctypes.c_int)
matrix_multiply = wrap_function(lib, 'matrix_multiply', None, [
    ctypes.c_size_t, c_int_p, c_int_p, c_int_p
])

Wrapper that takes NumPy arrays and/or memoryviews left as an exercise.

What ctypes argument type to use for variably modified multidimensional arrays

1 Answers1