Foreign function: who deallocates memory?

Question

I'm calling a Go function from Python. Go function returns a string, specifically, a GoString with the string itself being allocated on Go's side.

Question

Who's responsible for deallocation of this memory?

A very simplified example follows.

Go side:

func Create(optsEncoded string) (res string, serr string) {
    opts := map[string]interface{}{}
    if err := json.Unmarshal([]byte(optsEncoded), &opts); err != nil {
        return "", errWithStack(err)
    }
    options := translateCreateOptions(opts)

    result := ...

    payload, err := json.Marshal(result)
    if err != nil {
        return "", errWithStack(err)
    }
    return string(payload), ""
}

Cython bindings:

cpdef object py_create(object items, bytes options):
    cdef GoString opts = ...
    cdef bytes message

    cdef Create_return result = Create(opts)

    if result.r0.n == 0:
        message = result.r1.p
        raise Exception("Something happened")
    message = result.r0.p
    # Do I need to deallocate result.r0 and result.r1?
    return message.decode("utf-8")

Morty Choi · Accepted Answer · 2018-01-30T16:21:04.810

4

I don't think you should return a GoString to C. Since the memory of GoString is managed by the go runtime and will be garbage collected. It's unreliable in C land to use this string. What you should do is return a CString by calling cs := C.CString(s). Memory allocations made by C code are not known to Go's memory manager. So I think its up to you to mange which side to free the CString, here and here.

edited Jan 30 '18 at 16:21

answered Jan 29 '18 at 10:27

Morty Choi

2,466
1
18
26

Oh, that's great. I'll need to read the references first though. First link is a 404. – wvxvw Jan 29 '18 at 12:51
Morty is correct, I concur with them. The correct link to the 1st resource is [this](https://blog.golang.org/c-go-cgo). Please also read [this](https://github.com/golang/go/wiki/cgo) and—the most important one—[this](https://golang.org/cmd/cgo/)—esp. its "Go references to C" and "Passing pointers" parts. For truly hard-core details, you may find reading through [this](https://github.com/golang/go/issues/12416) to be enlightening. – kostix Jan 29 '18 at 13:58
1

One more comment: when using `CString` from Go, the memory will be allocated using `C.malloc` of the `libc` linked with the resulting Go code. Hence when combining this code with "plain" C code or C code which is an extension to Python (or whatever), two things are presupposed: 1) when that C code is compiled it will be linked with the same `libc`—to have the same implementation of `malloc` and `free`, and 2) your C extention must have a way to call unadorned `free` on that memory passed from the Go side (as opposed to some function from some CPython memory management code, if any). – kostix Jan 29 '18 at 14:02
1

@kostix makes a couple very important points about the same `free` and `malloc` used by Go and the extension. A few more details on that, python's own allocators are called `Py_Malloc` and `Py_Free` respectively so as not to be confused with the `libc` versions. In Cython code, use `from libc.stdlib cimport free, malloc` for the same functions Go is using via libc. – danny Jan 30 '18 at 16:36
1

Another approach to simplify this would be to convert the C string to a Python string in Cython. Since the C string was created via Go, its length is known, to make converting to a Python string faster. After it is made into a python object the C string can be de-allocated immediately as it has been copied and the Python string left up to Python to GC, since you'll need to return Python string from the Python function in any case. – danny Jan 30 '18 at 16:41

Foreign function: who deallocates memory?

Question

1 Answers1