2

If I put a cffi function in an lru_cache, I get a segfault, but cannot figure out why.

I am compiling simple C functions at runtime with the cffi package. I want to cache the results of this compilation because compilation has a non-trivial cost. The most obvious solution is lru_cache from functools. However, this causes a segfault when I try to call the function a second time. Why does this happen and what can I do to fix it? One clue is that calling Python's garbage collector between the calls is needed to guarantee the segfault, but I don't understand what is being collected to cause the segfault.

import gc
from functools import lru_cache
from tempfile import TemporaryDirectory

from cffi import FFI


@lru_cache()  # Segfault goes away if I get rid of this
def build():
    # FFI
    ffi = FFI()

    ffi.cdef("""
    void func(int a);
    """)
    ffi.set_source('_temp', """
    void func(int a) {
        return;
    }
    """)

    with TemporaryDirectory() as temp_dir:
        lib_path = ffi.compile(tmpdir=temp_dir)
        lib = ffi.dlopen(lib_path)
        func = lib.func

    return func


func = build()
func(1)
gc.collect()  # Guarantees segfault
func = build()
func(1)
drhagen
  • 8,331
  • 8
  • 53
  • 82
  • why do not simply create a factory for example, instead of the lru_cache? It is not the purpose of the lru_cache to deal with that IMO. – Netwave Mar 24 '19 at 12:17
  • I think there is some problem with the ffi lib.... The "factory" do not work also: https://repl.it/repls/LividSelfishBits – Netwave Mar 24 '19 at 12:22
  • Hmm, this works if I cache `lib` rather than `func`. The docs don't say one way or the other, but I wonder if `func` does not keep itself alive. When `lib` goes out of scope, is the memory freed and `func` is invalid? This does not explain why this works when NOT using an `lru_cache`, but that's UB for you. – drhagen Mar 24 '19 at 13:23
  • `func` should keep the ref to lib. It may be a real bug. – Netwave Mar 24 '19 at 13:53

0 Answers0