Passing Future as a function pointer to C++ using dart:ffi

Question

I want C++ code to execute a async function Future<int> inside of C++ itself as a function pointer using dart:ffi.

C++ code have to receive int value inside of C++ itself as result of Future<int>.
Also, I cannot avoid to use Future<int> because this function is provided by method channel.

Also, passing Future<int> as function pointer means that Future<int> is invoked as callback. Thus, the function which gets int from Future<int> using await is also Future<int> and it cannot be passed as function pointer.

dart:ffi accepts normal synchronous functions as function pointer.
So I tried to convert a async function Future<int> asyncFunction() async {...} to normal function int normalFunction() {...} like runBlocking {...} of Kotlin Coroutine, but I could not find any solution.

Do you have any idea to pass a async function Future<int> to C++ code as a function pointer, or evaluate Future<int> immediately and synchronously to invoke from normal function?

Answer 1

Simply put, you cannot pass a Future over the FFI boundary, nor is there a way in Dart to turn an async into a sync (nothing like runBlocking), but there are a few workarounds.

The simple rule for callbacks from C to Dart is that they must be called on the Dart main thread (i.e. you can only call a callback when in a C method that is being called from Dart) and must be synchronous (i.e. you cannot return a future from Dart to C).

--- Dart call through FFI to C method      -->
  <-- may make a callback to Dart on same thread

So, in your case, you want a Dart callback that returns an int that you can call from C (and you must only call that when handling a Dart->C call). However, you need to make a method channel call, or something else async like calling an API over HTTP.

There are a couple of workarounds, knowing that the Dart callback method must return immediately.

1. You can expect the callback and already have the answer ready.
2. You can return an invalid value, but note what was expected. Then reissue the whole request after you've got the value ready.

Real-life example

A C cryptography library needs to determine the decryption key to decrypt a file but doesn't know the key id until it reads the file header. It takes a callback that takes a string for the key id and returns the decryption key. The problem is that, in the Dart callback, we need to call an API to exchange the key id for the decryption key, and that would be async, so not allowed.

Workarounds:

1. In this real life example, it was actually relatively easy to read the header of the file and find the key id(s) contained in the header. (This is, of course, the first thing the C code does too.) So, before making the call to the C decrypt function, the Dart code simply read the header, ascertained the key ids, looked them up (async/await), and put the values in memory. It could then safely call the C decrypt method, knowing that the C code would discover the same key ids, and make the callback to Dart which could satisfy the key id -> key lookup from memory.

2. If it had not been possible to read the header in advance the second workaround is similar, but needs two bites of the cherry. Again in this example, the callback provided the key id, but had an invalid result value which meant can't find that key. On the first method invocation, the C code finds the key id, passes that to the callback which returns the invalid response, but notes the key id in memory. The C code returns a failure to decrypt error as it was not able to obtain the key. The Dart code notes the failure and further notes that it was called-back with a previously unknown key id, which it can then go and fetch (async/await again). Having fetched the key and cached it, it can retry the original method invocation. This time the callback finds the key id / key in memory, returns it and the C code is able to decrypt and return the message.

    --- Dart call to C decrypt method (1st)-->
      <-- C callback to Dart passing key id
      --- returns not found              -->
    <--- C code returns unable to decrypt
    Dart code looks up missing key id (async/await) and caches
    --- Dart call to C decrypt method (2nd)-->
      <-- C callback to Dart passing key id
      --- returns cached key             -->
    <--- C code returns decrypted message

So, if you can predict that the C will need a particular value, have it ready in advance, before calling the C method. Or, try it, see what it asks for on the first attempt, then get it ready and try again (assuming the C will ask for the same value it asked on the first callback).