24

I have a very large ArrayBuffer (or TypedArray) in JavaScript that I want to pass to an emscriptened function. I'd like to pass the raw bytes without incurring a copy.

If my C/C++ function takes an std::string as in:

void processBuffer(std::string const& buffer)

I can get the data, but IIUC, the conversion to std::string will incur a copy of the buffer.

Is there a way to pass the raw buffer without a copy?
My access is strictly read-only.

I tried:

void processBuffer(const char* str, size_t size);

with setting allow_raw_pointers() in the EMSCRIPTEN_BINDINGS, but this does not seem to work.
What am I missing?

Adi Shavit
  • 16,743
  • 5
  • 67
  • 137
  • Isn't `var ptr = Module._malloc(bytes); new Uint8Array(Module.HEAPU8.buffer, ptr, bytes);` good, instead of `new Uint8Array(bytes)` in JS? – zakki Mar 03 '17 at 09:54
  • 1
    Yes. If the buffer is allocated via `Module._malloc()` and passed directly to the asm.js function expecting a `char* ptr` it will not incur a copy (as was kindly explain to me on the emscripten forum). Will post a full answer later. – Adi Shavit Mar 03 '17 at 10:07
  • @zakki When reading data from a network or writing to an HTML5 canvas, you can't use a pre-allocated buffer; the buffer is allocated by the browser. It's really odd that emscripten doesn't seem to have a way to directly access native JS objects, even within `EM_ASM` blocks. – HRJ Apr 05 '17 at 06:28
  • Have you already seen http://stackoverflow.com/questions/17883799/how-to-handle-passing-returning-array-pointers-to-emscripten-compiled-code? – j4x Apr 10 '17 at 13:40

1 Answers1

6

Answering myself.
As it currently stands, there is no way to allow the emscriptened C/C++ code to access JS allocated memory buffers.

That being said, buffers allocated via Module._malloc() can be passed "by pointer" when using the C API.

Embinding will add additional copying into the C++ types.

For more info see this thread on the emscripten mailing list.

Adi Shavit
  • 16,743
  • 5
  • 67
  • 137
  • I'm tempted to think this should be possible using the File System API. – Jason Rice Feb 20 '18 at 22:47
  • I’m not familiar with it. What do you mean? Also note that things may have changed since this was asked and written, notably the publication of WASM. – Adi Shavit Feb 21 '18 at 06:38
  • I know you can load a file blob in a WorkerFS, but there doesn't seem to be support for an arbitrary ArrayBuffer (in File System API). Accessing them as files sounds feasible and perhaps should be added. – Jason Rice Feb 21 '18 at 17:28