MediaSteam interaction with WebAssembly

Question

Is there a known way to get a video stream from the browser back into WebAssembly? I know that you can do frame capture using HTML canvas, but I am unaware of how to get the entirety of the stream (audio/video) and send it to my WebAssembly application.

Answer 1

Take a look at the internals of the "WebAssembly Video Editor" sample:

• https://d2jta7o2zej4pf.cloudfront.net/
• https://github.com/shamadee/web-dsp

Answer 2

WebAssembly has a very simple interface, with only 4 types (two integeres, two floating points), the ability to import or export functions, and memory. It doesn't have any APIs for accessing the DOM, fetching data, etc ... therefore you have to provide this data to your WebAssembly module from the hosting JavaScript.

Are you trying to decode a video stream from a WebAssembly module? In that case, I'd expect the WebAssembly code to export a function, e.g. decode, that your JavaScript code invokes as data arrives from the stream. You'll likely need to write the data to the WebAssembly memory, invoking this decode function with the memory address and length of data that has been supplied.