DOMException when hot-loading Web Audio worklet processor with Wasm modules from a Blob?

Question

I'm trying to load dynamic Web Audio worklets with Wasm module imports transpiled from C++ code using Emscripten.

I want to manipulate worklets code in memory, using Blob to create objects, injecting them as modules into the audio worklet, and hot-swapping them.

I have followed the design pattern suggested in the web-audio-samples solution to implement worklets that import as Wasm module into a worklet. This seems to work well when the processor code is in a file.

Hot-loading and swapping worklets from memory without external Wasm module imports also seems to work well.

This is how I create the code for the Blob. To illustrate I'm using the simple-kernel.wasmmodule.js that is compiled with the same Emscripten configuration as the Wasm design pattern example in the web-audio-samples.

    const blobCode = () => {

      return `
      import Module from './simple-kernel.wasmmodule.js';
      import { RENDER_QUANTUM_FRAMES, MAX_CHANNEL_COUNT, HeapAudioBuffer } from '../lib/wasm-audio-helper.js';

      class WASMWorkletProcessor extends AudioWorkletProcessor {

        constructor() {
          super();

          // Allocate the buffer for the heap access. Start with stereo, but it can
          // be expanded up to 32 channels.
          this._heapInputBuffer = new HeapAudioBuffer(Module, RENDER_QUANTUM_FRAMES,
                                                      2, MAX_CHANNEL_COUNT);
          this._heapOutputBuffer = new HeapAudioBuffer(Module, RENDER_QUANTUM_FRAMES,
                                                      2, MAX_CHANNEL_COUNT);
          this._kernel = new Module.SimpleKernel();
        }

        process(inputs, outputs, parameters) {
        .
        .
        .
           return true;
        }
      }
      registerProcessor('wasm-worklet-processor', WASMWorkletProcessor);`;
}

And this is how I use the code with Blob to create an object and load it into the worklet.

const workletHotLoading = async (context) => {

  const blob = new Blob([ blobCode() ], { type: "application/javascript; charset=utf-8" });

  const workletUrl = window.URL.createObjectURL(blob);

  await context.audioWorklet.addModule(workletUrl);

  const oscillator = new OscillatorNode(context);

  const wasmBlobWorkletNode = new AudioWorkletNode(context, 'wasm-worklet-processor');

  wasmBlobWorkletNode.onprocessorerror = (event) => {
    console.log(`An error from WASMWorkletProcessor.constructor() was detected.`);
  };

  oscillator.connect(wasmBlobWorkletNode).connect(context.destination);

  oscillator.start();
};

I was expecting this to work as the processors with no Wasm imports do, or when I load them from a file. If comment the module imports and module code in the worklet constructor and process method, it works.

However, hot-loading a worklet with a Wasm import does not appear to be working... When I try to do that, I get "Error on loading worklet: DOMException" and no other clue.

I suspect that this might be a bit naive and that it might require more sophistication such as dynamic imports...

I created a fork of web-audio-samples solution where I added a small sample project (wasm-hot-loading) that creates the conditions to illustrate the problem.

It is available here: https://github.com/mimic-sussex/web-audio-samples/tree/master/audio-worklet/design-pattern/wasm-hot-loading

Can anybody help shed some light on what the problem might be and whether this is feasible?

Thanks

FØ

Kaiido · Accepted Answer · 2019-04-23T23:54:07.517

The problem is that your Worklet Processor's origin is opaque. This means that your relative imports can not be resolved from inside this script.

To circumvent this, you need to use absolute URIs.

So you have to rewrite your code in order to add the correct baseURI to your imports statements. Since this code is called from your main js page, you can then build this blob content with the correct format e.g using the URL constructor and its second parameter base.

const blobCode = () => {
    return `
import Module from "${
  // convert relative to absolute URL
  new URL('./simple-kernel.wasmmodule.js', location.href)
}";
import { RENDER_QUANTUM_FRAMES, MAX_CHANNEL_COUNT, HeapAudioBuffer } from "${
  new URL('../lib/wasm-audio-helper.js', location.href)
}";

class WASMWorkletProcessor extends AudioWorkletProcessor {

  constructor() {
    super();
[...]
 `
 };

console.log(blobCode());

score 1 · Answer 2 · answered Jul 31 '19 at 17:58

I had a similar bug. Dom Exception throwned only when import is in the worklet. the problem was due to a chrome extension called web audio inspector this extension adds certainly some wrapping function around the worklet, since the import is not authorized to be invoke into a function, the worklet import breaks. Anyway, uninstalled the extension and problem solved.

DOMException when hot-loading Web Audio worklet processor with Wasm modules from a Blob?

2 Answers2