1

I have a Worker that shares a SharedArrayBuffer with the "main thread". To work correctly, I have to make sure that the worker has access to the SAB before the main thread accesses to it. (EDIT: The code creating the worker has to be in a seperate function (EDIT2: which returns an array pointing to the SAB).) (Maybe, already this is not possible, you'll tell me).

The initial code looks like this:

function init() {
  var code = `onmessage = function(event) {
      console.log('starting');
      var buffer=event.data;
      var arr = new Uint32Array(buffer);// I need to have this done before accessing the buffer again from the main
      //some other code, manipulating the array
  }`
  var buffer = new SharedArrayBuffer(BUFFER_ELEMENT_SIZE);
  var blob = new Blob([code], { "type": 'application/javascript' });
  var url = window.URL || window.webkitURL;
  var blobUrl = url.createObjectURL(blob);
  var counter = new Worker(blobUrl);
  counter.postMessage(buffer);
  let res = new Uint32Array(buffer);
  return res;
}

function test (){
  let array = init();
  console.log('main');
  //accessing the SAB again
};

The worker code is always executed after test(), the console shows always main, then starting.

Using timeouts does not help. Consider the following code for test:

function test (){
  let array = [];
  console.log('main'); 
  setTimeout(function(){
    array = initSAB();
  },0);
  setTimeout(function(){
    console.log('main');
   //accessing the SAB again
  },0);
  console.log('end');
};

The console shows end first, followed by main, followed by starting.

However, assigning the buffer to a global array outside the test() function does the job, even without timeouts.

My questions are the following:

  • why does the worker does not start directly after the message was send (= received?). AFAIK, workers have their own event queue, so they should not rely on the main stack becoming empty?
  • Is there a specification detailing when a worker starts working after sending a message?
  • Is there a way to make sure the worker has started before accessing the SAB again without using global variables? (One could use busy waiting, but I beware...) There is probably no way, but I want to be sure.

Edit

To be more precise:

  • In a completly parallel running scenario, the Worker would be able to handle the message immediately after it was posted. This is obviously not the case.
  • Most Browser API (and Worker is such an API) use a callback queue to handle calls to the API. But if this applied, the message would be posted/handled before the timeout calbacks were executed.
  • To go even further: If I try busy waiting after postMessage by reading from the SAB until it changes one value will block the program infinitely. For me, it means that the Browser does not posts the message until the call stack is empty As far as I know, this behaviour is not documentated and I cannot explain it.

To summerize: I want to know how the browser determines when to post the message and to handle it by the worker, if the call of postMessage is inside a function. I already found a workaround (global variables), so I'm more interested in how it works behind the scenes. But if someone can show me a working example, I'll take it.

EDIT 2:

The code using the global variable (the code that works fine) looks like this

function init() {
//Unchanged
}

var array = init(); //global

function test (){
  console.log('main');
  //accessing the SAB again
};

It prints starting, then main to the console.

What is also worth noticing : If I debug the code with the Firefox Browser (Chrome not tested) I get the result I want without the global variable (starting before main) Can someone explain?

Community
  • 1
  • 1
ixolius
  • 202
  • 1
  • 11

2 Answers2

2

why does the worker does not start directly after the message was sen[t] (= received?). AFAIK, workers have their own event queue, so they should not rely on the main stack becoming empty?

First, even though your Worker object is available in main thread synchronously, in the actual worker thread there are a lot of things to do before being able to handle your message:

  • it has to perform a network request to retrieve the script content. Even with a blobURI, it's an async operation.
  • it has to initialize the whole js context, so even if the network request was lightning fast, this would add up on parallel execution time.
  • it has to wait the event loop frame following the main script execution to handle your message. Even if the initialization was lightning fast, it will anyway wait some time.

So in normal circumstances, there is very little chances that your Worker could execute your code at the time you require the data.

Now you talked about blocking the main thread.

If I try busy waiting after postMessage by reading from the SAB until it changes one value will block the program infinitely

During the initialization of your Worker, the message are temporarily being kept on the main thread, in what is called the outside port. It's only after the fetching of the script is done that this outside port is entangled with the inside port, and that the messages actually pass to that parallel thread.
So if you do block the main thread before the ports have been entangled it won't be able to pass it to the worker's thread.

Is there a specification detailing when a worker starts working after sending a message?

Sure, and more specifically, the port message queue is enabled at the step 26, and the Event loop is actually started at the step 29.

Is there a way to make sure the worker has started before accessing the SAB again without using global variables? [...]

Sure, make your Worker post a message to the main thread when it did.

// some precautions because all browsers still haven't reenabled SharedArrayBuffers
const has_shared_array_buffer = window.SharedArrayBuffer;

function init() {
  // since our worker will do only a single operation
  // we can Promisify it
  // if we were to use it for more than a single task, 
  // we could promisify each task by using a MessagePort
  return new Promise((resolve, reject) => {
    const code = `
    onmessage = function(event) {
      console.log('hi');
      var buffer= event.data;
      var arr = new Uint32Array(buffer);
      arr.fill(255);
      if(self.SharedArrayBuffer) {
        postMessage("done");
      }
      else {
        postMessage(buffer, [buffer]);
      }
    }`
    let buffer = has_shared_array_buffer ? new SharedArrayBuffer(16) : new ArrayBuffer(16);
    const blob = new Blob([code], { "type": 'application/javascript' });
    const blobUrl = URL.createObjectURL(blob);
    const counter = new Worker(blobUrl);
    counter.onmessage = e => {
      if(!has_shared_array_buffer) {
        buffer = e.data;
      }
      const res = new Uint32Array(buffer);
      resolve(res);
    };
    counter.onerror = reject;
    if(has_shared_array_buffer) {
      counter.postMessage(buffer);
    }
    else {
      counter.postMessage(buffer, [buffer]);
    }
  });
};

async function test (){
  let array = await init();
  //accessing the SAB again
  console.log(array);
};
test().catch(console.error);
Kaiido
  • 123,334
  • 13
  • 219
  • 285
0

According to MDN:

Data passed between the main page and workers is copied, not shared. Objects are serialized as they're handed to the worker, and subsequently, de-serialized on the other end. The page and worker do not share the same instance, so the end result is that a duplicate is created on each end. Most browsers implement this feature as structured cloning.

Read more about transferring data to and from workers

Here's a basic code that shares a buffer with a worker. It creates an array with even values (i*2) and it sends it to the worker. It uses Atomic operations to change the buffer values.

To make sure the worker has started you can just use different messages.

var code = document.querySelector('[type="javascript/worker"]').textContent;

var blob = new Blob([code], { "type": 'application/javascript' });
var blobUrl = URL.createObjectURL(blob);
var counter = new Worker(blobUrl);

var sab;

var initBuffer = function (msg) {
  sab = new SharedArrayBuffer(16);
  counter.postMessage({
    init: true, 
    msg: msg, 
    buffer: sab
  });
};

var editArray = function () {
  var res = new Int32Array(sab);
  for (let i = 0; i < 4; i++) {
    Atomics.store(res, i, i*2);
  }
  console.log('Array edited', res);
};

initBuffer('Init buffer and start worker');

counter.onmessage = function(event) {
  console.log(event.data.msg);
  if (event.data.edit) {
    editArray();
    // share new buffer with worker
    counter.postMessage({buffer: sab});
    // end worker
    counter.postMessage({end: true});
  }
};
<script type="javascript/worker">
  var sab;
  self['onmessage'] = function(event) {
    if (event.data.init) {
      postMessage({msg: event.data.msg, edit: true});
    }
    if (event.data.buffer) {
      sab = event.data.buffer;
      var sharedArray = new Int32Array(sab);
      postMessage({msg: 'Shared Array: '+sharedArray});
    }
    if (event.data.end) {
      postMessage({msg: 'Time to rest'});
    }
  };
</script>
rafaelcastrocouto
  • 11,781
  • 3
  • 38
  • 63
  • Your example does not initialize the array in the 'init' part and cannot do it, but that is what I'm looking for. – ixolius Jan 16 '20 at 16:39
  • Furthermore, your code does not work, if I put the creation of the worker in a separate function. (I edited my question to make clear that this is part of my problem) – ixolius Jan 16 '20 at 16:56
  • Sorry but I do not understand what you are trying to achieve. Can you rephrase it or give more details of what's the purpose of this array initialization? Why the worker needs to have access to the SAB before the main thread? – rafaelcastrocouto Jan 16 '20 at 17:28
  • I've updated the answer, if it don't fit your purpose plz explain what you need. I'll keep trying and we will end up getting it just the way you need. – rafaelcastrocouto Jan 16 '20 at 17:48
  • Thanks a lot for being so patient, I added a paragraph, trying to precise what I need. Unfortunately, I'm not allowed to tell you *why* I need exactly this behavoiur. As I said, I'm not sure if a solution even exists, so thanks again for trying to provide help. – ixolius Jan 16 '20 at 23:06
  • 1
    I saw your edits but I'm still a bit lost, without a reason it all seems pointless. Perhaps if you post the code with the global variable that's working and the one you want to work, I can help you further. – rafaelcastrocouto Jan 17 '20 at 00:56
  • I just read the accepted answer, and it seems to answer quite nicely. Do u still have any doubts? – rafaelcastrocouto Jan 22 '20 at 11:46
  • No, this was what I was looking for. – ixolius Jan 22 '20 at 12:29