1

I have a web worker running in a while(true) loop, the code is too deep to change it at this point in time. The worker can access a Shared Array Buffer, I have a means to signal that there is data present in it (right now it is just being used with strings and ints and stuff). I want to pass a local File object through the shared array buffer. How can I serialize the File Object and deserialize it on the other end (it is even ok with me to reconstruct a new file object on the other side if I have to)?

This is my rough structure, but add a couple thousand lines and you are there :)

<script type='text/javascript'>
        var s_curFile;

        var s_curFile2;

        var sab = new SharedArrayBuffer(65563); //I will probably make this bigger...

        function JSprocessFilePicker( input )
        {
            let url = input.value;
            let ext = url.substring( url.lastIndexOf( '.' ) + 1 ).toLowerCase();
            if ( input.files && input.files[0] && ( ext == "txt" ) )
            {
                s_curFile = input.files[0];

                //I know I can create another file object like this, can i pass just this info in the shared array buffer, so I can create a new file in the worker.
                //s_curFile2 = new File([s_curFile],s_curFile.name,{type:s_curFile.type});

                //how to serialize s_curFile or needed info here!
            }
        }
        var input = document.createElement( "input" );
        input.setAttribute( "id", "file_picker" );
        input.setAttribute( "type", "file" );
        input.setAttribute( "accept", ".txt" );
        input.setAttribute( "onchange", "JSprocessFilePicker(this)" );

        var worker = new Worker('worker.js');

        //I can't do another post message because worker enters a while(true loop) after getting the shared array buffer during startup. And i have to wait for the user to open the file to pass it to the worker
        try {
            worker.postMessage({
                'cmd' : 'data',
                'sb'  : sab
            });
        } catch(e) {
            alert('Can\'t spawn files to worker - '+e)
        }
</script>

<button onclick="input.click();">Open</button>

Or if there is some other way that doesn't involve a callback being processed (unless there is a way to manually process onmessage callbacks) where I can pass the file to the worker without postMessage, that would also be acceptable to me?

yosmo78
  • 489
  • 4
  • 13
  • I don't know that much about web worker but why do you want to send it as `SharedArrayBuffer` ?, according to docs postMessage can take `File` type data – bogdanoff May 04 '22 at 15:32
  • @bogdanoff Because I do not have access to the worker queue as my worker is in a deep loop, as stated in the first sentence – yosmo78 May 04 '22 at 16:21
  • 1
    in question you have mentioned 'shared array buffer or another buffer type of data structure to workers ', so does `ArrayBuffer` work for you ? you can get it using `await ele.files[0].arrayBuffer()` – bogdanoff May 04 '22 at 17:24
  • @bogdanoff the `.arrayBuffer()` returns the contents of the blob into an array buffer. I am looking at passing the file object through a shared array buffer since what the file object/blob refers to may be on the order of magnitude of gigabytes. so since I can't pass the contents of the file around, i need a way of sending the handle so that i can do read on a worker. – yosmo78 May 07 '22 at 19:08
  • Only solution I found for converting `File` into `SharedArrayBuffer` was using for loop (may be with async function wrap) , ya bad idea but it works and I think there is no way to achieve this without doubling memory allocation. – bogdanoff May 08 '22 at 15:07

0 Answers0