13

I'm building a simple WebSocket application that transfers binary snapshots of the current canvas to other listeners.

The current canvas snapshot is sent using WebSocket as:

var image = context.getImageData(0, 0, canvas.width, canvas.height);
var buffer = new ArrayBuffer(image.data.length);
var bytes = new Uint8Array(buffer);
for (var i=0; i<bytes.length; i++) {
    bytes[i] = image.data[i];
}
websocket.send(buffer);

Trying to render the data on the receiving end as:

var bytes = new Uint8Array(blob.size);
var image = context.createImageData(canvas.width, canvas.height);
for (var i=0; i<image.length; i++) {
    image[i] = bytes[i];
}
context.drawImage(image, 0, 0);

The blob is received correctly but the image is still not rendered.

Any idea ?

Arun Gupta
  • 3,965
  • 5
  • 31
  • 39

5 Answers5

12

Setting the binaryType attribute of WebSocket right after connection initialization did the trick. This can be done as:

var wsUri = "ws://localhost:8080/whiteboard/websocket";
var websocket = new WebSocket(wsUri);
websocket.binaryType = "arraybuffer";

This would allow both text and binary data to be transfered.

Arun Gupta
  • 3,965
  • 5
  • 31
  • 39
  • 1
    Yes because the binaryType is by default set to 'blob', when you expect to receive ArrayBuffer you have to set it explicitly. – bilal.haider Feb 05 '14 at 16:25
1

I guess you might have already seen this page.

http://www.adobe.com/devnet/html5/articles/real-time-data-exchange-in-html5-with-websockets.html

find "Receiving a binary message" on that page or better read whole page.

I think its better to put debug at onmessage events 1st line and inspect event.data object. As you are saying are able to capture binary data and send it to server it removes doubt that your browser might not be supporting binary data over web socket.

Now my only doubt is "is server really sending binary data ?" can you just try creating a visible img html element and assigning received data to src prop to that element just to check data ? don't know if this can help websocket for binary transfer of data & decode

Community
  • 1
  • 1
Akshay Ransing
  • 105
  • 7
  • I've begin to doubt that as well and following up internally with our WebSocket team. My complete code is available at: https://github.com/arun-gupta/javaee7-samples/tree/master/websocket/whiteboard – Arun Gupta Nov 16 '12 at 17:23
0

http://www.w3schools.com/tags/canvas_createimagedata.asp?output=print

Its should be image.data[] not image[]

akshay ransing
  • 9
  • 1
0

your bytes variable seems empty you used first type of constructor from http://www.javascripture.com/Uint8Array or https://developer.mozilla.org/en-US/docs/JavaScript_typed_arrays/Uint8Array where each element is assigned to zero.

Akshay Ransing
  • 1
  • 1
    Seems like blog.size is correct but how would Uint8Array need to be initialized with blob in this case ? – Arun Gupta Nov 15 '12 at 07:05
  • you have already did it in code you provided while taking snapshot :) – Akshay Ransing Nov 15 '12 at 07:14
  • try 2nd type of constructor new Uint8Array(blob); i dont like loop :) – Akshay Ransing Nov 15 '12 at 07:15
  • also do console.log(blob) and check browsers console for its type. It will give clear idea about type of Array it is so that you can convert it to Uint8Array; – Akshay Ransing Nov 15 '12 at 07:46
  • console.log(blog) prints "[object Blob]". Initializing as: var bytes = new Uint8Array(blob); console.log(bytes.size); prints 0. That's why I was initializing using the loop. Any thoughts ? – Arun Gupta Nov 15 '12 at 15:54
0

your final code should look like this

var bytes = new Uint8Array(blob);
var image = context.createImageData(canvas.width, canvas.height);
for (var i=0; i<bytes.length; i++) {
    image.data[i] = bytes[i];
}
context.drawImage(image, 0, 0);
Akshay Ransing
  • 1
  • 1
  • if this dont work please let me know ur browser application and version and a bigger code block from var blob is alive. i will prefer chrome :) – Akshay Ransing Nov 15 '12 at 17:01
  • try context.putImageData(image,0,0); – Akshay Ransing Nov 15 '12 at 17:08
  • Grabbing ImageData from one canvas using getImageData and rendering it on another canvas using putImageData in the same browser works fine. But receiving blob over WebSocket, converting to Uint8Array, copying the bytes as shown above, and then display the image using putImageData or drawImage clears up the canvas instead of rendering it correctly. Also read the blog at http://blog.digitalbackcountry.com/2012/01/dealing-with-binary-data-from-a-canvas-object-using-javascript-typedarrays/ but stil the same result. – Arun Gupta Nov 16 '12 at 00:26
  • The blob received from WebSocket endpoint returns false for "blob instanceof ArrayBuffer". "typeof blob" prints "object", "Object.prototype.toString.call(blob)" prints "object Blob". How can this blob be rendered as an image ? – Arun Gupta Nov 16 '12 at 01:40