3

I have a WebSocket server implemented in Java. When a client connects I want to send an image over this connection for the client to use in a canvas element. I have come up with the following server code:

public void onOpen(Connection connection) {
    try {
        BufferedImage image = ImageIO.read(new File("image.jpg"));
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        ImageIO.write(image, "jpg", baos);
        byte[] byteArray = baos.toByteArray();
        connection.sendMessage(byteArray, 0, byteArray.length);
    } catch (Exception e ){
        System.out.println("Error: "+e.getMessage());
    }
}

The client-side Javascript looks like this:

onmessage : function(m) {
    if (m.data) {
        if (m.data instanceof Blob) {
            var blob = m.data;

            var bytes = new Uint8Array(blob);
            var image = context.createImageData(canvas.width, canvas.height);
            for (var i=0; i<bytes.length; i++) {
                image.data[i] = bytes[i];
            }
        }
    }
}

The connection works and the data is sent (blob.size has the correct value), but the image is not drawn onto the canvas. Firefox gives me the error message "TypeError: Value could not be converted to any of: HTMLImageElement, HTMLCanvasElement, HTMLVideoElement.".

I am aware of the fact that this using WebSockets is not the best way to send an image to the client. After sending the image the WebSocket is only used to send text messages.

What do I need to change for the image to be sent an applied to the canvas?

Resources used:

how to convert image to byte array in java?

Receive Blob in WebSocket and render as image in Canvas

Community
  • 1
  • 1
Waboodoo
  • 508
  • 4
  • 17
  • Did you resolve your problem ? I am working on a similar project, take a look at this video : https://blogs.oracle.com/arungupta/entry/collaborative_whiteboard_using_websocket_in minutes 6:15, I also read that jpeg needs to be decoded : http://stackoverflow.com/questions/16194908/how-can-i-create-a-canvas-imagedata-array-from-an-arraybuffer-representation-of – Mehdi Karamosly Sep 11 '13 at 03:35
  • hey, what was it that the Connection class that you use? From what library is it? I'm questioning it here: http://stackoverflow.com/questions/23557300/what-connection-is-this –  May 09 '14 at 08:48
  • It is from the org.eclipse.jetty.websocket.WebSocket package, which I include from this jar-file: http://mvnrepository.com/artifact/org.eclipse.jetty/jetty-websocket/8.1.0.RC5 Hope that helps – Waboodoo May 10 '14 at 09:50

1 Answers1

2

Try converting the image to base64 before sending, for example:

function drawImage(imgString){
    var canvas = document.getElementById("canvas");
    var ctx = canvas.getContext("2d");

    var image = new Image();
    image.src = imgString;
    image.onload = function() {
        ctx.drawImage(image, 0, 0);
    };
}

Here's a link on how to convert the image to base64 in Java

Community
  • 1
  • 1
Jack burridge
  • 472
  • 7
  • 21
  • That looks good, I will see into it as soon as I have time again. Thanks :) – Waboodoo Dec 20 '13 at 10:15
  • Yes, this works! However I used a different class for the Base64 encoding than the one you linked me to. The one I used didn't automatically include the "data:image/png;base64,"-prefix so I had to add it myself. I don't know if the one you suggest does this. If not it would have to be added separately. – Waboodoo Dec 20 '13 at 21:59
  • 1
    base64 encoding and decoding has certain processing overheads as well as the resulting data is around 30% larger. Is there a better solution? – Roman Gaufman Feb 12 '16 at 10:39
  • what is imgString? – Teocci Jul 21 '16 at 07:40