Using websockets through the simple websockets for webgl asset in unity3d, can connect but can't transmit messages

Question

Just having a problem on my mac trying to send strings over web sockets using this https://www.assetstore.unity3d.com/en/#!/content/38367

Lots of adapted code below from here mainly http://www.codepool.biz/how-to-implement-a-java-websocket-server-for-image-transmission-with-jetty.html and the web socket sharp echotest example.

I can connect but there is no sign of strings in my Jetty server console window (on a ws server running in java(eclipse)).

I’m basically just trying to send a “1” to my server over a websocket connection with the unity editor (5) at the moment, to prompt the server to start sending PNG files encoded as byte arrays, so I can put them back together in a C# script and apply them to a texture.

this is the script, I want to attach it to a game object like a plane or a cube and display the updating images sent over the web socket from my Jetty server, but at the moment I'm just stuck trying to send a message and see it pop up in my eclipse console window.

using UnityEngine;
 using System.Collections;
 using System;

 public class socketTexture : MonoBehaviour {

     // Use this for initialization
     IEnumerator Start () {
         WebSocket w = new WebSocket(new Uri("ws://192.168.0.149:8080/"));
         yield return StartCoroutine(w.Connect());
         Debug.Log ("Connected");
         w.SendString("I'm client");
         w.SendString("1");
         while (true)
         {
             byte[] reply = w.Recv();
             if (reply != null)
             {
                 Debug.Log ("Received: "+reply);
                 var tex = new Texture2D(300, 300, TextureFormat.PVRTC_RGBA4, false);
                 // Load data into the texture and upload it to the GPU.
                 tex.LoadRawTextureData(reply);
                 tex.Apply();
                 // Assign texture to renderer's material.
                 GetComponent<Renderer>().material.mainTexture = tex;
             }
             if (w.Error != null)
             {
                 Debug.LogError ("Error: "+w.Error);
                 break;
             }
             yield return 0;
         }
         w.Close();
     }
 }

...And the relevant code from the jetty server, but this works, I've tested it with some javascript and I can load the PNGs back into the browser window, so I'm definitely doing something wrong in Unity

@OnWebSocketMessage  //part request from websocket client (remote browser)
     public void onMessage( String message) {
         System.out.println("message");
         if (message.equals("1") || message.equals("2") || message.equals("3") || message.equals("4") ) {
             System.out.println("Part " + message + " joined");  
             System.out.println( UIMain.usersPath + "/" + message + ".png" );
             final String testVar = ( UIMain.usersPath + "/" + message + ".png" );
             task = new FileWatcher( new File(testVar) ) {
                 protected void onChange( File file ) {
                     // here we code the action on a change
                     System.out.println( "File "+ file.getName() +" has changed!" );
                     try {            
                         File f = new File(testVar);
                         BufferedImage bi = ImageIO.read(f);
                         ByteArrayOutputStream out = new ByteArrayOutputStream();
                         ImageIO.write(bi, "png", out);
                         ByteBuffer byteBuffer = ByteBuffer.wrap(out.toByteArray());
                         mSession.getRemote().sendBytes(byteBuffer);
                         out.close();
                         byteBuffer.clear();
                         }
                            catch (IOException e) {
                                e.printStackTrace();
                         }    
             }
         };
         Timer timer1 = new Timer(); {
         timer1.schedule(task , new Date(), 40 );
         }

         }
         else if (message.equals( "0")) {
             zerocounter = zerocounter + 1;
             if (zerocounter >= 2) {
                 task.cancel();
             }
         }
         else if (message.equals( "Hi there, client here")) {
             System.out.println( "Client says: " + message );
         }
     }

Any help would be really appreciated, been lurking on here for years, hopefully getting to the stage soon where I can help out others a bit too. Benedict

Edit: This is my console error message in Unity

FormatException: Invalid length. System.Convert.FromBase64String (System.String s) (at /Users/builduser/buildslave/mono-runtime-and-classlibs/build/mcs/class/corlib/System/Convert.cs:146) EchoTest+c__Iterator0.MoveNext () (at Assets/Example/EchoTest.cs:11)

I'm pretty sure the problem arises from websocket sharp for webgl. I need to send the message as a byte array.

My own progress has been to get the PNG file byte code transmitting to Unity via the websocket connection and showing up as a texture! last trick is now to get Unity sending a string back to my server and unfortunately it seems like I can't work out the formatting used by websocket sharp. using websocket.send in javascript it works. when I use websocket.SendString() in websocket sharp the server doesn't see the message. stuck on this one for a few days any help would be appreciated :))) — Benedict Carey, Dec 09 '15 at 16:11
Can you show more about your setup of the WebSocket endpoint on the server? Just the @OnMessage isn't sufficient. Also, what sort of logs do you see? — Joakim Erdfelt, Dec 09 '15 at 16:32
It's pretty much the same still as in the link above http://www.codepool.biz/how-to-implement-a-java-websocket-server-for-image-transmission-with-jetty.html, can't paste much code in the answer box here — Benedict Carey, Dec 09 '15 at 17:33
Add a `@OnWebSocketConnect` and verify that this endpoint is even being used. Your implementation of `@OnWebSocketConnect` could even send a TEXT/BINARY message back to your client. If this doesn't even show activity, then you are not even connected to that WebSocket endpoint. — Joakim Erdfelt, Dec 09 '15 at 18:05
I have achieved a connection with a javascript client and it works fine bidirectionally (the one from Desmond Shaw's implementation basically). I have also managed to transmit byte arrays to my Unity client after the handshake is complete, but I register a connection from that IP on my Java Server and do not receive any subsequent messages from that IP. I haven't worked out how to check on the client end if a message is being sent from my Unity application because I am using this simple websockets for web gl asset which simplifies the scripting somewhat. I can just disconnect and reconnect. — Benedict Carey, Dec 09 '15 at 18:37
I just tested the client with this and it works http://www.websocket.org/echo.html, so it must be a flaw with my server implementation as you say — Benedict Carey, Dec 09 '15 at 18:37
Could just be a misconfiguration of your `Server`. The codepool.biz implementation is ridiculously simple, no contexts, 1 handler, no http support, very inflexible usage. If you change anything you'll have to rethink your `Server` setup, and how you connect. — Joakim Erdfelt, Dec 09 '15 at 20:02

score 0 · Answer 1 · answered Dec 10 '15 at 00:38

OK So Joakim Erdfelt was right, the server was not configured to handle Byte[] messages. Here's what I added to fix it:

@OnWebSocketMessage
public void onMessage(byte[] buffer, int offset, int length) throws UnsupportedEncodingException {
      System.out.println(buffer);
      String sFclientOutStr = new String(buffer, "UTF-8");
        sFclientOut = Integer.parseInt(sFclientOutStr);
      System.out.println(sFclientOut);
        if ((sFclientOut > 0) & (sFclientOut < 500)) {
            System.out.println("Part " + sFclientOut + " joined");  
            System.out.println( UIMain.usersPath + "/" + sFclientOutStr + ".png" );
            final String testVar = ( UIMain.usersPath + "/" + sFclientOutStr + ".png" );
            task = new FileWatcher( new File(testVar) ) {
                protected void onChange( File file ) {
                    // here we code the action on a change
                    System.out.println( "File "+ file.getName() +" has changed!" );
                    try {           
                        File f = new File(testVar);
                        BufferedImage bi = ImageIO.read(f);
                        ByteArrayOutputStream out = new ByteArrayOutputStream();
                        ImageIO.write(bi, "png", out);
                        ByteBuffer byteBuffer = ByteBuffer.wrap(out.toByteArray());
                        mSession.getRemote().sendBytes(byteBuffer);
                        out.close();
                        byteBuffer.clear();
                        }
                        catch (IOException e) {
                            e.printStackTrace();
                        }   
            }
        };
        Timer timer1 = new Timer(); {
        timer1.schedule(task , new Date(), 40 );
        }

        }
        else if (sFclientOutStr.equals("0")) {
            zerocounter = zerocounter + 1;
            if (zerocounter >= 2) {
                task.cancel();
            }
        }
        else if (sFclientOutStr.equals( "I'm client")) {
            System.out.println( "Client says: " + sFclientOutStr );
        }

}

These links helped explain it for me http://www.programcreek.com/java-api-examples/index.php?api=org.eclipse.jetty.websocket.api.annotations.OnWebSocketMessage

http://www.eclipse.org/jetty/documentation/current/jetty-websocket-api-annotations.html

Using websockets through the simple websockets for webgl asset in unity3d, can connect but can't transmit messages

1 Answers1