Best way to display high-speed data from a telnet interface on a web view?

Question

I'd like to display data coming off a telnet interface on a web view. I have a daemon running which reads CAN bus data and produces about 500 lines of ~40 characters per second onto a telnet port. It's a few messages running at 100 Hz and most others running at 10 or 5 Hz, so in sum it's about 500 / second. I want to grab the latest values in each packet and display those on a web page. The web page is loaded locally (not via HTTP) and the daemon might be on a different host, so there is cross-domain communication.

Here's what I tried and failed to do:

1. Use XMLHttpRequest. I can open the connection and read the data, but I can't clear the responseText field of prior values when I get the onprogress event. I can't afford to parse the responseText for the latest value as this grows very quickly. I will also run into memory issues, so this is a no-go.
   
2. WebSockets and Socket.IO: neither has proved successful in connecting to a telnet interface because it expects HTTP at first to then convert to direct socket.

So my question is, how do I best accomplish this? Some options I see, but I'm sure there are more:

1. Add HTTP to socket conversion in the canlogserver daemon I'm trying to attach to. How? (it's open source C so I could add to it)
2. Write a PHP interface which attaches to the daemon via telnet and can pump data back to web view via HTTP. This seems grossly inefficient with multiple trips through the IP stack.<
3. Anything else on the JS client code to circumvent my buffer issue and read messages from telnet server, display data and then dump the buffer? I need to make sure I get all the messages once the socket is opened, so opening, closing, reopening won't work since that will mean messages are lost.

Thanks,

Tim

Answer 1

Actually, WebSockets have an HTTP-like handshake and then have some framing for each message (they are not raw sockets after the handshake).

However, you could use websockify to bridge between a WebSocket client (browser) and a plain TCP socket. Websockify also enables you to send/receive binary data to/from the TCP server even when the WebSocket protocol (e.g. Hixie) or the browser doesn't support binary types directly (e.g. Typed Arrays).

noVNC uses websockify to be able to connect directly to a VNC server that does not yet have builtin WebSocket support. Websockify even has a small test included that demonstrates connect a simple browser-based terminal emulator to a telnetd server. Disclaimer: I made websockify and noVNC.

Answer 2

Best solution is to use WebSockets if you want pure HTML5 without any plugins. Bear in mind of supported browsers for such technology, you can find list of supported browsers and protocol version here.

WebSockets have to do handshake when connection is established, it is usual http lines. After that all messages should have framing, when receive and when send. Browser does it automatically, but you have to implement for your daemon this functionality.

There is ready solutions, depends of what language you use.

If you wish implement WebSockets protocol by your self, check sources of Java implementation for example, and use official specification: RFC6455

I believe that WebSockets is the best way to go. Java / Silverlight / Flash might be other solution, but it requires more efforts and might be not easy scalable like WebSockets implementation, that will be just processing messages and creating DOM elements like DIV's in another DIV container with received messages.