Parse an infinitley long input in Java

Question

I wrote a "push server" that, when a client opens a connection, it remains open while the server writes to the stream. I would like to know how to write a client in Java that reacts to commands as it recieves them through this infinite lengthed response.

The problem is I have no idea where to start. If you could point me in the direction of a class where I could read the Javadoc, that would be great. I would like it to be able to respond instantly, so threads are probably a must.

Thank you so so much for any help!

Edit: By infinite length I mean unknown length. The word infinite was used to strenghten the point that the connection is usually never closed by the server. Not a whole lot of data is being transfered.

Another Edit: Maybe it would help to point out that this is an HTTP server running a PHP script.

By "infintely long" I hope you mean "unknown length". Please don't clog the Internet for the rest of us with infinite data! — Ted Hopp, Dec 27 '12 at 17:30
How is this stream formatted? Is it of commands in XML or non text based? It might be worth looking into RMI or similar. — , Dec 27 '12 at 17:34
@ptay89 - Each individual command is seperated by a newline character "\n" and each command in itself is formatted as "_command arg1#arg2#arg3..._" — Osmium USA, Dec 27 '12 at 17:36
http://docs.oracle.com/javase/tutorial/networking/sockets/index.html — JB Nizet, Dec 27 '12 at 17:36
Ok, well sockets seem like a reasonable solution. Parsing each line and running relevant code for the command. — , Dec 27 '12 at 17:38

Alex · Accepted Answer · 2012-12-27T18:40:41.790

7

If they're simple commands separated by newlines, then it should be fairly simple to roll your own client:

EDIT: HTTP isn't the optimal protocol for keeping a long-lived connection open and you might run into timeout issues. The code below is untested but should give you a good starting point.

EDIT 2: As pointed out in the comments, the thread pool may or may not be necessary or useful here. You should note that using threads won't magically make things faster or increase the amount of requests you can handle. The benefit of using a thread pool is to allow the client to immediately receive new commands while a previous command is still executing. That might be important if processing the command requires some I/O task, e.g. calling out to another server, but in your situation it might be overkill.

ExecutorService svc = Executors.newCachedThreadPool();

URLConnection c = 
    new URL("http://192.168.1.122/push/out.php?nduid=1").openConnection();
c.setReadTimeout(30 * 60 * 1000); // Set the read timeout to 30 minutes
c.connect();
InputStream is = c.getInputStream();
BufferedReader r = new BufferedReader(new InputStreamReader(is, "UTF-8"));

String cmd = null;
while ((cmd = r.readLine()) != null) {
  svc.execute(new Runnable() {
    public void run() {
      processCmd(cmd); // work happens here in a different thread.
    }
  });
}

is.close();

The key here is that the input stream returned from the socket remains open as long as the underlying TCP connection remains open. The client can just keep reading from the stream, and the readLine call will simply block as necessary until new data arrives. When readLine returns null, that means the underlying input stream encountered end-of-file, which in turn means that the server closed the socket.

Once you have a full line of data, you can submit it to the thread pool to be parsed and executed in a different thread. Breaking the work up in this way allows you to focus on parsing and executing a command from a single line of text, which is much more manageable than trying to process an arbitrarily large stream of data.

edited Dec 27 '12 at 18:40

answered Dec 27 '12 at 17:43

Alex

13,811
1
37
50

That's because `"192.168.1.122/push/out.php?nduid=1"` is a URL, not a hostname. The hostname in this case is "192.168.1.122" - but if you're using HTTP then raw sockets is not the best way to go on the client side. – Alex Dec 27 '12 at 18:01
You're solution seems really good with the threads and all. Is there a way to send the HTTP request over the socket connection? – Osmium USA Dec 27 '12 at 18:03
1

Btw, this solution won't scale (hint: creating a thread per request is a bad idea) – miniBill Dec 27 '12 at 18:04
I agree. There is nothing in the OP that dictates a thread per request, or a thread pool. In fact it seems the requests will be extremely sporadic. – user207421 Dec 27 '12 at 18:06
@miniBill the threads should be cached and re-used by the pool. The thread pool may or may not be necessary depending on the processing time of a request relative to the amount of time between requests. – Alex Dec 27 '12 at 18:11
The client really doesn't need to be scalable. The server only sends something every half second, and even then most of the data is ignorable. Only one client will be run per computer so I don't think a short and sweet thread per command is the right way to go here (If that's what your're asking, @minibill ) – Osmium USA Dec 27 '12 at 18:11
I realize http isn't the best way to do this but until it needs to be scaled, the server is completley capable of using php for this. But I do agree - raw TCP sockets are required for this on a large scale. – Osmium USA Dec 27 '12 at 18:14
Good, good, just wanted to make sure you knew that :) If your architecture has a good separation between processing and transmission then when you want to scale you can simply change the actual code needed to transfer the data :) – miniBill Dec 27 '12 at 18:20

score -3 · Answer 2 · answered Dec 27 '12 at 17:39

-3

If you open a connection to the server, that check the server and use a while(server.isSending) { respond();} and then close the connection?

Example:

//Connection to the server
openConnection();

while(server.isSending()) {

    //Here can you respond in any way
    respond();
}

You would probably have to use an InputStream aswell.

I´m not so familiar with working on InternetConnections, but I hope that you got on the right track

answered Dec 27 '12 at 17:39

MAA

93
3
13

3

I can't see that this answer gets us very far. You need to at least disclose what your magical isSending() method is supposed to do. If you're not familiar with the topic it's hard to see why you are answering at all really. – user207421 Dec 27 '12 at 17:44
In @Mylleranton's defense, he included more code than the OP. Still, this answer isn't very helpful. – Jesse Webb Dec 27 '12 at 17:59
@JesseWebb Including more code than the OP doesn't add any value unless the code does, which it doesn't. It's just a generic answer with no actual content. The guesswork about 'probably' having to use an InputStream is particularly futile. – user207421 Dec 27 '12 at 18:04
@EJP The creator of the thread asked for guidance, and with my current knowledge I have, I gave him what I know. The isSending() method could be a InputStream method; int bytesRead = 0; `while(bytesRead = in.read(buffer)) >= 0)` – MAA Dec 27 '12 at 18:15
@Mylleranton We can see what you did. That alone doesn't make it useful. The code you have just posted is incorrect. It's not getting any better. If you have some concrete advice that you know works, by all means provide it. If you're not familiar with the topic you should be reading rather than posting. – user207421 Dec 28 '12 at 09:06

Parse an infinitley long input in Java

2 Answers2