Java: Not everything put into socket is coming out

Question

I've very new to JAVA and socket programming and I'm developing a distributed system of clients, servers and notification systems for a Uni assignment. There are a total of 3 processes running for this program, each with multiple threads. The notification systems generate notifications, sends them to the server which then distributes them to clients.

When a new client connects to the server I need them to pull down all notifications for services they have subscribed to. To do this I am sending three serialised JSON Object fetch requests (one for each notification severity) to the server through a socket. The server always receives the first two of these JSON objects but almost never the third.

Code for client sending fetch requests:

out = new PrintWriter(sd.getSocket().getOutputStream(), true);

            // First send a fetch for all severities
            for (Severity severity : Severity.values()){                
                JSONObject fetchRequest = new JSONObject();
                fetchRequest.put("sType",utils.config.MessageType.FETCH.toString());
                fetchRequest.put("sSeverity",severity.toString());
                fetchRequest.put("lSentTS",System.currentTimeMillis());
                sd.getSocketLock().lock();
                    //System.out.println(fetchRequest.toString());
                    out.println(fetchRequest.toString());
                sd.getSocketLock().unlock();
            }

Code for server receiving JSON object:

while (true) {     
                JSONObject newMessage = utils.socket.getJSONObject(socket);
                String messageType = (String) newMessage.get("sType");
                boolean isContinue = true;
                // Determine message type and handle
                switch (utils.config.MessageType.valueOf(messageType)){
                    case SERVICE:
                        isContinue = handleServiceMessage(newMessage);
                        break;
                    case NOTIFICATION:
                        isContinue = handleNotification(newMessage);
                        break;
                    case FETCH:
                        isContinue = handleFetchRequest(newMessage);
                        break;
                    default:
                        System.err.println(idServiceStr + "Invalid messageType '" + newMessage.get("sType") + "'");
                        System.err.println(idServiceStr + "Continuing to listen...");                       
                }

If I sleep the client thread between sending JSON objects then it works. I have also tested with only one thread running in each of the client and server processes.

Whats going on here? The sockets should be running over tcp so nothing should get dropped and I would of thought I would get an exception if the socket buffer overflowed and dropped one.

Edit:

Each client will have a thread for sending data down the socket and a thread for listening. Both sit below a thread that initialises them. sd stands for sharedData and is an object that contains the socket, a lock and a few other things that shouldn't be relevant to the problem.

MessageType is an enum.

Code for getJSONObject

static public JSONObject getJSONObject(Socket socket) throws IOException, JSONException{

        InputStream in = socket.getInputStream();
        BufferedInputStream bis = new BufferedInputStream(in);

        ByteArrayOutputStream inBuf = new ByteArrayOutputStream();
        int result;             
        while((result = bis.read()) != -1 && result != (byte)'\r') {                    
            inBuf.write((byte) result);                 
        }                

        return new JSONObject(inBuf.toString());
    }

Please post a MCVE - [Minimal, Complete, and Verifiable example](http://stackoverflow.com/help/mcve). Your example is not complete, it is referring to other things like `sd` and `utils` and we have no idea what they are, and how to run your code. — Erwin Bolwidt, Sep 10 '16 at 06:32
@ErwinBolwidt I have edited the code to explain sd & add the code for getJSONObject. I can't go posting too much code because this is a uni assignment but I apologise for forgetting to add getJSONObject which could be relevant. — Kyle Renfrey, Sep 10 '16 at 06:54
The *minimal* part is also important. Please read [the link that I posted](http://stackoverflow.com/help/mcve) Either "Restart from scratch" or "Divide and conquer." to get a minimal code snippet. See it from a different perspective: Sockets work fine. The problem is in your code. So create a minimal code snippet that reproduces the problem and then let people have a look a it. — Erwin Bolwidt, Sep 10 '16 at 07:12
Well if sockets do 'work fine' then thats really all I wanted to know, because that means I'm probably doing something wrong with threading since calling a Thread.sleep(10) after each iteration fixes the problem. Also I understand where your coming from with the MCVE framework but I simply don't have time to investigate this bug to that extent. I'm really after a 'your doing something wrong' or 'yeah sockets are dodgy' response. — Kyle Renfrey, Sep 10 '16 at 07:17

score 0 · Answer 1 · answered Sep 10 '16 at 07:00

0

Usually, there is a buffer behind IO stream. So you need to flush it, after you finished to send your data. Try out.flush(); when cycle ends.

answered Sep 10 '16 at 07:00

a.yekimov

316
1
8

I forgot to mention I did try that earlier and it doesn't help. I've tried flushing after every iteration of the loop and also only after it finishes. – Kyle Renfrey Sep 10 '16 at 07:04
In the past, I faced the similar problem. There was async socket behind the scene and my InputStream wasn't blocked on `read()` when no data in it. What kind of sockets are you using? Maybe you can show, how it was created? – a.yekimov Sep 10 '16 at 10:20
I'm using the standard java.net.Socket running on 'localhost' and port 2000. I don't believe the issue would be from another thread reading the socket given the consistency of it only occurring (or so I've observed) when I push 3 or packets in quick succession. I've decided to avoid the issue for now but thanks for the suggestions – Kyle Renfrey Sep 10 '16 at 12:37

score 0 · Answer 2 · answered Sep 10 '16 at 07:42

Some small points first:

Avoid PrintWriter here, it suppresses Exceptions a bit too much - IMHO. But I'll use it here. Use a defined charset so the software function between a Greek Windows and a Dutch one (very important).

out = new PrintWriter(
    new OutputStreamWriter(sd.getSocket().getOutputStream(),
        StandardCharsets.UTF_8),
    true);

return new JSONObject(inBuf.toString("UTF-8"));

Read the \n:

int result;             
while ((result = bis.read()) != -1) {
    if (result == '\r' || result == '\n') {
        if (result == '\r') {
            reuslt = bis.read(); // `\n`
        }
        break;
    }                    
    inBuf.write((byte) result);                 
}

And my guess is the following is more safe:

int result;             
while ((result = bis.read()) != -1) {
    inBuf.write((byte) result);                 
}                
return new JSONObject(inBuf.toString("UTF-8").trim());

but you are keeping the socket open and never close (also not on the reader and writer).

About the error: if the above did not remove the error magically, you must build in logging code. Especially the Socket class has many states (KeepAlive, closed).

Java: Not everything put into socket is coming out

2 Answers2