Can you determine how many bytes are present in an accepted Java socket?

Question

I'm writing a file transfer protocol server in Java that is utilizing the HTTP/1.1 standard outlined in RFC2616.

After the server accepts a connection, I'm trying to extract the HTTP request message. I want to do it in such a way that I'm not assuming the entire message will be sent through a single send operation. I feel like the only way to reliably do this is to track how many bytes are available for reading but I don't quite see anything in the socket API that enables me to do this.

Answer 1

Reading an HTTP/1.1 request is a two step operation:

1. read the header fields
2. use the information obtained by reading the header fields in order to read the body

The format of the request is covered by RFC 9112, 2.1 Message format:

  HTTP-message   = start-line CRLF
                   *( field-line CRLF )
                   CRLF
                   [ message-body ]

start-line refers to the line including the method, url and protocol version, then a CRLF (\r\n) and then optional field-lines. You're done reading the headers fields when you read two successive CRLF.

Reading the headers, you'll know whether a body is present, and how it's framed (meaning how to read it). There are three possibilities:

1. a content-length: header field tells you exactly how many bytes to read after the last CRLF
2. a transfer-encoding: header field tells you how the body is encoded, and which method to use to read it. The one method used in practice is chunked. See section 7.1 of RFC 9112 for a description of that format.
3. neither header fields are present, meaning that there is no body associated with the request (note that this only applies to requests, it's different for responses -- see section 6. for more details)

Finally, you'll have noticed that I've used RFC 9112, not RFC 2616. That's because 9112 is part of the series of RFCs that have obsoleted 2616. See this blog post and this one for more details.

Answer 2

A socket cannot know how many bytes will be available, but RFC2616 offers a solution. As you can read in section 8.1.2.1:

In order to remain persistent, all messages on the connection MUST have a self-defined message length (i.e., one not defined by closure of the connection), as described in section 4.4.

Answer 3

Can you determine how many bytes are present in an accepted Java socket?

The short answer is No. There is nothing in the Socket API that will tell you the overall stream length.

Why? Because if you look at the TCP protocol (as a typical example of a stream transport protocol) there is nothing in the protocol that transmits the stream length ... at the start. Indeed, you only know what the TCP stream length when (and if) the receiving end gets a FIN message. (Refer to the TCP spec or Wikipedia for more details.)

This means that if you need to know the number of (file) bytes that will be sent at the start, you need to handle this at the application protocol layer.

• If you use HTTP/1.x, you will need to deal with the 3 variants that Fredrik describes in his answer. And note that in the 3rd one, you won't be able to get the size ahead of time at all. (Of course, if you control the server side, you can ensure that doesn't happen ...)

  Note: if you are trying to implement this directly at the socket API level, then the onus is on you to read, (correctly) understand and (correctly) implement the subset of HTTP/1.x that you need. Some of the spec is rather complicated. And it gets potentially even more complicated with newer versions of HTTP ... which are increasingly used by browsers, servers, content delivery networks and so on.

  So my advice would be: Don't do it! Use an existing HTTP protocol implementation on both the client and server sides. You will save yourself a lot of time and (mostly) nugatory effort.

• If you change your mind about using HTTP, you can basically do what you want. But to achieve the goal of getting the file size at the start of a transfer, the sender will need to send it as part of your custom application protocol.

---

Non-solution:

Once you get an InputStream from a Socket you will see an available() that tells you how many bytes can be read without blocking. That does NOT tell you the length of the entire stream. Indeed it isn't necessarily even an accurate measure of the number of bytes available. It is best to ignore it and use a (constant) fixed sized buffer for reading. A socket InputStream.read(byte[], ...) call will return whatever bytes are currently available (up to the buffer size). It will only block if there are zero bytes currently available ... and the stream hasn't been closed.

(The available() method is pretty much useless. Most valid use-cases can be handled better in other ways; e.g. by using a Selector.)

Answer 4

When you create a java.net socket or a URLConnection class you must get a stream class from one of its methods after doing its settings. Then you obtain the stream class you can use a method available() from a stream class. NB: one or two stream class's of java.io don't have that method but can be cast to an inputstream our outputstream that has available() , if you need wait() from the stream you will need to also wrap in an interrupted exception. For counting http packets and headers and frames see https://docs.oracle.com/javase/7/docs/api/java/net/DatagramSocket.html