0

The man page of openssl's SSL_read() call states:

SSL_read() works based on the SSL/TLS records. The data are received in records (with a maximum record size of 16kB for SSLv3/TLSv1). Only when a record has been completely received, it can be processed (decryption and check of integrity). Therefore data that was not retrieved at the last call of SSL_read() can still be buffered inside the SSL layer and will be retrieved on the next call to SSL_read().

Given that:

  • HTTP headers of a single outgoing message can always be sent in one go
  • a single SSL/TLS record can apparently hold 16KB of data, which should be enough for everyone (or at least any non-perverse HTTP request)

Browsers have little reason to divide the headers into multiple SSL records, right? Or are there browsers out there that are so aggressive with regard to latency that they will chop up even these kinds of small payloads into multiple records?

I'm asking this because it would be nice to be able to parse an entire set of HTTP headers from a single read buffer filled by a single succesfull SSL_read() call. If that means denying the odd few (e.g. if only 0.0000X% of all requests), that might be worth it to me.

edit: Alexei Levenkov made the valid point that cookies can be really long. But let's consider then the scenario that cookies are never set or expected by this particular server.

edit2: This question was a little premature. I've meanwhile written code that stores per client state efficiently enough to accept an arbitrary number of SSL records while parsing without incurring a performance penalty of any significance. Prior to doing so I was wondering if I could take a shortcut, but general consensus seems to be that I better play by the book. Point conceded.

Community
  • 1
  • 1
Will
  • 2,014
  • 2
  • 19
  • 42
  • 4
    No, it is not reasonable to assume that. – President James K. Polk Jun 08 '13 at 01:12
  • I think default restriction on max size of all cookies per domain is about 100K... So it is unlikely to fit cookie header alone into 16K block. – Alexei Levenkov Jun 08 '13 at 01:12
  • I hate to think how slow browsing would be if you sent 100k of cookie crud with each request... On the 2G connections I often use with even-slower uplink, I'm thinking maybe a few minutes per request? – R.. GitHub STOP HELPING ICE Jun 08 '13 at 01:14
  • I don't know why you assert that a meaningful statement is meaningless. You can believe what you want, but you need some credible reason for doing so; otherwise you are just being merely credulous. All @GregS is implicitly doing here is asserting that you won't find such a reason, and I agree with him. You are welcome to prove us both wrong, if you can. – user207421 Jun 08 '13 at 10:23

4 Answers4

1

No, cookies alone could be well above 16K.

According to RFC2109 and IE implementation for cookie limits the recommended minimum restriction on cookie size per domain is 80K

RFC 2109 section "6.3 Implementation Limits":

  • at least 4096 bytes per cookie (as measured by the size of the characters that comprise the cookie non-terminal in the syntax description of the Set-Cookie header)

  • at least 20 cookies per unique host or domain name

Alexei Levenkov
  • 98,904
  • 14
  • 127
  • 179
  • Thanks, you have a good point there of course. I have edited my question though to also ask about cookie-less situations. – Will Jun 08 '13 at 01:20
  • The other answers are mostly good as well, but I'm clicking Accept on yours because you were the first to answer with a solid counterexample. – Will Jun 09 '13 at 23:56
1

You cannot make that assumption. Just like recv() can return fewer bytes than requested, so can SSL_read(). It is your responsibility to buffer inbound data, reading in however many records it takes, and then process the buffered data only when you have finished receiving everything you are expecting to receive. You should not be processing the HTTP header until you have received the <CRLF><CRLF> sequence at the end of the headers.

Remy Lebeau
  • 555,201
  • 31
  • 458
  • 770
  • To be 100% sure you handle 100% of all theoretical scenarios, one must indeed do as you say. But wouldn't trying to adhere to that principle in the real world be a bit naive/idealistic? Point in case: AFAIK all major webservers set a hard upper bound to the amount of HTTP header bytes they accept (without waiting further for a ` ` beyond that). – Will Jun 08 '13 at 01:29
  • 1
    I agree with the beginning of your answer, but buffering an unlimited amount of data is probably not a good design. There's no reason you can't process the header lines as your receive them, rejecting any request that exceeds a reasonable line-length limit and only saving the headers you care about. This would convert the whole header-processing phase from unbounded-storage to fixed-storage. – R.. GitHub STOP HELPING ICE Jun 08 '13 at 01:30
  • 1
    @R..: There is actually a reason: headers are unordered. The `Host:` header is necessary to know which virtual server should process the request; until you know that, it is probably impossible to know which headers "you" care about (or at least who the "you" in that sentence is). In an ideal world, the `Host:` header would be the first header, and it usually is, but there is no such requirement or even recommendation in the RFC, afaik. – rici Jun 08 '13 at 04:46
1

Perhaps a more careful statement would be that the assumption is wrong but I would expect that most of the time for the average scenario you would get the behavior you want. I don't know the answer for sure, but experiments could be done to find out.

I'm all for hackers ignoring the rules and abusing protocols for learning, fun, and profit, but otherwise I can see no benefit for restricting yourself to reading a single SSL record. If you wouldn't mind elaborating I am curious.

President James K. Polk
  • 40,516
  • 21
  • 95
  • 125
  • Thanks. You're right, I think. This question was a little premature. I've meanwhile managed to write code that stores state efficiently enough to accept an arbitrary number of SSL records while parsing without incurring a performance penality of any significance. – Will Jun 09 '13 at 23:47
1

Browsers have little reason to divide the headers into multiple SSL records, right? Or are there browsers out there that are so aggressive with regard to latency that they will chop up even these kinds of small payloads into multiple records?

There can be some reasons that lead to splitting the data into multiple records. One of them is the mitigation against the BEAST attack.

You can't generally make the assumption that the data won't be split into multiple records.

I'm asking this because it would be nice to be able to parse an entire set of HTTP headers from a single read buffer filled by a single succesfull SSL_read() call. If that means denying the odd few (e.g. if only 0.0000X% of all requests), that might be worth it to me.

This is not just an SSL/TLS issue, but this would also apply to TCP: not treating the incoming data as a stream will simply lead to bugs and bad code.

Community
  • 1
  • 1
Bruno
  • 119,590
  • 31
  • 270
  • 376
  • 1
    True, but an SSL_read() should return a full SSL record no matter how manner TCP packets it spans. – President James K. Polk Jun 08 '13 at 23:24
  • @GregS, as far as I'm aware, this isn't about TCP packets, but SSL/TLS records ([this patch in the JSSE](http://hg.openjdk.java.net/jdk6/jdk6/jdk/rev/0c65618b5ae1) seems to indicate the application stream is split in 1 + (n+1) records, so you would get `G` yielding one record and `ET / HTTP/1.1 ....` in other records, for example). – Bruno Jun 10 '13 at 09:29
  • Sorry if I wasn't clear, but I was talking about how to process application data on top of SSL/TLS or on top of TCP in general: in doesn't make any more sense to treat application data over SSL/TLS on a per-record basis, than it makes sense to treat application data over TCP (when SSL/TLS isn't used) on a per-packet basis. – Bruno Jun 10 '13 at 09:29