1

If I run the following:

sed 's/$/\r/g' <<EOF | netcat "www.google.com" 80
GET / HTTP/1.1
Host:www.google.com
Connection: close

EOF

I get the usual headers, followed by a hexadecimal number before the response body I haven't seen before (in this case, 5172, although another example is 51C8):

HTTP/1.1 200 OK
Date: Thu, 09 Sep 2021 20:03:53 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Set-Cookie: ...
Accept-Ranges: none
Vary: Accept-Encoding
Connection: close
Transfer-Encoding: chunked

5172
<!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en-GB"><head>...

In presumed this was content length, although the response body length did not match this by a large order.

What is this number?

Nick Bull
  • 9,518
  • 6
  • 36
  • 58
  • 1
    Is because the transfer encoding, the value is the size in bytes, see this too https://stackoverflow.com/questions/37092458/chunk-size-appears-on-browser-page, remember every chunk needs to be preceded by its size https://en.wikipedia.org/wiki/Chunked_transfer_encoding. – kip Sep 09 '21 at 20:45
  • 1
    "Each chunk starts with the number of octets of the data it embeds expressed as a hexadecimal number in ASCII followed by optional parameters (chunk extension) and a terminating CRLF sequence, followed by the chunk data. The chunk is terminated by CRLF." – kip Sep 09 '21 at 20:47
  • @kip That is very interesting, I will read more about this. Is there a way to remove these chunk sizes? – Nick Bull Sep 09 '21 at 22:40
  • Just use `curl https://www.google.com/` instead to fetch the page. `curl -D - https://www.google.com` if you want the headers too. – Shawn Sep 10 '21 at 01:11
  • @Shawn Sure, I know about `curl`. This is a bit of a challenge to try to use only preinstalled modules. For now I'm gonna try looping and reading until the previous chunk count. I'll post the answer once I get back round to this – Nick Bull Sep 10 '21 at 15:15

0 Answers0