0

I am trying to debug an issue we are having with some embedded devices when they are downloading Firmware over-the-air and closing connections too early, failing to get all data.

I tried downloading the file with curl and wget but it gets downloaded without problems and the checksums match.

The only "hint" that I could find is when I run curl with the verbose option (-v). I created a file with dummy content, only containing the word dummy many times over.

When I run curl -v http://myserver/dummy.txt I get this response:

dummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummydummClosing connection #0 ydummydummydummydummydummydummydummydummydummydummydummydummydummy

Why is there the "Closing connection #0" before the end of data?

The apache version is 2.2, we also have HAProxy in front of it, but I get the same "issue" if I curl locally on the machine.

markz
  • 173
  • 1
  • 4

1 Answers1

1

TLDR: what you see is only a display artefact on your terminal and no indication of any issue with either your webserver or HAproxy closing connections too early.

I think that reeks of the counter intuitive effects of output buffering.

Default behaviour of Linux glibc is to do line buffering when standard output is sent to a TTY. That will buffer the output and only display a block of data on your terminal when either:

  • an end-of-line is observed
  • the output buffer is full
  • the program terminates.

It wouldn't surprise me if the "Closing connection" message is sent to standard error which gets displayed immediately but the last bit of content from your URL only comes out of the buffer when curl terminates.

You'll probably see a more intuitive and sequential result when you change the buffering with stdbuff and make the error and output modes unbuffered: 

stdbuf -o0 -e0 curl -v http://myserver/dummy.txt

But also when you change your data to include more new-lines:

dummy
dummy
dummy
...
HBruijn
  • 77,029
  • 24
  • 135
  • 201
  • Thank you HBruijn, it is exactly as you described. stdbuf or a newline character displays the flow correctly – markz Jun 13 '18 at 12:48