I'm running a server on a Raspberry Pi Pico W that serves a webpage and some data. I'd like the http response to be consumable by a variety of clients (wget, curl, python requests, browsers) but I'm having trouble getting my HTTP response understood. Browsers handle it ok (Chrome parses the response headers, Firefox does not), curl and python requests think there are no HTTP headers and close the connection immediately, and wget receives the data but thinks the headers are part of the response body.
This snippet writes the HTTP response
WiFiServer server(80); // server on port 80
void loop(){
WiFiClient wificlient = server.available(); // Listen for incoming clients
// If a new client connects,
if (wificlient) {
HttpClient client(wificlient, WiFi.localIP());
net_time = millis();
net_prev_time = net_time;
while ((client.connected() || client.available()) && (!client.endOfBodyReached()) &&
(net_time - net_prev_time <= net_timeout))
{
net_time = millis();
if (client.available()) {
char c = client.read(); // read a byte
Serial.write(c);
httpbody += c;
if (!client.available()) { // client is done, respond
if (httpbody.indexOf("GET /api/last") == 0){
Serial.println("Serving api request...");
client.println("HTTP/1.1 200 OK");
client.println("Content-type: text/plain; charset=utf-8");
client.println("Content-length: " + String(last_str.length()));
client.println("Content-Disposition: inline");
client.println("Connection: close");
client.println("");
client.println(last_str);
client.println("");
break;
}
}
}
...
}
In the above client is an instance of HttpClient from the ArduinoHttpClient library (via the arduino-pico core)
And when requesting with wget it says "No headers, assuming HTTP/0.9".
$ wget -O - http://10.0.0.27/api/last
--2023-02-24 22:09:34-- http://10.0.0.27/api/last
Connecting to 10.0.0.27:80... connected.
HTTP request sent, awaiting response... 200 No headers, assuming HTTP/0.9
Length: unspecified
Saving to: ‘STDOUT’
-
HTTP/1.1 200 OK
Content-type:text/plain; charset=utf-8
Content-length:136
Content-Disposition: inline
Connection: close
3006, MHZTemp (C): 17, Temp (C): 22.40, Humidity (%): 30.00, Feels like (C): 21.48, CO2 (ppm): 653, Dust (pcs/283ml): 599.09, GPS Fix: 0
-
Using curl, it seems like a similar assumption is made, since I get "Received HTTP/0.9 when not allowed".
How should I format the HTTP response so that it is correctly parsed by these clients?
Update:
Substituting WifiClient instead of HttpClient solves the issue, and both wget and curl no longer complain of missing headers or HTTP/0.9. Looking at the traffic in wireshark, it appears that when WifiClient is responding it sends a TCP [PSH, ACK] packet with the HTTP status code whereas and then after wget or curl respond with a TCP [ACK] WifiClient responds with the rest of the data. HttpClient was not sending [PSH, ACK] (probably because it isn't meant to) and curl closes the connection in response to more data.
In the following is the wireshark traffic with different configurations. 10.0.0.27 is my Pico server and 10.0.0.193 is my desktop machine (the requestor).
Requesting with curl and responding with WifiClient (expected behaviour):
Requesting with curl and responding with HttpClient (connection gets closed):
So I think the PSH packet is the answer. WifiClient knows to send it while HttpClient isn't meant for this use and doesn't send it.
