1

I am building a web server from an ESP8266 that will send environmental data to any web client as a web page. I'm using the Arduino IDE.

The problem is that the data can get rather large at times, and all of the examples I can find show assembling a web page in memory and sending it all at once to the client via ESP8266WebServer.send(). This is ok for small web pages, but won't work with the amount of data I need to send.

What I want to do is send the first part of the web page, then send the data out directly as I gather it, then send the closing parts of the web page. Is this even possible? I've looked unsuccessfully for documentation and there doesn't seem to be any examples anywhere.

David Powell
  • 537
  • 1
  • 4
  • 16
  • 1
    If [this](https://github.com/espressif/arduino-esp32/blob/master/libraries/WebServer/src/WebServer.h) is the API you're using then I see there are seperate `sendHeader()` and `sendContent()` functions you can use to break the transmission up and send things in small pieces. – President James K. Polk Jul 28 '19 at 15:24
  • My apoligies, I'm not using a library from espressif. I'm also working on a project that uses an ESP32 which *does* use those libraries, and I got confused. I've removed the espressif references in the question. – David Powell Jul 28 '19 at 15:42
  • I'm not sure that streamFile() will help, but I'll look at it. Something like sendHeader() and sendContent() is more like what I'm looking for. – David Powell Jul 28 '19 at 15:44

3 Answers3

4

For future reference, I think I figured out how to do it, with help from this page: https://gist.github.com/spacehuhn/6c89594ad0edbdb0aad60541b72b2388

The gist of it is that you still use ESP8266WebServer.send(), but you first send an empty string with the Content-Length header set to the size of your data, like this: 

server.sendHeader("Content-Length", (String)fileSize);
server.send(200, "text/html", "");

Then you send buffers of data using ESP8266WebServer.sendContent() repeatedly until all of the data is sent.

Hope this helps someone else.

David Powell
  • 537
  • 1
  • 4
  • 16
1

I was having a big issue and a headache in serving big strings concatenating together with other strings variables to the ESP32 Ardunio webserver with

server.send(200, "text/html", BIG_WEBPAGE);

and often resulted in a blank page as I reported in my initial error. What was happening was this error

E (369637) uart: uart_write_bytes(1159): buffer null

I don't reccommend to use the above server.send() function

After quite a lot of reaserch I found this piece of code that simply works like a charm. I just chunked my webpage in 5 pieces like you see below.

  server.sendHeader("Cache-Control", "no-cache, no-store, must-revalidate");
  server.sendHeader("Pragma", "no-cache");
  server.sendHeader("Expires", "-1");
  server.setContentLength(CONTENT_LENGTH_UNKNOWN);
  // here begin chunked transfer
  server.send(200, "text/html", "");
  server.sendContent(WEBPAGE_BIG_0); 
  server.sendContent(WEBPAGE_BIG_1); 
  server.sendContent(WEBPAGE_BIG_2); 
  server.sendContent(WEBPAGE_BIG_3);
  server.sendContent(WEBPAGE_BIG_4);
  server.sendContent(WEBPAGE_BIG_5);
  server.client().stop();

I really own much to this post. Hope the answer hepls someone else.

After some more experiments I realized it is faster and more efficient the code if you do not feed the string variable into the server.sendContent function. Instead you just paste there the actual string value.

server.sendContent("<html><head>my great page</head><body>");
server.sendContent("my long body</body></html>");

It is very important the when you chunk the webpage you don't chunk html tags and you don't chunk an expression of a javascript code (like cutting in half a while or an if), while chunking scripts just chunk after the semicolon or better between two function declarations.

Pietro
  • 127
  • 6
0

Chunked transfer encoding is probably what you want, and it's helpful in the situation where the web page you are sending is being dynamically created on-the-fly and is also too large to fit into memory. In this situation, you have two problems. One, you can't send it all at once, and two, you don't know ahead of time how big the result is going to be. Both problems can be fixed like this:

String webPageChunk = "some html";
server.setContentLength(CONTENT_LENGTH_UNKNOWN);
server.send ( 200, "text/html", webPageChunk);
while (<page is being generated>) {
  webPageChunk = "some more html";
  server.sendContent(webPageChunk);
}
server.sendContent("");

Sending a blank line will tell the client to terminate the session. Be careful not to send one in your loop before you're done generating the whole page.

Allen Derusha
  • 1