3

I am building a Single Page Application for Arduino. It graphically displays analog pin values on a wifi connected tablet.

I have built the sketch but want to clean it up. I have been able to upload a sketch to my (Uno Wifi Rev 2) Arduino, initialize the Wifi, and connect to it with a tablet. I am able send the static page "frame" to the tablet. That static frame is able to request and receive Arduino analog pin values using the XMLHttpRequest object.

But sending the bulky static page is clunky. Tutorials do stuff like,

client.println("<html><body>");
client.println("Hello World!");
client.println("</body></html>");

I tried to get slick and create a FileText.h header file:

#define constFileText=
"<html><body>"
"Hello World!"
"</body></html>";

and combine that with:

#include "FileText.h"
client.println(constFileText);

What I would like to do is create a standard FileText.html:

Hello World!

And process it with something like:

ifstream hFile ("FileText.html");
while (getline(hFile, strLine))
  client.println(strLine);

That would make it much easier to edit the html file. It would eliminate the waste of including all those serial.println calls. It would also eliminate the maximum length constraint on constant values.

Is there any way to provide a text file to the Arduino compiler and have the Arduino Server send it to the Arduino's client?

Alan
  • 31
  • 4
  • do you have a SD-Card connected to your Arduino? Then you could put your HTML-File there. Otherwhise you could try to store it in EEPROM (Atmega328p has 1KB EEPROM) – Tom Mekken Jan 25 '19 at 10:32
  • No, I am not using an SD-Card. But even with an SD-Card, the challenge remains of sending the file to the client. EEPROM won't work; I plan on putting the majority of the functionality in the browser. I want the Arduino to spend as much time as possible sampling the analog inputs. So far, I am using 14952 bytes of the 49152 bytes available for the sketch, as uploaded. That includes real time graphics I am hopeful that I will be able to have the browser save the data for later analysis. – Alan Jan 26 '19 at 04:26

3 Answers3

1

C++ has 'raw string literals'. You can put a constant string, without escaping special characters, into the source code between an opening and closing 'tag'. You can choose the tag to be something that is not in the raw string. In following example the tag is =====.

const char* s1 = R"=====(Hello
"World")=====";

is same as

const char* s2 = "Hello\n\"World\"";

This way you can put your large strings into separate .h files and include them. On AVR use PROGMEM to save RAM.

Juraj
  • 3,490
  • 4
  • 18
  • 25
  • I am tempted to rewrite my script to use your format. It would eliminate encoding logic and make resulting files readable. But, your method fails if the tag is within the file. Perhaps the script could generate a file-specific tag in the format `*%d*`, where the script keeps incrementing %d until it finds a `*%d*` that is not in the target file. – Alan Jan 28 '19 at 02:16
  • the tag is you choice. any character sequence of any length. – Juraj Jan 28 '19 at 06:02
0

You can use the xxd tool to generate an include file from your HTML. For example, give a file test.html:

<html><body>
Hello World!
</body></html>

Using xxd -i test.html > test_html.h results in test_html.h containing :

unsigned char test_html[] = {
  0x3c, 0x68, 0x74, 0x6d, 0x6c, 0x3e, 0x3c, 0x62, 0x6f, 0x64, 0x79, 0x3e,
  0x0d, 0x0a, 0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x57, 0x6f, 0x72, 0x6c,
  0x64, 0x21, 0x0d, 0x0a, 0x3c, 0x2f, 0x62, 0x6f, 0x64, 0x79, 0x3e, 0x3c,
  0x2f, 0x68, 0x74, 0x6d, 0x6c, 0x3e
};
unsigned int test_html_len = 42;

You can then #include "test_html.h" in your sketch and pass the array to client.print(). This gets round the limits on the size of a string. Unfortunately you lose the ability to loop through the array line by line, so you would have to write a function to do that yourself if required.

xxd is a *nix tool, but there are Windows ports if you need one.

jfowkes
  • 1,495
  • 9
  • 17
  • Thank you. I am running this on Win 10. Installing xxd seems to be a pain, so I will write a Python script to generate the well formatted *.h file. – Alan Jan 26 '19 at 04:01
  • I also looked at Powershell's Format-Hex, but I couldn't find good documentation. – Alan Jan 26 '19 at 04:12
  • Actually, this does not work for my application. I created a Python program to recreate the xxd output. The compiler didn't like it, `Global variables use 11571 bytes (188%) of dynamic memory, leaving -5427 bytes for local variables. Maximum is 6144 bytes.` When I prefixed the `const` keyword, the compiler said, `Global variables use 409 bytes (6%) of dynamic memory, leaving 5735 bytes for local variables. Maximum is 6144 bytes.` – Alan Jan 28 '19 at 01:17
  • Sorry, yes - the `xxd` method will put the array into RAM. The advantage of your Python method is that you can make it `const` (and even use `PROGMEM` since you're on Arduino/AVR). – jfowkes Jan 28 '19 at 10:25
0

Armed with @jfowkes format (but not @Juraj) I created a Python program. It works on both Windows and Linux.

To use it, place all text files (for me, .html and .js) in a subdirectory of the sketch. Then, from the sketch folder, run python TextEncode.py "SubdirectoryName". Add an #include "SubdirectoryName.h" line at the beginning of the sketch. That header file includes a function, void SendPage(WiFiClient hClient) which sends the contents of the files in subdirectory to the client; call it when appropriate.

(It does send the files in alphabetical order, so I precede the files with numbers such as "F210". I think of files as modules. By having many modules like this, I can disable modules by selectively commenting out code. I actually have two development modules [a .js and a .html] and one production module [a .js]; I have a copy of the SendPage function in the main sketch. By selectively commenting out code, I can choose whether or not I want to see the results of the XMLHttpRequest function calls.)

I know that this is a lot more complex than either of the other proposed solutions, but it helps the development cycle: (1) Edit html/js code in my favorite IDE (2) run the python program (3) compile the sketch.

Here's the contents of my TextEncode.py:

# program to convert text files to file with constant array of ascii code of file characters
# converted file is to be used by Arduino compiler to efficiently send html/js code to Arduino
# Usage:
# 1) place files to be encoded into subfolder, "ClientHtml"
# 2) from console, 'python TextEncode.py "ClientHtml"'.
# 

import os
import sys
import binascii

c_nCharsPerLine = 16
strFolderIn = sys.argv[1]
astrPseudos = []                                        # array of file pseudonyms.  to be used to create inclusive [ClientHtml].h 
if len(sys.argv) > 2:
    strClientHandle = sys.argv[2]
else:
    strClientHandle = "hClient"

for strFileIn in os.listdir(strFolderIn):
# encode each file in subdirectory
# it is easier to re encode every file than it is to check timestamps to re encode only updated files
    strFilePseudo = strFileIn.replace (".", "_")        # to be used in name of encoded file and name of variable with contents of file.
    astrPseudos.append(strFilePseudo)
    strContents = "";                                   # contents read from file itself, in pairs of hex digits

    with open(strFolderIn + "/" + strFileIn, "r") as fileIn:
        nChar = 0;
        for strLineIn in fileIn:
            for chIn in strLineIn:
#               strContents = strContents + chIn.encode("hex") + ","        # works on Linux
                strContents = strContents + hex(ord(chIn)) + ","
                nChar += 1
                if nChar % c_nCharsPerLine == 0:
                    strContents += "\n"
    # truncate trailing \n, if it exists
    if nChar % c_nCharsPerLine == 0:
        strContents = strContents[:-1]
    strContents += "0\n"

    with open (strFilePseudo + ".h", "w") as fileOut:
        fileOut.write("const unsigned char c_" + strFilePseudo + "[] = {\n")
#       fileOut.write("unsigned char c_" + strFilePseudo + "[] = {\n")
        fileOut.write(strContents)
        fileOut.write("};\n")

with open (strFolderIn + ".h", "w") as fileOut:
    fileOut.write("// .h files with encoded files to be included:\n")
    astrPseudos.sort()
    for strFilePseudo in astrPseudos:
        fileOut.write("#include \"" + strFilePseudo + ".h\"\n")
    fileOut.write("/*\n")
    fileOut.write("// Arduino Compiler function to send encoded files to web client:\n")
    fileOut.write("// Comment these out if you don't want to use the functionality\n")
    fileOut.write("void SendPage(WiFiClient " + strClientHandle + ")\n")
    fileOut.write("{\n")
    fileOut.write(" String strData;\n")
    for strFilePseudo in astrPseudos:
        fileOut.write(" strData=c_" + strFilePseudo + ";\n")
        fileOut.write(" " + strClientHandle + ".println(strData);\n")
    fileOut.write("}\n")
    fileOut.write("*/\n")
Alan
  • 31
  • 4
  • Yes, this solution exceeds the question parameters. To process just one file, use the code within the `for strFileIn ...` loop. Or, put exactly one file in the subdirectory. – Alan Jan 28 '19 at 02:20