ESP32-WROOM-32 callback function and memory issue

Question

We are currently testing a library related to the IoT Platform with ESP32-WROOM-32 now.

_token is defined in .h file. In one file HttpDownstream.cpp:

void HttpDownstreamClient::tokenCreate(char* URL, char* Subname, char* Suber, int expiretime) {

  Serial.print("The subscription is associated with the Subscripter: ");
  Serial.println(Suber);

  StaticJsonDocument<200> root;
 
  root["subscriber"] = Suber;
  root["subscription"] = Subname;
  root["expiresInMinutes"] = expiretime;
  String body2send = "";

  serializeJsonPretty(root, body2send);

  if (_networkClient->connect(URL, 443)) {
    Serial.println("Connected to the server to request token!");
    // Make a HTTP request:
    _networkClient->println("POST /notification2/token HTTP/1.1");
    _networkClient->print("Host: ");
    _networkClient->println(URL);
    _networkClient->print("Authorization: Basic ");
    _networkClient->println(_base64);
    _networkClient->println("Content-Type: application/json");
    _networkClient->print("Content-Length: ");
    _networkClient->println(body2send.length());
    _networkClient->println("Accept: application/json");
    _networkClient->println();
    _networkClient->println(body2send);

  }

//Start to handle incoming message sent from server
while (_token.length() == 0) {
    String msg = "";
    while (_networkClient->available()) {
      char c = _networkClient->read();
      msg += c;
    }
    Serial.println("debug!!!");
    Serial.println(msg);
    int start_init = msg.indexOf("\"token\"");
    int start = msg.indexOf(":",start_init);
    int until_n = msg.indexOf("\"", start + 3);
    Serial.println(until_n);
    if (start != -1 && until_n != -1 ) {
      _token = msg.substring(start + 2, until_n);
      Serial.print("Token is: ");
      Serial.println(_token);
     }
  }
 
}

In test.io we have:

void setup(){

//WiFi setting is not showed here

c8yclient.tokenCreate(host, SubscriptionName, SubscriberName, expiretime);

}

void loop()
{
  while (wifisecure.available()) {
    char c = wifisecure.read();
    Serial.print(c);
  }  
}

What I get is something like this:

21:31:57.993 -> The subscription is associated with the Subscripter: Test2Sub1
21:31:58.033 -> Connected to the server to request token!
21:31:58.073 -> debug!!!
21:31:58.073 -> ⸮
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!
21:31:58.073 -> 
21:31:58.073 -> -1
21:31:58.073 -> debug!!!

What I can make sure now is that the https request has been sent out. However, the incoming message from the server is not "well" processed. IMO there must be something wrong in memory since we have a "⸮" printed.

We tried this library with MKR wifi 1010, Arduino Uno rev 2 and nano 33 IoT. None of these boards has this issue. What has happened to ESP32? How can I fix this?

Many thanks in advance!

We think it might be a memory allocation issue since we have seen "⸮" printed. We need more inputs from your side. Thanks!

.h file


#ifndef HttpDownstream_h
#define HttpDownstream_h

#include "Arduino.h"
#include <Base64.h> 
#include <ArduinoJson.h>
#include <string.h>
#include <Client.h>


class HttpDownstreamClient{


 private:

 String _token;
 char* _base64;
 //String _deviceID;

 //Base64 encoder
 void base64(char* username, char* password);
 
 
 public:

 Client* _networkClient;
 
 HttpDownstreamClient(Client& networkClient);

 //Delete the already existed subscription
 void subscriptionDelete(char* user, char* password,char* URL, char* context,char* source);
 
 //Create subscription
 void subscriptionCreate(char* user, char* password, char* URL, char* device_ID,char* context ,char* SubName, char* API,char* typeFilter, char* fragmentsToCopy);
 
 //Create token 
 void tokenCreate(char* URL, char* Subname, char* Suber, int expiretime);
 
 //Create token by giving password
 void tokenCreatePassword(char* user, char* password, char* URL, char* Subname, char* Suber, int expiretime);

 //Open WSS
 void connectWSS(char* URL,char* Suber);


};


#endif

.cpp file

#include "HttpDownstream.h"


HttpDownstreamClient::HttpDownstreamClient(Client& networkClient) {

  _networkClient = &networkClient;

}


//Base64 encoder
void HttpDownstreamClient::base64(char* username, char* password) {
  char temp[100];

  strcpy(temp , username);
  strcat(temp, ":");
  strcat(temp, password);
  Serial.println();
  //For debugging
  Serial.print("Information needs to be coded is:");
  Serial.println(temp);

  int inputStringLength = strlen(temp);
  int encodedLength = Base64.encodedLength(inputStringLength);
  char encodedString[encodedLength];
  Base64.encode(encodedString, temp, inputStringLength);
  //Serial.print("Encoded string is: ");
  //Serial.println(encodedString);
  //Memory allocation
  if (_base64) free (_base64);
  _base64 = (char*) malloc(sizeof(char) * strlen(encodedString));
  strcpy(_base64, encodedString);

  Serial.print("Stored Based64 string is: ");
  Serial.println(_base64);
}



void HttpDownstreamClient::subscriptionDelete(char* user, char* password,char* URL,char* context,char* source){
  
  base64(user, password);
  //Serial.println(_base64);

  if (_networkClient->connect(URL, 443)){
    Serial.println("Start to clear the subscription");
    // Make a HTTP request:
    _networkClient->print("DELETE /notification2/subscriptions?context=");
    _networkClient->print(context);
    _networkClient->print("&");
    _networkClient->print("source=");
    _networkClient->print(source);
    _networkClient->println(" HTTP/1.1");
    _networkClient->print("Host: ");
    _networkClient->println(URL);
    _networkClient->println("Accept: application/vnd.com.nsn.cumulocity.error+json");
    _networkClient->print("Authorization: Basic ");
    _networkClient->println(_base64);
    
    }
  
  }

void HttpDownstreamClient::subscriptionCreate(char* user, char* password, char* URL, char* device_ID, char* context , char* SubName, char* API, char* type_Filter, char* fragments_To_Copy) {

  //Serial.print("The subscription is associated with the device: ");
  //Serial.println(device_ID);

  base64(user, password);
  Serial.println(_base64);

  StaticJsonDocument<400> root;
  root["context"] = context;
  
  if(context == "mo"){
    if(device_ID != NULL){
  
  JsonObject source  = root.createNestedObject("source");
  source["id"] = device_ID;
  
      }else{
        Serial.println("Please enter the device ID");
        }
    }
  //JsonObject source  = root.createNestedObject("source");
  //source["id"] = device_ID;

  root["subscription"] = SubName;
  
  JsonObject subFilter = root.createNestedObject("subscriptionFilter");
  if (*type_Filter != NULL) {
    subFilter["typeFilter"] = type_Filter;
  } else {}

  JsonArray apis = subFilter.createNestedArray("apis");
  apis.add(API);

  if (*fragments_To_Copy != NULL) {
    JsonArray fragtocopy = root.createNestedArray("fragmentsToCopy");
    fragtocopy.add(fragments_To_Copy);
  } else {}


  String body2send = "";
  serializeJsonPretty(root, body2send);
  Serial.println(body2send);

  if (_networkClient->connect(URL, 443)) {
    Serial.println("Connected to the server");
    // Make a HTTP request:
    _networkClient->println("POST /notification2/subscriptions HTTP/1.1");
    _networkClient->print("Host: ");
    _networkClient->println(URL);
    _networkClient->println("Content-Type: application/json");
    _networkClient->print("Content-Length: ");
    _networkClient->println(body2send.length());
    _networkClient->println("Accept: application/json");
    _networkClient->print("Authorization: Basic ");
    _networkClient->print(_base64);
    _networkClient->println();
    _networkClient->println();
    _networkClient->println(body2send);

  } else {
    Serial.println("Connection fail!");
  }
}


void HttpDownstreamClient::tokenCreate(char* URL, char* Subname, char* Suber, int expiretime) {

  Serial.print("The subscription is associated with the Subscripter: ");
  Serial.println(Suber);

  StaticJsonDocument<150> root;

  root["subscriber"] = Suber;
  root["subscription"] = Subname;
  root["expiresInMinutes"] = expiretime;
  String body2send = "";

  serializeJsonPretty(root, body2send);
  //Serial.println(body2send);
  Serial.println("Start to connect to the server!");
  if (_networkClient->connect(URL, 443)) {
    Serial.println("Connected to the server");
    // Make a HTTP request:
    _networkClient->println("POST /notification2/token HTTP/1.1");
    _networkClient->print("Host: ");
    _networkClient->println(URL);
    _networkClient->print("Authorization: Basic ");
    _networkClient->println(_base64);
    _networkClient->println("Content-Type: application/json");
    _networkClient->print("Content-Length: ");
    _networkClient->println(body2send.length());
    _networkClient->println("Accept: application/json");
    _networkClient->println();
    _networkClient->println(body2send);

  }

while (_token.length() == 0) {
    String msg = "";
    while (_networkClient->available()) {
      char c = _networkClient->read();
      msg += c;
    }
    Serial.println("debug!!!");
    Serial.println(msg);
    int start_init = msg.indexOf("\"token\"");
    int start = msg.indexOf(":",start_init);
    int until_n = msg.indexOf("\"", start + 3);
    Serial.println(until_n);
    if (start != -1 && until_n != -1 ) {
      _token = msg.substring(start + 2, until_n);
      Serial.print("Token is: ");
      Serial.println(_token);
     }
  }
 
}


void HttpDownstreamClient::tokenCreatePassword(char* user, char* password, char* URL, char* Subname, char* Suber, int expiretime) {
  base64(user, password);
  Serial.println(_base64);

  Serial.print("The subscription is associated with the Subscripter: ");
  Serial.println(Suber);

  StaticJsonDocument<150> root;

  root["subscriber"] = Suber;
  root["subscription"] = Subname;
  root["expiresInMinutes"] = expiretime;
  String body2send = "";

  serializeJsonPretty(root, body2send);
  //Serial.println(body2send);

  if (_networkClient->connect(URL, 443)) {
    Serial.println("Connected to the server");
    // Make a HTTP request:
    _networkClient->println("POST /notification2/token HTTP/1.1");
    _networkClient->print("Host: ");
    _networkClient->println(URL);
    _networkClient->print("Authorization: Basic ");
    _networkClient->println(_base64);
    _networkClient->println("Content-Type: application/json");
    _networkClient->print("Content-Length: ");
    _networkClient->println(body2send.length());
    _networkClient->println("Accept: application/json");
    _networkClient->println();
    _networkClient->println(body2send);

  }

  while (_token.length() == 0) {
    String msg = "";
    while (_networkClient->available()) {
      char c = _networkClient->read();
      msg += c;
    }
    Serial.println(msg);
    int start_init = msg.indexOf("\"token\"");
    int start = msg.indexOf(":",start_init);
    int until_n = msg.indexOf("\"", start + 3);
    Serial.println(until_n);
    if (start != -1 && until_n != -1 ) {
      _token = msg.substring(start + 2, until_n);
      Serial.print("Token is: ");
      Serial.println(_token);
     }
  }


}



void HttpDownstreamClient::connectWSS(char* URL, char* Suber) {
  if (_token.length() != 0) {
    if (_networkClient->connect(URL, 443)) {
      Serial.println("Websocket port is opened");
      // Make a HTTP request:
      _networkClient->print("GET ");
      _networkClient->print("/notification2/consumer/?token=");
      _networkClient->print(_token);
      _networkClient->print("&consumer=");
      _networkClient->print(Suber);
      _networkClient->println(" HTTP/1.1");

      _networkClient->print("Host: ");
      _networkClient->println(URL);
      _networkClient->println("Sec-WebSocket-Key: q4xkcO32u266gldTkKaSOw==");
      _networkClient->println("Upgrade: websocket");
      _networkClient->println("Connection: Upgrade");
      _networkClient->println("Sec-WebSocket-Version: 13");
      _networkClient->println();
    }
  }
}

.ino file

#include <HttpDownstream.h>
#include <WiFi.h>
#include <WiFiClientSecure.h>

WiFiClientSecure wifisecure;
HttpDownstreamClient c8yclient(wifisecure); //initiate the HttpDownstreamClient 

char* host = "tenant.cumulocity.com";
char* username = "tenant";
char* password = "tenant";
char* device_id = "13225";

//Subscription Test
char* SubscriptionName = "NotificationV2Test12";
char* SubscriberName = "Test2Sub1";
char* APIs = "measurements";
char* typeFilter= "";
char* fragmentsToCopy= "";
int expiretime = 1100;


char ssid[] = "**********";        // your network SSID (name)
char pass[] = "**********";    // your network password (use for WPA, or use as key for WEP)
int status = WL_IDLE_STATUS;

void setup()
{
    Serial.begin(115200);
    delay(10);

 wifisecure.setInsecure();
 while (status != WL_CONNECTED) {
    Serial.print("Attempting to connect to SSID: ");
    Serial.println(ssid);
    // Connect to WPA/WPA2 network. Change this line if using open or WEP network:
    status = WiFi.begin(ssid, pass);

    // wait 5 seconds for connection:
    delay(5000);
  }

 Serial.println("Connected to WiFi"); 
 
  c8yclient.subscriptionCreate(username,password,host, device_id,"mo" ,SubscriptionName, APIs,typeFilter,fragmentsToCopy);
  delay(6000);
  c8yclient.tokenCreate(host, SubscriptionName, SubscriberName, expiretime);
  delay(6000);
  //c8yclient.connectWSS(host,SubscriberName);
  //delay(5000);
    
}


void loop()
{



  while (wifisecure.available()) {
    char c = wifisecure.read();
    Serial.print(c);
  }

   
}

You're reading random data and interpret it as ascii. Are you sure the client sends only ASCII characters and no UTF-8 encoded stuff? — PMF, Nov 14 '22 at 13:45
Yes, they are only ASCII characters. Is there any memory allocation issue when I defined the parameter String msg="msg"? — YIL, Nov 14 '22 at 14:16
What type is `_networkClient`? Can you provide a more minimal yet more complete example? — Codo, Nov 14 '22 at 15:15
No, I don't see a memory issue here. String concatenation that way is not very efficient, but on the ESP32 there should be enough memory for that. — PMF, Nov 14 '22 at 15:17
Hi all,I have just editted the post and you can see the .h and .cpp file now. Hope this can help you understnd the idea of the code. The problem lies on the tokenCreate() method. While the _token is null then just wait and process the message sent back from the server. I tried printing "msg" and got nothing else but a "⸮". — YIL, Nov 14 '22 at 17:07
@PMF I have just updated ocntent on the original question. Now you can see the code and the structure. — YIL, Nov 14 '22 at 18:58

score 1 · Answer 1 · answered Nov 16 '22 at 05:54

1

You have an off-by-one issue when you allocate memory for _base64.

Your code currently reads:

 if (_base64) free (_base64);
  _base64 = (char*) malloc(sizeof(char) * strlen(encodedString));
  strcpy(_base64, encodedString);

strlen() returns the number of characters in the string, not including the null terminator. You need to allocate 1 more character than the length. Even better, just use strdup(), which takes care of it for you.

So either

 if (_base64) free (_base64);
  _base64 = (char*) malloc((sizeof(char) * (strlen(encodedString)+1));
  strcpy(_base64, encodedString);

or

 if (_base64) free (_base64);
  _base64 = strdup(_encodedString);

You really should check if malloc() or strdup() fails and do something appropriate in that case, too.

answered Nov 16 '22 at 05:54

romkey

6,218
3
16
12

Good point! But still it doesn't explain the problem in the tokenCreate() method. `while (_token.length() == 0) { String msg = ""; while (_networkClient->available()) { char c = _networkClient->read(); msg += c; } Serial.println("debug!!!"); Serial.println(msg);` I tried to print "msg" and I got a "⸮" instead. I really don't know what had happened to my code. – YIL Nov 16 '22 at 17:21
Did you apply this fix and try it? – romkey Nov 16 '22 at 17:36
I have already applied the fix and it still diplayed "⸮". There must be something wrong in the while loop of the function tokenCreate(). – YIL Nov 16 '22 at 19:33

ESP32-WROOM-32 callback function and memory issue

1 Answers1