2

I found quite some info on output redirection, creation of streambuffers and ostream classes, but I did not manage to apply this succesfully yet for my purpose. This post has become quite lengthy because I wanted to describe my step by step approach.

I have an application that uses a class MyNotifier that captures events in the application and composes log messages based on the event data. By default it sends the log messages to std::cout, but the constructor of MyNotifier accepts a variable of type std::ostream& to overide this. I am trying to construct a class of that type which should send the logs to an different output channel, e.g. via an MQTT client. I have MQTT up and running well. My question is about the creation of the custom ostream class.

Here is the code that should use the new class (see the commented lines in app_main) and it's output when I use std::cout. For testing, the events are generated by calling MyNotifier::Notify directly.

    class MyNotifier {    
    public:
        //constructor
        MyNotifier(std::ostream& os = std::cout) : ost(os) {}
        //takes eventdata as a code for the event
        //composes some output string based on the input and outputs it to the customizable output stream ost
        virtual void Notify( unsigned long eventdata);
    protected:
        std::ostream& ost;        
      }; //class MyNotifier

Implementation:

    void MyNotifier::Notify(unsigned long eventdata) {
        //takes eventdata as dummy for an event
        //composes some output string based on the input and outputs it to the customizable output stream ost
        char s[200];
        int wr = sprintf(s, "RECEIVED EVENT %s ", "of type 1 ");
        sprintf( s + wr , "with number %lu\n", eventdata);
        std::cout << "MyNotifier::Notify" << s << std::endl;                     //this shows up
        ost << "dummy custom_ostream output: " << eventdata << std::endl;
        //trial to send over MQTT,  in the end ost should generate MQTT output
        esp_mqtt_client_publish(mqtt_client, "/fckx_seq/GUI", "value", 0, 1, 0); //works fine
    } //MyNotifier::Notify


void app_main(void) {
    MyNotifier notifier;                    //instantiate with default output stream (std::cout)
    //MyNotifier notifier(std::cout);       //instantiate with default output stream explicitly, also works with the same result
    //MyNotifier notifier(custom_ostream)   //desired way to send logs over a Custom_ostream object
    notifier.Notify(3142727);  //notify some event
} 

This gives the desired output over cout:

RECEIVED EVENT of type 1 with number 3142727

In my first step to customize the output I only customize the streambuf class (OutStreamBuf). It is used by a "plain" ostream class:

class OutStreamBuf : public std::streambuf {       
  protected:      
    /* central output function
     * - print characters in uppercase mode
    */          
    //converts each character to uppercase
    virtual int_type overflow (int_type c) {
        if (c != EOF) {
        // convert lowercase to uppercase
        c = std::toupper(static_cast<char>(c),getloc());
        //output to standard output
        putchar(c);

        }
        return c;
    }
   
    // write multiple characters  MUST USE CONST CHAR* ?
    virtual std::streamsize xsputn (char* s, std::streamsize num) {  
        std::cout << "**size: " << num << std::endl;    
        std::cout << "OutStreamBuf contents: " << s << std::endl;
        return 1;
    }    
}; //OutStreamBuf


Implementation:

OutStreamBuf outStreamBuf;                                    
std::ostream custom_ostream(&outStreamBuf);
MyNotifier notifier(custom_ostream);         //instantiate with customized output stream  
notifier.Notify(314132345);  //notify some event  
custom_ostream << "Test << operator" << std::endl;

Output:

**MyNotifier::Notify direct: RECEIVED EVENT of type 1  with number 314132345   
DUMMY CUSTOM_OSTREAM OUTPUT: 314132345    <------ THIS IS THE DESIRED OUTPUT   
TEST << OPERATOR**


In my second step I want to get hold of the buffer contents to be able to forward this to my MQTT handler. So I decided that I need a customized ostream object. In the second trial I therefore created a customized ostream class (OutStream) with an *embedded* customized streambuf class:

class OutStream : public std::ostream {     
    private:
        //private local Outbuf for OutStream
        class Outbuf : public std::streambuf {        
    protected:
    /* central output function
     * - print characters in uppercase mode
     */     
     
        //converts each character to uppercase
        virtual int_type overflow (int_type c) {
            if (c != EOF) {
            // convert lowercase to uppercase
            c = std::toupper(static_cast<char>(c),getloc());
            //output to standard output
            putchar(c);

            }
            return c;
        }
   
        // write multiple characters  MUST USE CONST CHAR* (?)
        virtual std::streamsize xsputn (char* s, std::streamsize num) {  
            std::cout << "**size: " << num << std::endl;    
            std::cout << "OUTBUF contents: " << s << std::endl;
            return 1;
        }    
    }; //Outbuf

        Outbuf outbuf;
        std::streambuf * buf;
     public:
        //constructor
        OutStream() {
        //buf = this->rdbuf();  //compiles OK, but what does it do ?
         buf = rdbuf();         //compiles OK, but what does it do ?
         std::cout << "SOME MESSAGE FROM OutStream constructor" <<std::endl;                         
         esp_mqtt_client_publish(mqtt_client, "/fckx_seq/GUI", "OutStream constructor", 

, 1, 0);
};

        // << operator
        //https://www.geeksforgeeks.org/overloading-stream-insertion-operators-c/
        //have a good look on what parameters the operator should take , see the above article       
        friend std::ostream & operator << (std::ostream &stream, const OutStream& outStream){
            esp_mqtt_client_publish(mqtt_client, "/fckx_seq/GUI", "OutStream << operator", 0, 1, 0); //doesn't show
            stream << "Test << operator inside " << std::endl;                                       //doesn't show
            return stream; //return the stream              
        };     
}; //OutStream

Implementation:

``` OutStream custom_ostream;             //using a composite ostream/streambuf object       
    MyNotifier notifier(custom_ostream);  //instantiate with customized output stream
    notifier.Notify(314132345);           //notify some event  
    custom_ostream << "Test << operator" << std::endl;

This does not show the customised output. Therefore I added a log in the constructor (properly shown) and a modified << operator with a log (also not shown):

SOME MESSAGE FROM OutStream constructor

MyNotifier::Notify direct: RECEIVED EVENT of type 1 with number 314132345

As the << operator log also fails I think that something is wrong with the constructor of the ostream object and/or it's binding with the streambuf. This is pretty complex stuff for me. Some help would be appreciated.

[EDIT] After discussion with Stephen M. Webb I focused on finding an example of a class based on std::streambuf that contains additional buffering. I found the following code that will hopefully be a good basis for further steps:

//p. 837 The C++ Standard Library Second Edition, Nicolai M. Josuttis

class Outbuf_buffered : public std::streambuf {
    protected:
        static const int bufferSize = 10; // size of data buffer
        char buffer[bufferSize]; // data buffer
    public:
        // constructor
        // - initialize data buffer
        // - one character less to let the bufferSizeth character cause a call of overflow()
        Outbuf_buffered() {
        setp (buffer, buffer+(bufferSize-1));
        }
        // destructor
        // - flush data buffer
        virtual ~Outbuf_buffered() {
        sync();
        }

    protected:
        // flush the characters in the buffer
        int flushBuffer () {
        int num = pptr()-pbase();
        if (write (1, buffer, num) != num) {
        return EOF;
        }
        pbump (-num); // reset put pointer accordingly
        return num;
        }
        
        // buffer full
        // - write c and all previous characters
        virtual int_type overflow (int_type c) {
        if (c != EOF) {
        // insert character into the buffer
        *pptr() = c;
        pbump(1);
        }
        // flush the buffer
        if (flushBuffer() == EOF) {
        // ERROR
        return EOF;
        }
        return c;
        }

        // synchronize data with file/destination
        // - flush the data in the buffer
        virtual int sync () {
        if (flushBuffer() == EOF) {
        // ERROR
        return -1;
        }
        return 0;
        }
};  //Outbuf_buffered
f ckx
  • 49
  • 1
  • 6
  • excuse me for poor formatting. I gave it a good try to get rid of the editor warnings on the code parts. In the end the post was accepted, but with some flaws.... – f ckx Mar 24 '22 at 13:49

2 Answers2

2

You don't want to touch ostream (the formatting layer) at all. You should do everything in the streambuf (the transport layer). You can use manipulators to set or change the state of the underlying transport layers through the generic ostream interface, if required.

Stephen M. Webb
  • 1,705
  • 11
  • 18
  • That would mean that I have to start from my first step (untouched ostream, customized streambuf). Could you give a hint how I could capture the full stream content there to compose a MQTT (or other ) message. Current output is generated one character at a time by overflow in OutStreamBuf, I presume. – f ckx Mar 24 '22 at 14:38
  • 3
    The standard ostream calls the `sputn()` member function of its contained streambuf object, which should append the string to its buffer. When it's time, the streambuf object will calls its `overflow()` member function to flush the entire buffer to its destination (eg call `esp_mqtt_client_publish()`. You might want to examine how std::basic_filebuf works on your favourite C++ standard library implementation. You might want to derive your streambuf – Stephen M. Webb Mar 24 '22 at 19:02
  • thanks for you comments! Could you perhaps hint me why the code that I have now does not output anything at all that has been entered via the << operator (ost << "dummy custom_ostream output: " << eventdata << std::endl; in MyNotifier::Notify). Is something wrong with the constructor? I know that the constructor is called because it outputs "SOME MESSAGE FROM OutStream constructor". – f ckx Mar 25 '22 at 21:05
  • I have read several texts about std::ostream and std:: streambuf (e.g https://www.cplusplus.com/reference/ostream/ostream/ and https://en.cppreference.com/w/cpp/io/basic_streambuf) but I simply do not grasp how they work very well. My first trial (with class OutStreamBuf) is the only solution where what I inserted into the object leads to some output. Now I added an MQTT publish after the putchar in the overflow override. I get a set of MQTT messages, one for each character. This confirms my suspection on how this works. But how do I force output of the data in one go? – f ckx Mar 26 '22 at 11:06
  • 1
    Your streambuf needs to buffer its output, and only when xsputn() detects the buffer is full should it call overflow() -- that is, when the buffer would overflow if one more character is added to it. The overflow() member function should flush the *entire* buffer to its sink and reset all the internal buffer pointers. – Stephen M. Webb Mar 26 '22 at 11:35
  • OK. That sounds logical. But how do I achieve that "only when xsputn() detects the buffer is full should it call overflow()". I think that you want to say that currently my streambuf does not buffer it's output, correct? How do I make it buffer it's output? – f ckx Mar 26 '22 at 11:43
  • and.... output should also be created when the buffer is not full. So, if I insert a certain string, that string should be sent to the output (e.g. MQTT) when the string is complete. How is this detected? – f ckx Mar 26 '22 at 11:51
  • 1
    The std::streambuf class takes care of buffering the stream for you -- which explains its name. You just have to provide a write operation from overflow() and reset the pointers. You use the pbase() and pptr() functions to get the buffer pointers and the setp() function to reset them. This is all nicely documented with a cunning diagram at https://en.cppreference.com/w/cpp/io/basic_streambuf – Stephen M. Webb Mar 26 '22 at 12:00
  • OK. Write is a member of ostream, so I have to start from a class that derives from ostream, my second trial in the original post. There I have the problem that the shown solution does not output anything at all (only a log from the constructor). When I have some output I have at least a starting point. Can you see what is wrong with class OutStream? – f ckx Mar 26 '22 at 12:17
  • No no no, do not touch ostream. By "write" I mean something that sends bytes to the data sink you're streaming to. `esp_mqtt_client_publish()` is a write operation. You have to send a buffer full of data to wherever and then reset the buffer so that it will take more data. – Stephen M. Webb Mar 26 '22 at 21:28
  • OK. I'll give it a go again. So, use untouched ostream. Only tune streambuf ? Use streambuf member functions to get data from the buf and store that in a buffered variable that can be transferred to mqtt publish. – f ckx Mar 26 '22 at 21:37
  • I tried to find an example of a class based on std::streambuf which also applies an additional buffer that could be output via MQTT publish. I pasted the code as an edit of my original question. Does this approach match with your suggestions? If so, I'll give it a try to adapt the code for my needs. – f ckx Mar 26 '22 at 22:26
  • That is exactly the way to go! I now can use buffer in the flushBuffer routine for creation of a message over MQTT (or to do anything else with the data. – f ckx Mar 27 '22 at 14:20
0

By using a class that is derived from std::streambuf that contains an additional buffer member I can do the trick. The main ostream output stream is just std::ostream. I found an example from The C++ Standard Library Second Edition, Nicolai M. Josuttis, p. 837 .

See the EDIT in my original post. Thanks @Stephen M. Webb for your perseverance in guiding me to the answer!

f ckx
  • 49
  • 1
  • 6
  • I still have some trouble in getting the contents of the additional buffer into my output sink (MQTT message) correctly. The shown code, as is, does not do this properly. Although this takes a better understanding on handling the buffer and it's pointers, I consider this as out of scope for the original question. – f ckx Mar 28 '22 at 14:10
  • I would suggest to update this reply, above, to list the full source of your solution so far (instead of appending it at the bottom of the Opening Post (OP). – ankostis Jul 15 '22 at 20:01