How do you disable auto-flush for Boost Log

Question

I am using Boost.Log to log various sets of data to different files. I would like to have the auto_flush feature enabled for some files, but disabled for others (so that a newline is not inserted on each consecutive log statement). I couldn't get this to work in my larger project. So I simplified the problem down to just one file, but it appears that auto_flush is still cannot be disabled. Here is code for a minimal example:

test.hpp:

#include <fstream>
#include <boost/shared_ptr.hpp>
#include <boost/smart_ptr/make_shared_object.hpp>
#include <boost/log/core.hpp>
#include <boost/log/sinks/sync_frontend.hpp>
#include <boost/log/sinks/text_ostream_backend.hpp>
#include <boost/log/trivial.hpp>

void init();

test.cpp:

#include "test.hpp"

void init() {
    // Initialize sink.
    typedef boost::log::sinks::synchronous_sink<boost::log::sinks::text_ostream_backend> text_sink;
    // Grab the Boost Log core.
    auto coreHandle = boost::log::core::get();
    // Add stream 1.
    boost::shared_ptr<text_sink> sink = boost::make_shared<text_sink>();
    sink->locked_backend()->add_stream(
            boost::make_shared<std::ofstream>("category1.log"));
    sink->locked_backend()->auto_flush(false);
    coreHandle->add_sink(sink);
}

main.cpp:

#include <iostream>
#include <string>
#include "test.hpp"

// Main entry point to program.
int main(int numArg, char const * const arguments[]) {

    double number1 = 42;

    init();

    BOOST_LOG_TRIVIAL(info) << "This message should go to category 1 log..." << number1;
    BOOST_LOG_TRIVIAL(info) << "This message should also go to category 1 log on the same line..." << number1;

    return EXIT_SUCCESS;
}

The output written to category1.log shows that a newline was applied between the two calls to BOOST_LOG_TRIVIAL, even though I explicitly set auto_flush to false in the init() function:

This message should go to category 1 log...42
This message should also go to category 1 log on the same line...42

I also use BOOST_LOG_CHANNEL_SEV to log to multiple files, but setting auto_flush to false still appears to have no effect...

How do you disable auto_flush, and have consecutive log statements print to the same line in the log file?
Can a solution be scaled to multiple log files, such that you can enable auto_flush for some files, but not others?

Andrey Semashev · Accepted Answer · 2019-10-17T17:43:42.750

How do you disable auto_flush, and have consecutive log statements print to the same line in the log file?

First, auto_flush has no relation to the trailing newline after each log record. It makes the sink flush its buffers after each log record is written, whether or not it contains a newline. Second, auto_flush can only be enabled or disabled on per-sink basis. In the particular case of text_ostream_backend it means either all streams attached to the sink will be flushed, or none of them. Third, the trailing newline is output by the sink backend internally, and this behavior cannot be disabled currently.

If you want to flush the log file only after select log records, your best approach would be to use an attribute to indicate when flushing is needed. Then you would have to create your own sink backend that would examine that attribute in the log records it processes and act appropriately. Creating sinks is described here.

Can a solution be scaled to multiple log files, such that you can enable auto_flush for some files, but not others?

Yes, of course. You have to create a sink per each file and set up auto_flush accordingly in those sinks.

Update 2019-10-17:

There has been development regarding the trailing newline insertion. In Boost 1.71, in text-based sinks there appeared an option to disable automatic trailing newline insertion. See auto_newline_mode enum and set_auto_newline_mode methods in the sink backends.

Thanks for clearing up my confusion, Andrey. This helped immensely! — Kevin, Dec 29 '16 at 16:38

score 0 · Answer 2 · edited Jun 20 '20 at 09:12

UPDATE October 2019:

New Answer for Boost 1.71

Boost.Log now provides a set_auto_newline_mode capability to control how new-lines are handled for each sink backend. This removes the need for a custom backend (as implemented in my old answer). Now, we can easily create a logging backend with automatic new lines disabled. The auto_newline_mode options are:

auto_newline_mode::disabled_auto_newline
auto_newline_mode::always_insert
auto_newline_mode::insert_if_missing

Here is a much simpler version of my multi-file Boost.Log example which tests this. The Category1 log features the normal auto newline. The Category2 log doesn't print because its severity level is insufficient. Finally, the Category3 log has auto-newline disabled, so consecutive logs are placed on the same line.

#include <iostream>
#include <string>
#include <fstream>

#include <boost/smart_ptr/shared_ptr.hpp>
#include <boost/smart_ptr/make_shared_object.hpp>
#include <boost/log/core.hpp>
#include <boost/log/sinks/sync_frontend.hpp>
#include <boost/log/sinks/text_ostream_backend.hpp>
#include <boost/log/sources/severity_channel_logger.hpp>
#include <boost/log/trivial.hpp>
#include <boost/log/expressions/keyword.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/keywords/severity.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>

#include <boost/filesystem.hpp>

// Includes from the example: http://www.boost.org/doc/libs/1_62_0/libs/log/example/doc/extension_stat_collector.cpp
#include <boost/date_time/posix_time/posix_time_types.hpp>
#include <boost/phoenix.hpp>
#include <boost/log/sinks/basic_sink_backend.hpp>
#include <boost/log/sources/logger.hpp>
#include <boost/log/sources/record_ostream.hpp>
#include <boost/log/attributes/value_visitation.hpp>
#include <boost/log/utility/manipulators/add_value.hpp>

// Includes from example: https://www.ociweb.com/resources/publications/sett/may-2016-boostlog-library/
#include <boost/log/sources/global_logger_storage.hpp>

namespace logging = boost::log;
namespace src = boost::log::sources;
namespace expr = boost::log::expressions;
namespace sinks = boost::log::sinks;
namespace keywords = boost::log::keywords;

BOOST_LOG_GLOBAL_LOGGER(Channel1Logger, src::severity_channel_logger<logging::trivial::severity_level>);
BOOST_LOG_GLOBAL_LOGGER(Channel2Logger, src::severity_channel_logger<logging::trivial::severity_level>);
BOOST_LOG_GLOBAL_LOGGER(Channel3Logger, src::severity_channel_logger<logging::trivial::severity_level>);

BOOST_LOG_GLOBAL_LOGGER_CTOR_ARGS(Channel1Logger, src::severity_channel_logger<logging::trivial::severity_level>, (keywords::channel = "Category1"));
BOOST_LOG_GLOBAL_LOGGER_CTOR_ARGS(Channel2Logger, src::severity_channel_logger<logging::trivial::severity_level>, (keywords::channel = "Category2"));
BOOST_LOG_GLOBAL_LOGGER_CTOR_ARGS(Channel3Logger, src::severity_channel_logger<logging::trivial::severity_level>, (keywords::channel = "Category3"));

#define LOG_CATEGORY1(LEVEL) BOOST_LOG_SEV(Channel1Logger::get(), logging::trivial::LEVEL)
#define LOG_CATEGORY2(LEVEL) BOOST_LOG_SEV(Channel2Logger::get(), logging::trivial::LEVEL)
#define LOG_CATEGORY3(LEVEL) BOOST_LOG_SEV(Channel3Logger::get(), logging::trivial::LEVEL)

BOOST_LOG_ATTRIBUTE_KEYWORD(severity, "Severity", logging::trivial::severity_level)
BOOST_LOG_ATTRIBUTE_KEYWORD(channel, "Channel", std::string)


/** Initialize the Boost.Log logging. */
void init() {
    // Alias the backend and sink types.
    typedef boost::log::sinks::text_ostream_backend MyBackendType;
    typedef boost::log::sinks::synchronous_sink< MyBackendType > MyBackendSinkType;
    // Grab the Boost Log core.
    auto coreHandle = boost::log::core::get();
    // Define the path where the log files should reside.
    boost::filesystem::path destinationDir("D:\\test");

    // Create a minimal severity table filter
    typedef expr::channel_severity_filter_actor< std::string, logging::trivial::severity_level > min_severity_filter;
    min_severity_filter minSeverity = expr::channel_severity_filter(channel, severity);

    // Set up the minimum severity levels for different channels.
    minSeverity["Category1"] = logging::trivial::info;
    minSeverity["Category2"] = logging::trivial::info;
    minSeverity["Category3"] = logging::trivial::info;

    // Define log file 1.
    boost::filesystem::path logFile1(destinationDir / "category1.log");
    // Create a log backend, and add a stream to it.
    boost::shared_ptr< MyBackendType > customBackend(new MyBackendType());
    customBackend->add_stream(boost::shared_ptr< std::ostream >(
            new std::ofstream(logFile1.string().c_str())));
    // Enable auto flush for this backend.
    customBackend->auto_flush(true);
    // Create a sink with the custom backend.
    boost::shared_ptr< MyBackendSinkType > customSink(new MyBackendSinkType(customBackend));
    // Add a filter to the sink.
    customSink->set_filter((channel == "Category1") && minSeverity && (severity >= logging::trivial::info));
    // Add the sink to the Boost.Log core.
    coreHandle->add_sink(customSink);

    // Define log file 2.
    boost::filesystem::path logFile2(destinationDir / "category2.log");
    // Create a log backend, and add a stream to it.
    customBackend = boost::make_shared< MyBackendType >();
    customBackend->add_stream(boost::shared_ptr< std::ostream >(
            new std::ofstream(logFile2.string().c_str())));
    // Enable auto flush for this backend.
    customBackend->auto_flush(true);
    // Create a sink with the custom backend.
    customSink = boost::make_shared< MyBackendSinkType >(customBackend);
    // Add a filter to the sink.
    customSink->set_filter((channel == "Category2") && minSeverity && (severity >= logging::trivial::info));
    // Add the sink to the Boost.Log core.
    coreHandle->add_sink(customSink);

    // Define log file 3.
    boost::filesystem::path logFile3(destinationDir / "category3.log");
    // Create a log backend, and add a stream to it.
    customBackend = boost::make_shared< MyBackendType >();
    customBackend->add_stream(boost::shared_ptr< std::ostream >(
            new std::ofstream(logFile3.string().c_str())));
    // Enable auto flush for this backend.
    customBackend->auto_flush(true);
    // Disable the auto newline for this backend.
    customBackend->set_auto_newline_mode(boost::log::sinks::auto_newline_mode::disabled_auto_newline);
    // Create a sink with the custom backend.
    customSink = boost::make_shared< MyBackendSinkType >(customBackend);
    // Add a filter to the sink.
    customSink->set_filter((channel == "Category3") && minSeverity && (severity >= logging::trivial::info));
    // Add the sink to the Boost.Log core.
    coreHandle->add_sink(customSink);
}

// This is the main entry point to the program.
int main (int numArgs, char const * const argList) {
    double number1 = 42;

    // Initialize the Boost.Log logging.
    init();

    // The Category1 logger has normal auto-newline. 
    LOG_CATEGORY1(info) << "New Category1 log1.";
    LOG_CATEGORY1(info) << "New Category1 log2.";
    // Category2 logger won't print to file b/c doesn't meet severity requirements.
    LOG_CATEGORY2(trace) << "New Category2 log.";

    // Category3 logger has auto-newline disabled. 
    LOG_CATEGORY3(info) << "[Put this on line 1]";
    LOG_CATEGORY3(info) << "[Put this on line 1 also]" << std::endl;
    LOG_CATEGORY3(info) << "[Put this on line 2]";

    std::cout << "Successful Completion!" << std::endl;
    return EXIT_SUCCESS;
}

Old Answer for Boost versions <1.71

With @Andrey's answer, I was able to create a custom sink backend that allows the user to pass an optional "NoNewline" value to certain logs in order to remove the newline character (\n). This is achieved via the add_value Boost.Log keyword. Here's an example of one solution using severity_channel_logger to filter logs via their severity and channel, where logs of a specific channel will be sent to a specific file. I pulled from other examples to form this solution, and these are cited in the code.

main.cpp:

#include <iostream>
#include <string>
#include <fstream>

#include <boost/smart_ptr/shared_ptr.hpp>
#include <boost/smart_ptr/make_shared_object.hpp>
#include <boost/log/core.hpp>
#include <boost/log/sinks/sync_frontend.hpp>
#include <boost/log/sinks/text_ostream_backend.hpp>
#include <boost/log/sources/severity_channel_logger.hpp>
#include <boost/log/trivial.hpp>
#include <boost/log/expressions/keyword.hpp>
#include <boost/log/expressions.hpp>
#include <boost/log/keywords/severity.hpp>
#include <boost/log/utility/setup/common_attributes.hpp>

#include <boost/filesystem.hpp>

// Includes from the example: http://www.boost.org/doc/libs/1_62_0/libs/log/example/doc/extension_stat_collector.cpp
#include <boost/date_time/posix_time/posix_time_types.hpp>
#include <boost/phoenix.hpp>
#include <boost/log/sinks/basic_sink_backend.hpp>
#include <boost/log/sources/logger.hpp>
#include <boost/log/sources/record_ostream.hpp>
#include <boost/log/attributes/value_visitation.hpp>
#include <boost/log/utility/manipulators/add_value.hpp>

// Includes from example: https://www.ociweb.com/resources/publications/sett/may-2016-boostlog-library/
#include <boost/log/sources/global_logger_storage.hpp>

namespace logging = boost::log;
namespace src = boost::log::sources;
namespace expr = boost::log::expressions;
namespace sinks = boost::log::sinks;
namespace keywords = boost::log::keywords;

BOOST_LOG_GLOBAL_LOGGER(Channel1Logger, src::severity_channel_logger<logging::trivial::severity_level>);
BOOST_LOG_GLOBAL_LOGGER(Channel2Logger, src::severity_channel_logger<logging::trivial::severity_level>);
BOOST_LOG_GLOBAL_LOGGER(Channel3Logger, src::severity_channel_logger<logging::trivial::severity_level>);

BOOST_LOG_GLOBAL_LOGGER_CTOR_ARGS(Channel1Logger, src::severity_channel_logger<logging::trivial::severity_level>, (keywords::channel = "Category1"));
BOOST_LOG_GLOBAL_LOGGER_CTOR_ARGS(Channel2Logger, src::severity_channel_logger<logging::trivial::severity_level>, (keywords::channel = "Category2"));
BOOST_LOG_GLOBAL_LOGGER_CTOR_ARGS(Channel3Logger, src::severity_channel_logger<logging::trivial::severity_level>, (keywords::channel = "Category3"));

#define LOG_CATEGORY1(LEVEL) BOOST_LOG_SEV(Channel1Logger::get(), logging::trivial::LEVEL)
#define LOG_CATEGORY2(LEVEL) BOOST_LOG_SEV(Channel2Logger::get(), logging::trivial::LEVEL)
#define LOG_CATEGORY3(LEVEL) BOOST_LOG_SEV(Channel3Logger::get(), logging::trivial::LEVEL)

BOOST_LOG_ATTRIBUTE_KEYWORD(severity, "Severity", logging::trivial::severity_level)
BOOST_LOG_ATTRIBUTE_KEYWORD(channel, "Channel", std::string)


// The backend collects records being forwarded from the front-end loggers.
class MyCustomBackend : public sinks::basic_sink_backend< 
        sinks::combine_requirements<
            sinks::synchronized_feeding,            
            sinks::flushing                         
        >::type>
{
private:
    // The file to write the collected information to.
    std::ofstream logFile;

public:
    // The constructor initializes the internal data
    explicit MyCustomBackend(const char * file_name);

    // The function consumes the log records that come from the frontend
    void consume(logging::record_view const& rec);
    // The function flushes the file
    void flush();

};

// The constructor initializes the internal data
MyCustomBackend::MyCustomBackend(const char * file_name) :
        logFile(file_name) {
    if (!logFile.is_open()) {
        throw std::runtime_error("Could not open the specified file!");
    }
}

// This function consumes the log records that come from the frontend.
void MyCustomBackend::consume(logging::record_view const & rec) {
    // If the NoNewline attribute is present, skip the newline.
    if (rec.attribute_values().count("NoNewline")) {
        logFile << *rec[boost::log::expressions::smessage];
    } else {
        logFile << *rec[boost::log::expressions::smessage] << std::endl;
    }
    // This is the equivalent of setting auto_flush.
    this->flush();
}

/** The function flushes the file. */
void MyCustomBackend::flush() {
    logFile.flush();
}

/** Initialize the Boost.Log logging. */
void init() {
    // Alias the custom sink types.
    typedef boost::log::sinks::synchronous_sink<MyCustomBackend> CustomBackendType;
    // Grab the Boost Log core.
    auto coreHandle = boost::log::core::get();
    // Define the path where the log files should reside.
    boost::filesystem::path destinationDir("C:\\test");

    // Create a minimal severity table filter
    typedef expr::channel_severity_filter_actor< std::string, logging::trivial::severity_level > min_severity_filter;
    min_severity_filter minSeverity = expr::channel_severity_filter(channel, severity);

    // Set up the minimum severity levels for different channels.
    minSeverity["Category1"] = logging::trivial::info;
    minSeverity["Category2"] = logging::trivial::info;
    minSeverity["Category3"] = logging::trivial::info;

    // Define log file 1.
    boost::filesystem::path logFile1(destinationDir / "category1.log");
    // Create a custom backend.
    boost::shared_ptr<MyCustomBackend> customBackend(new MyCustomBackend(logFile1.string().c_str()));
    // Create a sink with the custom backend.
    boost::shared_ptr<CustomBackendType> customSink(new CustomBackendType(customBackend));
    // Add a filter to the sink.
    customSink->set_filter((channel == "Category1") && minSeverity && (severity >= logging::trivial::info));
    // Add the sink to the Boost.Log core.
    coreHandle->add_sink(customSink);
    
    // Define log file 2.
    boost::filesystem::path logFile2(destinationDir / "category2.log");
    // Create a custom backend.
    customBackend = boost::make_shared<MyCustomBackend>(logFile2.string().c_str());
    // Create a sink with the custom backend.
    customSink = boost::make_shared<CustomBackendType>(customBackend);
    // Add a filter to the sink.
    customSink->set_filter((channel == "Category2") && minSeverity && (severity >= logging::trivial::info));
    // Add the sink to the Boost.Log core.
    coreHandle->add_sink(customSink);

    // Define log file 3.
    boost::filesystem::path logFile3(destinationDir / "category3.log");
    // Create a custom backend.
    customBackend = boost::make_shared<MyCustomBackend>(logFile3.string().c_str());
    // Create a sink with the custom backend.
    customSink = boost::make_shared<CustomBackendType>(customBackend);
    // Add a filter to the sink.
    customSink->set_filter((channel == "Category3") && minSeverity && (severity >= logging::trivial::info));
    // Add the sink to the Boost.Log core.
    coreHandle->add_sink(customSink);
}

int main (int numArgs, char const * const argList) {
    double number1 = 42;

    // Initialize the Boost.Log logging.
    init();

    LOG_CATEGORY1(info) << "New Category1 log.";
    // Won't print to file b/c doesn't meet severity requirements.
    LOG_CATEGORY2(trace) << "New Category2 log.";

    // Remove newline character ('\n') from specific logs. 
    LOG_CATEGORY3(info) << logging::add_value("NoNewline", true) << "[Put this on line 1]";
    LOG_CATEGORY3(info) << "[Put this on line 1 also]";
    LOG_CATEGORY3(info) << "[Put this on line 2]";

    return EXIT_SUCCESS;
}

How do you disable auto-flush for Boost Log

2 Answers2

UPDATE October 2019: