With Windows API, we can use pipes to forward process output and error streams, so we can read process output without any temporary files. Instead of this:
std::system("my_command.exe > out.tmp");
we can work much faster and without risk to generate a lot of forgotten temporary files (on system crash for example).
Linux has similar functionality. However, implementation of OS-specific code for each OS is time consuming and complex task, so it looks like good idea to use some portable solution.
boost::process claims to be such solution. However, it is fundamentally unreliable. See following sample program:
#include <fstream>
#include <iostream>
#include <memory>
#include <vector>
#include <boost/asio.hpp>
#include <boost/process.hpp>
void ReadPipe(boost::process::async_pipe& pipe, char* output_buffer, size_t output_buffer_size, boost::asio::io_context::strand& executor, std::ofstream& output_saver)
{
namespace io = boost::asio;
using namespace std;
io::async_read(
pipe,
io::buffer(
output_buffer,
output_buffer_size
),
io::bind_executor(executor, [&pipe, output_buffer, output_buffer_size, &executor, &output_saver](const boost::system::error_code& error, std::size_t bytes_transferred) mutable
{
// Save transferred data
if (bytes_transferred)
output_saver.write(output_buffer, bytes_transferred);
// Handle error
if (error)
{
if (error.value() == boost::asio::error::basic_errors::broken_pipe)
cout << "Child standard output is broken, so the process is most probably exited." << endl;
else
cout << "Child standard output read error occurred. " << boost::system::system_error(error).what() << endl;
}
else
{
//this_thread::sleep_for(chrono::milliseconds(50));
ReadPipe(pipe, output_buffer, output_buffer_size, executor, output_saver);
}
})
);
}
int main(void)
{
namespace io = boost::asio;
namespace bp = boost::process;
using namespace std;
// Initialize
io::io_context asio_context;
io::io_context::strand executor(asio_context);
bp::async_pipe process_out(asio_context);
char buffer[65535];
constexpr const size_t buffer_size = sizeof(buffer);
ofstream output_saver(R"__(c:\screen.png)__", ios_base::out | ios_base::binary | ios_base::trunc);
// Schedule to read standard output
ReadPipe(process_out, buffer, buffer_size, executor, output_saver);
// Run child
bp::child process(
bp::search_path("adb"),
bp::args({ "exec-out", "screencap", "-p" }),
bp::std_in.close(),
bp::std_out > process_out,
bp::std_err > process_out
);
asio_context.run();
process.wait();
output_saver.close();
// Finish
return 0;
}
This code works nice; it runs ADB, generates Android device screenshot and saves it with asynchronous pipe, so no temporary files are involved. This specific example saves the screenshot as a file, but in real application you can save data in memory, load and parse it.
I use ADB in my sample, because this tool gives good example of data that generated comparably slowly and that sent via USB or Wi-Fi (so also slowly), and the data size is comparably big (for full HD device with complex image the PNG file will be 1M+).
When I uncomment following line:
this_thread::sleep_for(chrono::milliseconds(50));
The pipe reading operation becomes completely unreliable. The program reads only part of data (of unpredictable size).
So, even so short delay as 50 milliseconds forces boost implementation of asynchronous pipe to fail.
It is not normal situation. What if CPU usage is near 100% (i. e. we are on highly loaded server)? What if the thread runs other ASIO jobs that may execute during 50 milliseconds or less? So, it is just easily reproducible implementation of fundamental boost ASIO bug: asynchronous pipe can not tolerate any delays when you started to read it; you have to call async_read again instantly after you received data, otherwise you are at risk to loose your data.
In practice when I use the same ASIO context to run multiple jobs (not only one async_read that reads process standard output), async_pipe fails in 50% of attempts to read 1M of data or more.
Does anyone know a workaround how to make async_pipe reliable and not to break connection if ASIO context runs async_read with very small delays required to run other jobs?
