Keep ANSI Escape Sequences in QProcess Output

Question

I am creating a program where I run processes in Qt using the QProcess framework on Ubuntu 16.04 Qt 5.5.1 with C++ 11 enabled. I am directing the process output stream to a QTextEdit.

I would like to colorize this output to use the same colors which native terminals interpret by using the embedded ANSI escape color sequences. However, I am unable to parse the escape sequences as they appear to be missing from the QProcess output. I originally thought QString was stripping them, but after some testing I do not believe this to be the case.

I found some information to point me in the ANSI escape color interpretation direction if I could just keep the escape sequences in the QProcess output.

Here is an example project of what I am doing in Qt code.

The source file...

#include "mainwindow.h"
#include "ui_mainwindow.h"
#include <QString>
#include <QProcess>
#include <QStringList>

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);

    QStringList input = {"gcc will_not_build.c"};
    QProcess * proc = new QProcess();

    proc->setReadChannel(QProcess::StandardOutput);
    proc->setProcessChannelMode(QProcess::MergedChannels);
    proc->setWorkingDirectory("/path/to/test/c/file/");

    //Start bash
    proc->start("bash");
    proc->waitForStarted();

    // Write as many commands to this process as needed
    foreach(QString str, input){
        proc->write(str.toUtf8() + "\n");
        proc->waitForBytesWritten(-1);
    }

    // Let bash close gracefully
    proc->write("exit $?\n");
    proc->waitForBytesWritten(-1);

    proc->closeWriteChannel();
    proc->waitForFinished();
    proc->waitForReadyRead();

    QByteArray read_data = proc->readAll();

    // The use of tr(read_data) also works here.
    QString output = tr(read_data);//QString::fromStdString (read_data.toStdString ());

    proc->closeReadChannel(QProcess::StandardOutput);

    proc->close();
    delete proc;

    // Add the output to the text box
    ui->textEdit->append (output);
}

MainWindow::~MainWindow()
{
    delete ui;
}

The header file...

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QMainWindow>

namespace Ui {
class MainWindow;
}

class MainWindow : public QMainWindow
{
    Q_OBJECT

public:
    explicit MainWindow(QWidget *parent = 0);
    ~MainWindow();

private:
    Ui::MainWindow *ui;
};

#endif // MAINWINDOW_H

The form file...

<?xml version="1.0" encoding="UTF-8"?>
<ui version="4.0">
 <class>MainWindow</class>
 <widget class="QMainWindow" name="MainWindow">
  <property name="geometry">
   <rect>
    <x>0</x>
    <y>0</y>
    <width>400</width>
    <height>300</height>
   </rect>
  </property>
  <property name="windowTitle">
   <string>MainWindow</string>
  </property>
  <widget class="QWidget" name="centralWidget">
   <widget class="QTextEdit" name="textEdit">
    <property name="geometry">
     <rect>
      <x>33</x>
      <y>19</y>
      <width>331</width>
      <height>211</height>
     </rect>
    </property>
   </widget>
  </widget>
  <widget class="QMenuBar" name="menuBar">
   <property name="geometry">
    <rect>
     <x>0</x>
     <y>0</y>
     <width>400</width>
     <height>19</height>
    </rect>
   </property>
  </widget>
  <widget class="QToolBar" name="mainToolBar">
   <attribute name="toolBarArea">
    <enum>TopToolBarArea</enum>
   </attribute>
   <attribute name="toolBarBreak">
    <bool>false</bool>
   </attribute>
  </widget>
  <widget class="QStatusBar" name="statusBar"/>
 </widget>
 <layoutdefault spacing="6" margin="11"/>
 <resources/>
 <connections/>
</ui>

The C source file...

int main(){
    // Intentionally will not build
    I will not build :)
}

My output looks like this:

QProcess gcc output

The output of the native Linux terminal looks like this:

Linux terminal gcc output with colors

Does anyone know how I might go about keeping the ANSI escape color sequences in the QProcess output so I can simulate the Linux terminal colors?

As a side note I have dug around in the Qt Creator source code and there is a class which can convert ANSI escape colors to Rich Text colors so I know someone has been down this road. Then again, when building projects, Qt Creator does not colorize the build output in it's own Terminal for some reason.

score 3 · Accepted Answer · answered Sep 22 '17 at 19:21

3

QProcess doesn't interfere with the process output, it's just that gcc - as many other programs that emit colored output - by default emit color escape sequences only when it detects that it's writing on a TTY device.

If you want to disable this heuristic and ask to always produce colored output, you have to add the -fdiagnostics-color=always option to the compiler command line.

answered Sep 22 '17 at 19:21

Matteo Italia

123,740
17
206
299

Thank you for your answer! This explains exactly the results I am having. Gcc is one of the many programs I will be running so the command line options may not always work. A quick google shows it may be possible to trick the programs into thinking they are writing to a TTY device. https://superuser.com/questions/352697/preserve-colors-while-piping-to-tee – konakid Sep 25 '17 at 21:46
@konakid: yep, that's a solution, although keep in mind that VT100 escape sequences encompass way more than just colors, so you'll probably have to heavily filter output from "smarter" programs. – Matteo Italia Sep 25 '17 at 22:02

score 3 · Answer 2 · answered Sep 28 '17 at 22:52

Thanks to a very insightful answer to my question, I was able to find a solution to my problem. I will share...

QProcess is not at fault nor is QString. The problem lies in the environment the programs are executed. Since the output for these programs (gcc etc) is not connected to a TTY device, all ANSI escape sequences are stripped. There is a way to trick the output to appear as if it were connected to a TTY device though.

Just prepend unbuffer to the command.

Since my use is actually creating a Qt Creator plugin, I was already linking against much of the Qt Creator source code. It just so happens a handy class named AnsiEscapeCodeHandler already exists to convert ANSI escape sequences into QTextCharFormat's and corresponding ANSI escaped sequence stripped strings.

To illustrate how I used this class, but now in my example, I will just copy the ansieescapecodehandler.h and ansiescapecodehandler.cpp to my test project from the downloadable Qt Creator source code. I had to delete a few lines from the AnsiEscapeCodeHandler source files to compile outside the context of the rest of the Qt Creator source, but that was it.

The new source file...

#include "mainwindow.h"
#include "ui_mainwindow.h"
#include <QString>
#include <QProcess>
#include <QStringList>

MainWindow::MainWindow(QWidget *parent) :
    QMainWindow(parent),
    ui(new Ui::MainWindow)
{
    ui->setupUi(this);

    QStringList input = {"unbuffer gcc will_not_build.c"};
    QProcess * proc = new QProcess();

    proc->setReadChannel(QProcess::StandardOutput);
    proc->setProcessChannelMode(QProcess::MergedChannels);
    proc->setWorkingDirectory("/path/to/test/c/file/");

    //Start bash
    proc->start("bash");
    proc->waitForStarted();

    // Write as many commands to this process as needed
    foreach(QString str, input){
        proc->write(str.toUtf8() + "\n");
        proc->waitForBytesWritten(-1);
    }

    // Let bash close gracefully
    proc->write("exit $?\n");
    proc->waitForBytesWritten(-1);

    proc->closeWriteChannel();
    proc->waitForFinished();
    proc->waitForReadyRead();

    QByteArray read_data = proc->readAll();

    // The use of tr(read_data) also works here.
    QString output = tr(read_data);//QString::fromStdString (read_data.toStdString ());

    proc->closeReadChannel(QProcess::StandardOutput);

    proc->close();
    delete proc;

    // Strip default character set escape sequences, since those seem to be left
    // See https://stackoverflow.com/questions/36279015/what-does-x1bb-do
    output.remove("\x1b(B", Qt::CaseInsensitive);

    // Since it is just one single text stream define here instead of globally
    Utils::AnsiEscapeCodeHandler ansi_handler;

    FormattedTextList result = ansi_handler.parseText (Utils::FormattedText(output, ui->textEdit->currentCharFormat ()));

    // Loop through the text/format results
    foreach(Utils::FormattedText ft, result){
        ui->textEdit->setCurrentCharFormat (ft.format);
        ui->textEdit->insertPlainText (ft.text);
    }
}

MainWindow::~MainWindow()
{
    delete ui;
}

The new header file...

#ifndef MAINWINDOW_H
#define MAINWINDOW_H

#include <QMainWindow>

// This exists in the qtcreator-src code and handles ansi escape code color parsing
#include "ansiescapecodehandler.h"

namespace Ui {
class MainWindow;
}

class MainWindow : public QMainWindow
{
    Q_OBJECT

public:
    explicit MainWindow(QWidget *parent = 0);
    ~MainWindow();

private:
    Ui::MainWindow *ui;

    typedef QList<Utils::FormattedText> FormattedTextList;
};

#endif // MAINWINDOW_H

The new colorized output... QProcess gcc output

Keep ANSI Escape Sequences in QProcess Output

2 Answers2