UTF-16LE Encoding woes with Qt text editor written in C++

Question

So I have a QT text editor that I have started creating. I started with this http://doc.qt.io/archives/qt-5.7/gettingstartedqt.html and i have added on to it. So far I have added a proper save/save as function (the version in the link only really has a save as function), a "find" function, and an "open new window" function. Very soon, I will add a find and replace function.

I am mainly doing this for the learning experience, but I am also going to eventually add a few more functions that will specifically help me create PLC configuration files at work. These configuration files could be in many different encodings, but most of them seem to be in UTF-16LE (according to Emacs anyway.) My text editor originally had no problem reading the UTF-16LE, but wrote in plain text, I needed to change that.

Here is the snippet from the Emacs description of the encoding system of one of these UTF16-LE files.

U -- utf-16le-with-signature-dos (alias: utf-16-le-dos)
UTF-16 (little endian, with signature (BOM)).
Type: utf-16
EOL type: CRLF
This coding system encodes the following charsets:
  unicode

And here is an example of the code that I am using to encode the text in my QT text editor.

First... This is similar to the link that I gave earlier. The only difference here is that "saveFile" is a global variable that I created to perform a simple "Save" function instead of a "Save As" function. This saves the text as plain text and works like a charm.

void findreplace::on_actionSave_triggered()
{
    if (!saveFile.isEmpty())
    {
        QFile file(saveFile);
        if (!file.open(QIODevice::WriteOnly))
        {
            // error message

        }
        else
        {
            QTextStream stream(&file);
            stream << ui->textEdit->toPlainText();
            stream.flush();
            file.close();

        }
    }


}

Below is my newer version which attempts to save the code in "UTF-16LE." My text editor can read the text just fine after saving it with this, but Emacs will not read it at all. This to me says that the configuration file will probably not be readable by the programs that read it. Something changed, not sure what.

void findreplace::on_actionSave_triggered()
{
    if (!saveFile.isEmpty())
    {
        QFile file(saveFile);
        if (!file.open(QIODevice::WriteOnly))
        {
            // error message

        }
        else
        {
            QTextStream stream(&file);
            stream << ui->textEdit->toPlainText();
            stream.setCodec("UTF-16LE");
            QString stream3 = stream.readAll();
            //QString stream2 = stream3.setUnicode();
            //QTextCodec *codec = QTextCodec::codecForName("UTF-16LE");
            //QByteArray stream2 = codec->fromUnicode(stream3);
            //file.write(stream3);
            stream.flush();
            file.close();

        }
    }


}

The parts that are commented out I also tried, but they ended up writing the file as Asian (Chinese or Japanese) characters. Like I said my text editor, (and Notepad in Wine) can read the file just fine, but Emacs now describes the encoding as the following after saving.

= -- no-conversion (alias: binary)

Do no conversion.

When you visit a file with this coding, the file is read into a
unibyte buffer as is, thus each byte of a file is treated as a
character.
Type: raw-text (text with random binary characters)
EOL type: LF

This indicates to me that something is not right in the file. Eventually this text editor will be used to create multiple text files at once and modify their contents via user input. It would be great if I could get this encoding right.

Probably revealing my ignorance, here. I found something about that in my research, but it was unclear how to specifically write it using QTextCodec. — CigEmacs, Jan 14 '18 at 03:21
It's kind of pointless to set the codec *after* you've already written all the data to the stream. — Igor Tandetnik, Jan 14 '18 at 05:06

score 2 · Accepted Answer · answered Jan 14 '18 at 21:07

Thanks to the kind fellows that commented on my post here, I was able to answer my own question. This code here solved my problem.

void findreplace::on_actionSave_triggered()
{
    if (!saveFile.isEmpty())
    {
        QFile file(saveFile);
        if (!file.open(QIODevice::WriteOnly))
        {
            // error message

        }
        else
        {
            QTextStream stream(&file);
            stream.setCodec("UTF-16LE");
            stream.setGenerateByteOrderMark(true);
            stream << ui->textEdit->toPlainText();
            stream.flush();
            file.close();

        }
    }


}

I set the codec of the stream, and then set the the generate BOM to "True." I guess that I have more to learn about encodings. I thought that the byte order mark had to be set to a specific value or something.I wasn't aware that I just had to set this value to "True" and that it would take care of itself. Emacs can now read the files that are generated by saving a document with this code, and the encoding documentation from Emacs is the same. I will eventually add options for the user to pick which encoding they need while saving. Glad that I was able to learn something here.

UTF-16LE Encoding woes with Qt text editor written in C++

1 Answers1