PostgreSQL C++ libpq encoding UTF-8 problem

Question

I don't have problem inserting UTF-8 (korean character in my case) using direct sql command in PGAdmin or psql console. But when I need to insert values in C++ by using libpq library, I got the encoding error and I couldn't resolve it until now.

I have tested DB connection, etc and works well. So I will just share the insertion code only :

    /* INSERT demo */
    res = PQexec(conn,
        "insert into db_test values('testval', '군포지사','N00225','영동선','0500','E',13.67,14.18);");
    if (PQresultStatus(res) != PGRES_COMMAND_OK) {
        std::cout << "Insert into table failed: " << PQresultErrorMessage(res)
            << std::endl;
    }
    else
        std::cout << "inserted" << std::endl;
    PQclear(res);

Below is the error :

Insert into table failed: ERROR:  invalid byte sequence for encoding "UTF8": 0xb1

Is your source code saved as utf-8? `char` literals are normally encoded in the encoding of your source file. Use the [u8 literal](https://en.cppreference.com/w/cpp/language/string_literal) to guarantee the encoding — Alan Birtles, May 22 '20 at 06:57
@AlanBirtles yes, I checked using Notepad++ and it was saved in UTF-8 — gameon67, May 22 '20 at 07:18

score 1 · Accepted Answer · answered May 22 '20 at 02:12

I have managed to solve it, sorry for my lack of understanding in encoding processing. I just need to convert ANSI to UTF-8, below is my working code. If there's a better solution, please post your answer.

int AnsiToUTF8(char* szSrc, char* strDest, int destSize)
{
    WCHAR szUnicode[255];
    char szUTF8code[255];
    int nUnicodeSize = MultiByteToWideChar(CP_ACP, 0, szSrc, (int)strlen(szSrc), szUnicode, sizeof(szUnicode));
    int nUTF8codeSize = WideCharToMultiByte(CP_UTF8, 0, szUnicode, nUnicodeSize, szUTF8code, sizeof(szUTF8code), NULL, NULL);
    assert(destSize > nUTF8codeSize);
    memcpy(strDest, szUTF8code, nUTF8codeSize);
    strDest[nUTF8codeSize] = 0;
    return nUTF8codeSize;
}

/* INSERT demo */
    char queryUtf8[100];
    std::string queryAnsi = "insert into db_test values('testval', '군포지사','N00225','영동선','0500','E',13.67,14.18);";
    AnsiToUTF8(&queryAnsi[0u], queryUtf8, 100);

    res = PQexec(conn, queryUtf8);
    if (PQresultStatus(res) != PGRES_COMMAND_OK) {
        std::cout << "Insert into table failed: " << PQresultErrorMessage(res)
            << std::endl;
    }
    else
        std::cout << "inserted" << std::endl;
    PQclear(res);

score 1 · Answer 2 · answered May 22 '20 at 07:44

1

The proper solution is to set the client_encoding parameter to the correct value (the encoding your client uses). You can do that when connecting to the database:

PGconn *conn;

conn = PQconnectdb("user=carl password=secret client_encoding=LATIN1");

answered May 22 '20 at 07:44

Laurenz Albe

209,280
17
206
263

I changed to `UTF8` but still got the same error msg – gameon67 May 24 '20 at 23:55
Right, you should *not* change to UTF8 (unless I misunderstand what you mean by "change": change what?). `client_encoding` has to match the encoding of the data you feed the client. – Laurenz Albe May 25 '20 at 06:15

PostgreSQL C++ libpq encoding UTF-8 problem

2 Answers2