Passing unicode arguments to subprocess in ShellExecuteEx

Question

First of all, I'm new to using Windows' C++ API, so there's probably something obvious I'm missing.

I'm trying to start an elevated subprocess in C++ in Windows. I managed to write this code which starts an elevated subprocess, passes 2 arguments to it and then the subprocess pops up a window with those arguments:

#include <shlobj.h>
#include <shlwapi.h>
#include <objbase.h>
#include <string>
#include <QString>
#include <QDebug>
#include <QMessageBox>
#include <QApplication>

auto getWinError()
{
    auto dw =GetLastError();
    LPTSTR* lpMsgBuf;
    FormatMessage(
                FORMAT_MESSAGE_ALLOCATE_BUFFER |
                FORMAT_MESSAGE_FROM_SYSTEM |
                FORMAT_MESSAGE_IGNORE_INSERTS,
                NULL,
                dw,
                MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
                (LPTSTR) &lpMsgBuf,
                0, NULL );
    auto str = QString::fromWCharArray(*lpMsgBuf);
    LocalFree(*lpMsgBuf);
    return str;
}

int main(int argc, char** argv)
{
    if (argc == 1){
        // start self as admin with 2 arguments
        SHELLEXECUTEINFO info = {};
        info.cbSize = sizeof(SHELLEXECUTEINFO);
        info.fMask = SEE_MASK_NOCLOSEPROCESS | SEE_MASK_UNICODE;
        info.lpVerb = L"runas";
        auto filestr = QString{argv[0]}.toStdWString();
        info.lpFile = filestr.c_str();
        info.lpParameters = LR"("first parameter" "żółć")";
        info.nShow = SW_SHOW;

        auto success = ShellExecuteEx(&info);
        if (!success || (int)info.hInstApp <= 32){
            qDebug()<<getWinError();
            return -1;
        }
        HANDLE handle = info.hProcess;

        auto exitCode = [handle]{
            DWORD status;
            GetExitCodeProcess(handle, &status);
            return status;
        };

        while (exitCode() == STILL_ACTIVE) Sleep(100);

        qDebug()<<"process exited with exit code "<<exitCode();

        CloseHandle(handle);
        return 0;
    } else {
        // show popup with arguments
        QApplication a(argc, argv);
        QStringList s;
        for (int i = 1; i<argc; i++){
            s += argv[i];
        }
        QMessageBox::information(0, "", s.join('\n'));
        return 42;
    }
}

It mostly works, but it mangles non-ASCII characters:

What should I change to make it handle unicode correctly?

no, `ShellExecuteEx` not exist. this is macro which expanded to `ShellExecuteExW` or `ShellExecuteExA`. you need call `ShellExecuteExW` — RbMm, Jul 13 '17 at 12:20
Thanks for the hint. I was calling `ShellExecuteW`, when I changed it to `ShellExecuteA` it worked. — user697683, Jul 13 '17 at 12:22
but call *A* version of any api - bad choice - you need almost always call *W* version and pass wide string arguments to it (bat not ansi strings) — RbMm, Jul 13 '17 at 12:24
Well, the W version mangled my strings, while the A version did not. If you have a better solution, please say so. — user697683, Jul 13 '17 at 12:27
no, **W** version not `mangled` your strings. simply you pass to it incorrect strings. you must use unicode (wide char) strings only. if strings displayed incorectly - this is only because you pass incorrect string to **W** version. and any way - **A** version call internal **W** vesrion after convert ansi strings to unicode. this is why need use **W** — RbMm, Jul 13 '17 at 12:38
The most correct way to deal with non ASCII characters in Windows applications is to make the application use Unicode encoding. Your problem is that your application is using "main" as its entry point - which accepts arguments as "chars" which will be encoded using an 8bit codepage. Complicated. See here for how to switch to using "wmain": [Using wmain (MSDN)](https://msdn.microsoft.com/en-us/library/bky3b5dh.aspx) — Chris Becke, Jul 13 '17 at 13:38
my previous comment is a comment, not an answer, as I don't have the time or inclination to figure out how to make QT use a unicode friendly entrypoint. The windows API is natively Unicode, so every time characters have to make a round trip through an 8 bit encoding there is a risk that codepages will not be set up / not support the characters at all. — Chris Becke, Jul 13 '17 at 13:43
@RbMm the code in question is using wide strings when calling `ShellExecute` and `FormatString`, so clearly the project is configured for Unicode and thus will be calling `ShellExecuteW` — Remy Lebeau, Jul 13 '17 at 15:38
@ChrisBecke the whole `wmain` issue is moot, because [`QApplication` ignores the `argc`/`argv` values on Windows](https://stackoverflow.com/questions/4072016/) and uses `GetCommandLine()` instead so it can grab Unicode command-line data. So just pass NULL for `argv` — Remy Lebeau, Jul 13 '17 at 15:42
When using wide string literals in source code like `LR"("first parameter" "żółć")"`, make sure to save the source file with a Unicode encoding (preferably UTF-8 with BOM). — zett42, Jul 13 '17 at 16:06
@RemyLebeau - but he is literally using argc and argv directly to build his message string. — Chris Becke, Jul 13 '17 at 16:15
@ChrisBecke: good point, I missed that loop. He should be using the `QApplication::arguments()` method instead of building the `QStringList` manually: `QStringList s = a.arguments();` — Remy Lebeau, Jul 13 '17 at 17:12

Remy Lebeau · Answer 1 · 2017-07-13T17:45:09.790

The problem is that you are relying on the argv array of the narrow version of main(), which cannot receive Unicode input on Windows. You would need to use the wide version of wmain() instead.

However, QApplication parses the command-line for you. On Windows, QApplication ignores argv and instead uses the Win32 API GetCommandLine() function so it can receive Unicode input. But you are going straight to argv instead of using what Qt parsed. You should be using the QApplication::arguments() method to retrieve the parsed command-line as a QStringList.

You are also using FormatMessage() incorrectly. And you should use WaitForSingleObject() instead of a GetExitCodeProcess() loop.

Try this instead:

#include <windows.h>
#include <shlobj.h>
#include <shlwapi.h>
#include <objbase.h>

#include <string>

#include <QString>
#include <QDebug>
#include <QMessageBox>
#include <QApplication>

QString getWinError()
{
    DWORD dw = GetLastError();
    LPWSTR lpMsgBuf = NULL;
    FormatMessageW(
                FORMAT_MESSAGE_ALLOCATE_BUFFER |
                FORMAT_MESSAGE_FROM_SYSTEM |
                FORMAT_MESSAGE_IGNORE_INSERTS,
                NULL,
                dw,
                MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
                (LPWSTR) &lpMsgBuf,
                0, NULL );
    QString str = QString::fromWCharArray(lpMsgBuf);
    LocalFree(lpMsgBuf);
    return str;
}

int main(int argc, char** argv)
{
    if (argc == 1)
    {
        // start self as admin with 2 arguments
        SHELLEXECUTEINFOW info = {};
        info.cbSize = sizeof(info);
        info.fMask = SEE_MASK_NOCLOSEPROCESS | SEE_MASK_UNICODE;
        info.lpVerb = L"runas";

        WCHAR szFileName[MAX_PATH] = {};
        GetModuleFileNameW(NULL, szFileName, MAX_PATH);
        info.lpFile = szFileName;

        /* alternatively:

        std::wstring filePath = QCoreApplication::applicationFilePath().toStdWString();
        info.lpFile = filePath.c_str();

        or:

        QString filePath = QCoreApplication::applicationFilePath();
        info.lpFile = (LPCWSTR) filePath.utf16();
        */

        info.lpParameters = L"\"first parameter\" \"żółć\"";
        info.nShow = SW_SHOW;

        if (!ShellExecuteEx(&info)){
            qDebug() << getWinError();
            return -1;
        }

        WaitForSingleObject(info.hProcess, INFINITE);

        DWORD status = 0;
        GetExitCodeProcess(info.hProcess, &status);

        CloseHandle(info.hProcess);

        qDebug() << "process exited with exit code " << status;

        return 0;
    }
    else
    {
        // show popup with arguments
        QApplication a(argc, argv);
        QStringList s = a.arguments();
        QMessageBox::information(0, "", s.join('\n'));
        return 42;
    }
}

But polling via `GetExitCodeProcess` is fine if I don't want to block the thread until the process finishes, right? — user697683, Jul 13 '17 at 20:06
You can do the same thing with `WaitForSingleObject()` using a non-infinite timeout, even 0. The process handle is signaled when the process exits. But in your earlier example, you were simply calling `Sleep()` in a loop until the process exits, so you were blocking the calling thread. That is better handled with a single call to `WaitForSingleObject(INFINITE)` instead of a `while (STILL_ACTIVE)` loop. — Remy Lebeau, Jul 13 '17 at 20:09

Passing unicode arguments to subprocess in ShellExecuteEx

1 Answers1