Printing wchar_t to console in C++

Question

I have made a simple program that will read in a PS1 Memory Card binary file and display it's contents in the console using C++ in Visual Studio. The game titles are encoded in the memory in Shift-JIS format, so I used the MultiByteToWideChar function to convert them:

            // Converting Shift-JIS
            char lTitle[65];
            strcpy_s(lTitle, mymemcard[lFrame - 1].title);

            int lTitleChars = MultiByteToWideChar(932, 0, lTitle, -1, NULL, 0);
            wchar_t* lTitleL = new wchar_t[lTitleChars];
            MultiByteToWideChar(932, 0, lTitle, -1, lTitleL, lTitleChars);

My problem is now that I can't get the lTitleL variable to print to the console. I've tried cout, wcout, printf, wprintf, I can't get them to work! I know that the lTitleL variable proiperly holds the title since I can see it in the debugger. When I call any of the print functions I've tried, simply nothing gets printed. ANy help would be appreciated! Thanks.

BIG EDIT:

OK, so here's where I am with this:

            // Converting Shift-JIS to UTF-8 //
            ///////////////////////////////////
            // First, convert the multi-byte Shift-JIS format into wide characters
            strcpy_s(lTitle, mymemcard[lFrame - 1].titleMB);
            lTitleChars = MultiByteToWideChar(932, 0, lTitle, -1, NULL, 0);
            wchar_t* lTitleFW = new wchar_t[lTitleChars];
            MultiByteToWideChar(932, 0, lTitle, -1, lTitleFW, lTitleChars);
            wprintf(L"FW, WriteConsoleW - ");
            WriteConsoleW(consoleHandle, lTitleFW, lTitleChars, NULL, 0);
            wprintf(L"\n");
            // Memory card seems to store all characters in their "Full Width" forms, we need to convert them to Half-Width so they display nicely in the console
            wchar_t* lTitleHW = new wchar_t[lTitleChars];
            LCMapStringEx(LOCALE_NAME_USER_DEFAULT, LCMAP_HALFWIDTH, lTitleFW, lTitleChars, lTitleHW, lTitleChars, NULL, NULL, 0);
            wprintf(L"HW, WriteConsoleW - ");
            WriteConsoleW(consoleHandle, lTitleHW, lTitleChars, NULL, 0);
            wprintf(L"\n");
            wprintf(L"HW, wprintf()     - %s\n", lTitleHW);
            // Now we can convert it into UTF-8 format to allow it to be printed using std::cout
            // This step isn't necessary as we could call WriteConsoleW on lTitleHW instead, but why not
            lTitleChars = WideCharToMultiByte(CP_UTF8, 0, lTitleHW, -1, NULL, 0, NULL, NULL);
            char* lTitleUTF = new char[lTitleChars];
            WideCharToMultiByte(CP_UTF8, 0, lTitleHW, -1, lTitleUTF, lTitleChars, NULL, NULL);
            strcpy_s(mymemcard[lFrame - 1].titleUTF, lTitleUTF);
            wprintf(L"UTF, wprintf()    - %S\n\n", lTitleUTF);
            delete[] lTitleFW;
            delete[] lTitleHW;
            delete[] lTitleUTF;
            ///////////////////////////////////

• MultiByteToWideChar to get the Shift-JIS input string into wide chars.
• Print that to console for debugging.
• Problem now is that the PS1 Memory Card stores all characters in their full-width forms, so I use LCMapStringEX to convert them to half-width for nicer output.
• Print that to console for debugging.
• This is enough if I am happy to use WriteConsoleW, but if not then one problem is that certain punctuations are encoded strangely and don't print nicely using std::wcout or wprintf(). For example, hyphens are stored as U+FF70 - "Halfwidth Katakana-Hiragana Prolonged Sound Mark" (after converting to half-width) and these just show up as question marks regardless of the font being used (they do, however, print correctly using WriteConsoleW).
• So, I can now use WideCharToMultiByte, using the UTF-8 code page to get a version of the string that prints nicely to the console using std::wcout or wprintf(). I do, however, have to call both SetConsoleCP(65001) and SetConsoleOutputCP(65001) for them to print properly, otherwise the multi-byte chars (like the U+FF70 I mentioned) get printed byte-by-byte.
• Lastly, for any of the unusual characters to show up, I need to have a font selected that can display them. The only ones I have found that work in the default fonts in my console are NSimSun and SimSun-ExtB, none of the others seems to contain the Japanese characters (at least for the U+FF70 character).

To help visualise the output of this, here's a screenshot of the console using the NSimSun font:

And here with Consolas font:

Now for my question: I don't particularly like the NSimSun font, are there any others out there that look more like the Consolas font that do contain all the glyphs for the full/half width Japanese characters? If so, how can I package them into my Console app so that it's portable?

Thanks!

Answer 1

Windows has limited support for printing Unicode on console. Test your conversion by using MessageBoxW(0, lTitleL, 0, 0) which is guaranteed to succeed, as long as the conversion was okay.

To print to console window, you have to call _setmode(_fileno(stdout), _O_U16TEXT); in order to print characters out of ASCII range.

Stick with std::wcout or wprintf(L"%s", str)

There is an additional problem if the console does not support the right font. You have to try different fonts. See example below. Also consider using std::wstring which has automatic cleanup.

#include <iostream>
#include <string>
#include <io.h> 
#include <fcntl.h> 
#include <Windows.h>

int main()
{
    const wchar_t *src = L"ABC 日本語";

    int size;
    size = WideCharToMultiByte(932, 0, src, -1, NULL, 0, 0, 0);
    std::string temp(size, 0);
    WideCharToMultiByte(932, 0, src, -1, temp.data(), size, 0, 0);

    size = MultiByteToWideChar(932, 0, temp.data(), -1, 0, 0);
    std::wstring dst(size, 0);
    MultiByteToWideChar(932, 0, temp.c_str(), -1, dst.data(), size);

    _setmode(_fileno(stdout), _O_U16TEXT);

    wprintf(L"%s\n", dst.c_str());

    //Try with different font
    CONSOLE_FONT_INFOEX font = { sizeof(font) };
    HANDLE hcout = GetStdHandle(STD_OUTPUT_HANDLE);
    GetCurrentConsoleFontEx(hcout, FALSE, &font);
    wcscpy_s(font.FaceName, L"MS Gothic");
    SetCurrentConsoleFontEx(hcout, FALSE, &font);
    std::wcout << dst << "\n";

    return 0;
}