c++ wchar_t array and char array in programming for win32 console

Question

I am writing a program includes output chinese characters using Dev C++.

I've added -finput-charset=big5 -fexec-charset=big5 in compiler parameters. I also set the code page of the console to be 950 (traditional chinese)

It works perfectly while in a simple cout like this:

cout << "中文字";

while it comes to characters array it goes wrong as expected:

char chin[] = "中文字"; 
cout << chin[0];//output nothing
cout << chin[0] << chin[1];//output the first chinese character as one chinese character occupies 2 bytes.

So I decided to use wchar_t instead and I have to use wcout with wchar_t or else a number will be shown.

However, wcout show nothing in the console. All of the below show nothing:

wcout << L"中文字";
wchar_t chin2[] = L"中文字";
wcout << chin2[0]; 

What did I missed to use wchar_t to output chinese (or other east asian) characters? I really don't want to write 2 array member to show one single chinese chracters.

Answer 1

There are subtle problems going on here.

The C++ compiler does not understand Big5 encoding. When you create a source code file and display it, you may see your familiar Chinese characters but the compiler sees a string of bytes. Big5 is a double byte charset so each input character will be represented by 2 bytes inside the compiler.

When that string of bytes is fed to a suitable output device the Chinese characters appear again. Code page 950 is compatible with Big5 so you see the "right" thing. But then you try to build on this and confusion is the result. Your second code sample uses L"" strings, but I expect those strings will contain half a character in each short.

The only "safe" character set you can use is Unicode. Windows internals are historically UCS-2 (char is a single short) but is now theoretically UTF-16 (char is short, but may include multi-byte sequences). Not all existing software and older APIs fully support UTF-16 (or need to). Windows has very limited support for UTF-8 or other encodings. Everything gets converted into Unicode, so best to just leave it that way.

In practice, you should build your C++ code with Unicode settings, for UCS-2, and exercise caution if you need characters that would require multibyte sequences. You should ensure that any source code you write and any input text files are identified as whatever encoding they need to be, but are translated into Unicode internally. Leave your console as the default Unicode encoding, and everything will just work.

It is almost impossible to sensibly use Big5 as an internal encoding in a Windows program. Best not to try.