boost::program_option::store throws exception when option string contains mixed language characters

Question

I have simple code that works perfectly well with input option contains just ASCII characters, but throws an exception with error message of "error: character conversion failed". Is there a solution?

Background info:

    1. Compiler and OS: VC++2012 running on Windows 8.1 64 bit    
    2. "_UNICODE" option is ON    It works with command like: tmain.exe --input
    3. "c:\test_path\test_file_name.txt"    It fails with command like:
         tmain.exe --input "c:\test_path\test_file_name_中文.txt"    My default
    4. I am using boost v1.53.
    5. locale is Australian English.

This is the source code:

#include <boost/program_options.hpp>
int _tmain(int argc, _TCHAR* argv[])
{
    try {
        std::locale::global(std::locale(""));
        po::options_description desc("Allowed options");
        desc.add_options()
            ("input",  po::value<std::string>(), "Input file path.")
        ;

        po::variables_map vm;        
        po::store(po::parse_command_line(argc, argv, desc), vm);
        po::notify(vm);    

        if (vm.count("input")) { 
            std::cout << "Input file path: " << vm["input"].as<std::string>();
        }
        return 0;
    }
    catch(std::exception& e) {
        std::cerr << "error: " << e.what() << "\n";
        return 1;
    }
    catch(...) {
        std::cerr << "Exception of unknown type!\n";
    }
    return 0;
}

I have stepped into the boost codes, and found that the exception was thrown from this function (boost_1_53_0\libs\program_options\src\convert.cpp):

BOOST_PROGRAM_OPTIONS_DECL std::string 
to_8_bit(const std::wstring& s, 
            const std::codecvt<wchar_t, char, std::mbstate_t>& cvt)
{
    return detail::convert<char>(
        s,                 
        boost::bind(&codecvt<wchar_t, char, mbstate_t>::out,
                    &cvt,
                    _1, _2, _3, _4, _5, _6, _7));
}

When I stepped into boost code, I found out that this statement

boost::program_options::parse_command_line(argc, argv, desc)

actually works fine, it is the boost::program_options::store() function that fails to convert string in utf-8 back to wstring. The reason of this failure could be that my current code-page does not support non-ASCII characters. I guess my code would work well if my current locale were a Chinese based one. Is there any solution to my problem? Many thanks in advance.

Answer 1

For an option to be Unicode aware it has to be added with wvalue instead of value. Try this:

desc.add_options()("input",  po::wvalue<std::string>(), "Input file path.");