Error while parsing Japanese Kanji with RapidXml

Question

I have a problem when I try to parse a xml file containing a specific Kanji:

退

After debugging, I see that the problem is in this function of RapidXml :

struct text_pure_no_ws_pred
{
    static unsigned char test(Ch ch)
    {
        return internal::lookup_tables<0>::lookup_text_pure_no_ws[static_cast<unsigned char>(ch)];
    }
};


const unsigned char lookup_tables<Dummy>::lookup_text_pure_no_ws[256] = 
    {
      // 0   1   2   3   4   5   6   7   8   9   A   B   C   D   E   F
         0,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // 0
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // 1
         1,  1,  1,  1,  1,  1,  0,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // 2
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  0,  1,  1,  1,  // 3
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // 4
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // 5
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // 6
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // 7
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // 8
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // 9
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // A
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // B
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // C
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // D
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  // E
         1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1,  1   // F
    };

where ch is the kanji 退. This function returns false. Why? With all the others characters, it returns true. Do you have any idea?

score 1 · Accepted Answer · answered Apr 24 '12 at 08:23

1

It looks like Ch contains a Unicode value. static_cast<unsigned char>(0x9000) is 0.

You need a table that holds a lot more than 256 values.

answered Apr 24 '12 at 08:23

Windows programmer

7,871
1
22
23

score 1 · Answer 2 · edited May 23 '17 at 12:21

1

RapidXML does not support full Unicode only UTF-8.

http://rapidxml.sourceforge.net/manual.html#namespacerapidxml_1character_types_and_encodings

~~See: Rapidxml and UTF8~~

The only options you have are: Convert the Kanji to UTF-8 and hope it works. Convert to non Unicode code-page and hope that that works with RapidXML.

edited May 23 '17 at 12:21

Community

1
1

answered Apr 24 '12 at 08:31

RedX

14,749
1
53
76

"UTF-8" is pretty much "full Unicode". Perhaps you wanted to say that RapidXML does not fully support UTF-16/. – himself Dec 17 '12 at 13:24

Error while parsing Japanese Kanji with RapidXml

2 Answers2