0

Recently our software has had an issue with certain obscure kanji (Chinese characters) not being picked up by our Shift-JIS encoding. I made an algorithm to read through any Shift-JIS string to try to find any "out of bounds" Kanji and switch the string to UTF-8 instead (which has more characters, but uses more space).

In order to find what Kanji won't be covered, I need to get my hands on a character map of the ibm-943_P14A-2000 encoding.

Where does one go about finding maps for these character sets? It's pretty easy via web search to find UTF8 lookups and the like, but I simply cannot find a chart/table/file of what values correspond to what values in this encoding.

If you could point me in any direction, no matter how obscure, I'd be very grateful.

HaveNoDisplayName
  • 8,291
  • 106
  • 37
  • 47
MojiBake
  • 23
  • 3

1 Answers1

2

The ICU project has a fairly large set of character set mapping tables, including ibm-943_P14A-1999. The difference between '1999' and '2000' is explained in this thread—and you can check out older versions of the ICU source code for the old table. The format of the table is described in the ICU User Guide.

As for the original character mappings (the character set of IBM-943), they are documented here.

一二三
  • 21,059
  • 11
  • 65
  • 74
  • Thanks! I had seen one of these .ucm files, I just wasn't sure if they were of significance. – MojiBake Aug 13 '15 at 15:44
  • After more research, I think some of these roundtrip values may be incorrect. fb8d is the one giving me problems. Check this other source where they mention a lot more values with roundtrip problems: https://support.microsoft.com/en-us/kb/170559 Then again, with encoding it could be anything. – MojiBake Aug 13 '15 at 17:53