Invalid unicode character causing MySQL string error

Question

I need to add a record to our MySQL database (via Omeka) that includes an invalid unicode character (this one)

The error message I get via Omeka is:

Mysqli statement execute error : Incorrect string value: '\xF0\xAA\xA8\xA7\xE7\x94...' for column 'text' at row 1

The database field is longtext with collation utf8_unicode_ci. There are already a lot of records in this table and I'm not quite sure what I should change without affecting the other data already in it. Suggestions?

are you using prepared statements to insert this value rather than some string concatenation? Some chinese characters will require utf8mb4. The utf8 is only the 3 character version. — danblack, Dec 12 '18 at 10:22
I'm using Omeka; it's a standard webapp in php that inserts the values. — Another_Omeka_User, Dec 17 '18 at 12:35

score 0 · Answer 1 · answered Dec 14 '18 at 02:50

0

ALTER TABLE tbl CONVERT TO utf8mb4;

Meanwhile, the text for that row in that column is probably truncated or the whole row is missing.

As best as I can tell, F0AAA8A7 is not yet assigned, but I think it is in the area of Chinese characters, not Emoji, which also need utf8mb4. It is Unicode "codepoint" 2AA27.

answered Dec 14 '18 at 02:50

Rick James

135,179
13
127
222

Yes, it's this Chinese character: http://www.fileformat.info/info/unicode/char/2aa27/index.htm. I make most MySQL changes via phpMyAdmin these days, so I can change to utf8mb4 that way (utf8mb4_unicode_ci). I'll try it out after making a backup. – Another_Omeka_User Dec 17 '18 at 12:37
@Another_Omeka_User - Aha. Firefox can render that; Chrome cannot. fileformat.info uses an image; scarfboy.com (http://unicode.scarfboy.com/?s=%F0%AA%A8%A7) expects the font to take care of rendering. – Rick James Dec 17 '18 at 15:06

Invalid unicode character causing MySQL string error

1 Answers1