7

The hex string '\xd3' can also be represented as: Ó.

The easiest way I've found to get the character representation of the hex string to the console is:

print unichr(ord('\xd3'))

Or in English, convert the hex string to a number, then convert that number to a unicode code point, then finally output that to the screen. This seems like an extra step. Is there an easier way?

dreftymac
  • 31,404
  • 26
  • 119
  • 182
Kevin Burke
  • 61,194
  • 76
  • 188
  • 305

3 Answers3

12
print u'\xd3'

Is all you have to do. You just need to somehow tell Python it's a unicode literal; the leading u does that. It will even work for multiple characters.

If you aren't talking about a literal, but a variable:

codepoints = '\xd3\xd3'
print codepoints.decode("latin-1")

Edit: Specifying a specific encoding when printing will not work if it's incompatible with your terminal encoding, so just let print do encode(sys.stdout.encoding) automatically. Thanks @ThomasK.

agf
  • 171,228
  • 44
  • 289
  • 238
  • But, what do you do if you have a string composed of code points? How would you convert the string to the actual string representation? Is `u` a function? Can we write `u(some_string)`? – Geo Aug 09 '11 at 17:00
  • @Geo: `u` is not a function, it's part of the string literal syntax. But what is a 'string composed of code points'? – Thomas K Aug 09 '11 at 17:04
  • @agf: OK, great. btw, I think `print u'\xd3'.encode("latin-1")` is wrong - on a UTF-8 terminal, that shows nothing at all. – Thomas K Aug 09 '11 at 17:09
  • What is latin-1? I was stuck because I was trying `.encode("utf-8")` and getting an error. – Kevin Burke Aug 09 '11 at 17:09
  • 1
    @ThomasK I can't check because I think my terminal is latin-1 or that silly, almost the same winodws encoding. It worked-for-me. – agf Aug 09 '11 at 17:11
  • @Kevin latin-1 is another ascii compatible encoding. It's similar to cp-whatever that is a common windows encoding so it often works when utf-8 fails because you're not actually using utf-8. You should read http://docs.python.org/howto/unicode.html. `\xd3` doesn't seem to be valid by itself in utf-8, but part of a two byte sequence. `print u'\xd3'.encode("utf-8")` gives two characters. – agf Aug 09 '11 at 17:12
  • @agf: latin-1 or c1252 (almost the same) is a fallback for *decoding*, because it can accept any byte. It doesn't work the other way, because it will only encode certain unicode characters. UTF-8 will encode any character. For printing cross platform, try `.encode(sys.stdout.encoding)` (or just print the unicode - CPython at least does the same thing inside `print`). – Thomas K Aug 09 '11 at 17:18
  • 2
    It’s a bit better in Python3 (Python2 is being sunsetted) but Unicode and Python do not get along very well even on the best of days, and you cannot use Python’s `re` library for Unicode per UTS#18’s level-1 reqs. Matthew Barnett’s [regex library](http://pypi.python.org/pypi/regex) for both Python2 and Python3 helps a lot. See the chart on Slide 6 of [my Unicode Support Shootout talk](http://training.perl.com/OSCON2011/index.html) to see how dramaticaly better MRAB’s `regex` lib is than normal Python `re`. Alas,you still have Python’s **fatal multiple personality disorder** on UCS‐²⁄₄ though. – tchrist Aug 09 '11 at 19:33
  • 2
    **Moderator Note** _Several comments under this answer were removed because they provided more noise than signal. Please try to keep comments constructive and on topic, related to the answer at hand. If you want to have a 'side chat', use the chat system._ – Tim Post Aug 11 '11 at 07:36
1

if data is something like this "\xe0\xa4\xb9\xe0\xa5\x88\xe0\xa4\xb2\xe0\xa5\x8b \xe0\xa4\x95\xe0\xa4\xb2"

sys.stdout.buffer.write(data)

would print 

हैलो कल
mouserat
  • 1,565
  • 13
  • 9
0

Not long ago, I had a very similar problem. I had to decode files that contained unicode hex (e.g., _x0023_) instead of special characters (e.g., #). The solution is described in the following code:

Script

from collections import OrderedDict
import re


def decode_hex_unicode_to_latin1(string: str) -> str:
    hex_unicodes = list(OrderedDict.fromkeys(re.findall(r'_x[?:\da-zA-Z]{4}_', string)))

    for code in hex_unicodes:
        char = bytes.fromhex(code[2:-1]).decode("latin1")[-1]
        string = string.replace(code, char)

    return string


def main() -> None:
    string = "|_x0020_C_x00f3_digo_x0020_|"
    decoded_string = decode_hex_unicode_to_latin1(string)
    print(string, "-->", decoded_string)

    return


if __name__ == '__main__':
    main()

Output

|_x0020_C_x00f3_digo_x0020_| --> | Código |
joao8tunes
  • 81
  • 5
  • I think you can re-write that loop with just `decoded_string = re.sub(r'_x([a-fA-F0-9]{4})_', lambda x: bytes.fromhex(x.group(1)).decode("utf-8"), string)`. – Wiktor Stribiżew Feb 10 '23 at 09:40