JSON value is converted to unicode - python

Question

My current Python Appium test automation framework reads data from JSON file using the json.load()

The value "Ελλάδα" stored in JSON is converted to "Î•Î»Î»Î¬Î´Î±" when json.load() method is called.

Pleases point me to a solution were I can maintain the the actual string value "Ελλάδα"

JSON data in testdata.json

{
    "country" : {"country_name" : "Ελλάδα"}
}

JSON load in .py file

with open(file_location) as test_data_obj: 
    pytest.data = json.load(test_data_obj)

This prints out "Î•Î»Î»Î¬Î´Î±"

print(pytest.data["country"]["country_name"])

Your code (which you didn't post) has a bug. Python 3 strings are Unicode. This page is UTF8, just like 99.9% of web pages. What you posted is what you'd get if you tried to read this page as if it was Latin1 - the bytes that represent a single character in UTF8 were treated as individual characters — Panagiotis Kanavos, Oct 26 '22 at 07:35
Did you try opening a text file with a hard-coded encoding? Did you use Python 2 instead of 3? — Panagiotis Kanavos, Oct 26 '22 at 07:43
To get what you posted I'd have to write `"Ελλάδα".encode('utf-8').decode('latin-1')` in Python. — Panagiotis Kanavos, Oct 26 '22 at 07:43
Very strange i am exactly getting what is intended, somehow your code is not actually reproducing the problem that's because this webpage has changed the character encoding which are are copying from the code — Zain Ul Abidin, Oct 26 '22 at 07:53

score 2 · Accepted Answer · answered Oct 26 '22 at 07:46

Use with open(file_location,encoding='utf-8'). For whatever reason, your system uses Latin 1 as the default encoding instead of UTF8.

The file is fine. This page, like 99.9% of web pages is UTF8. That's why you were able to write Ελλάδα without any problem.

Python 3 strings are Unicode too. If you type "Ελλάδα" in a Python console you'll get the string back.

To get what you posted I had to decode the bytes using latin-1 :

>>> "Ελλάδα".encode('utf-8').decode('latin-1')
'Î\x95Î»Î»Î¬Î´Î±'

1 Answers1