How to not escape backslash when writing to JSON

Question

I am trying to read a JSON file and add a new key,value pair to it. Then write the JSON file back. Example

# Read the JSON and add a new K,V
input_file_path = os.path.join(os.path.dirname(__file__), 'test.json')
input_file = open(input_file_path, 'r')
data = json.load(input_file)
data['key'] = os.environ.get('KEY', '') #<-- Adding a new K,V
input_file.close()

# Write the JSON to tmp.
output_file = open('/tmp/' + 'test.json', 'w')
json.dump(data, output_file, indent=4)
output_file.close()

My input JSON looks like this

{
  "type": "account"
}

My env variable called KEY looks like this -----BEGINKEY-----\nVgIBMIIE

The final JSON file written to tmp looks like this

{
  "private_key": "-----BEGINKEY-----\\nVgIBMIIE",
  "type": "account"
}

I am not able to figure out why the backslash is escaped? How can I avoid this?

No I am writing to `/tmp`. That part of the logic works fine. When I open the file `\n` in the value is escaped. — yesh, Jul 25 '19 at 01:42
Ah yea `/tmp` error while I was copy pasting the code here. Wanted to just paste the relevant part. — yesh, Jul 25 '19 at 01:43
Have you tried with `ensure_ascii=False` argument to json.dump? — ipaleka, Jul 25 '19 at 01:50

Akaisteph7 · Accepted Answer · 2019-07-25T02:14:22.727

The program is treating your input string as a raw string and so adds the extra \. The original '\' is not actually escaping anything, so for it to be represented in Python as a string, you need to escape it. As you saw this can be problematic however. You can force the string back to unicode format using:

import codecs

raw_val = os.environ.get('KEY', '')
val = codecs.decode(raw_val, 'unicode_escape')
data['key'] = val

score 0 · Answer 2 · answered Jul 25 '19 at 01:50

0

If you want to represent -----BEGINKEY-----\nVgIBMIIE in JSON, it has to go in a double quoted string.

In double quoted strings, backslashes have special meaning. It's not possible to write a backslash character simply by typing "\". This would generate an error.

Therefore, the backslash must be escaped. Hence -----BEGINKEY-----\nVgIBMIIE becomes "-----BEGINKEY-----\\nVgIBMIIE" in a JSON string.

If you want to get rid of the backslash, you need to represent

-----BEGINKEY-----
VgIBMIIE

rather than

-----BEGINKEY-----\nVgIBMIIE

These two things are not the same thing.

answered Jul 25 '19 at 01:50

Shawn Presser

69
5

He's reading json file and, I guess, wants to save it back as is. – ipaleka Jul 25 '19 at 01:53
They value that is stored in the env variable called KEY is of the form `-----BEGINKEY-----\nVgIBMIIE`. It is a huge string which is a private key. I am adding the private key into the JSON. I don't have control over how its formatted and I can't have it as line separated. – yesh Jul 25 '19 at 01:57

How to not escape backslash when writing to JSON

2 Answers2