Python program to change unicode characters to entities

Question

This is a program to change the unicode symbols to their respective entities from file x.input and the output should be placed in y.output. However, the program doesn't do that and only creates a copy of the file.

I have both Python27 and 35 displaying this issue and the platform is Win 7.

Where am I going wrong? Please help.

#!/usr/bin/env python
# -*- coding: utf-8 -*-
#import io

f1 = open('x.input').read()
f2 = open('y.output','w')
for line in f1:
    x = line.replace('“', '&#x201C;')
    f2.write(x)
#f1.close()
f2.close()

A screenshot of the entire program: Actual program with the double quote which is creating issues

you only replace all `"` with `“`. And you should close f1 — Yevhen Kuzmovych, Nov 11 '16 at 20:43
Have you ever worked with python before? This code is pretty simple to understand, so I'm struggling to understand what exactly YOU don't understand about it. — Charles D Pantoga, Nov 11 '16 at 20:50

maki · Accepted Answer · 2016-11-14T14:14:45.247

0

The issue is a bit tricky, you have a copy/paste error from a document, where the character “ (ord 226) is not the " you expect (ord 34) (note they are similar, but slightly different). Quite probably you copied this example from a Word document.

Just replace this character by the correct one and your program will work. The result should be (copy/paste from here so you get the correct char):

#!/usr/bin/env python
# -*- coding: utf-8 -*-
#import io

f1 = open('x.input').read()
f2 = open('y.output','w')
for line in f1:
    x = line.replace(ord(34), '&#x201C;')
    f2.write(x)
f1.close()
f2.close()

Even if not needed (the file will be closed when the program finishes), as good citizens close f1 too.

Note: Edited for making more clear the solution, look at how is changed the replace line.

edited Nov 14 '16 at 14:14

answered Nov 11 '16 at 20:47

maki

431
3
8

Please check the attached screenshot in the question. The replacement doesn't work for that character and for characters like em dash and en dash. It works fluently for the normal replacements. – user1469502 Nov 14 '16 at 07:28
My solution was valid, I changed the replace line to help you see better the solution. Do a copy/paste and check it. – maki Nov 14 '16 at 14:15
Thanks, Maki. I tried this code but still it was showing errors. I then changed the program file format to UTF-8 (maybe it had Windows default encoding) and then put the actual character. It worked fine after that. – user1469502 Nov 17 '16 at 13:10

Python program to change unicode characters to entities

1 Answers1