Open a text file with accents in python

Question

I try to open a text file in French with Python 2.7. I used the command

f=open('textfr','r')

but when I use

f.read()

I lose accented characters: I get u"J'\xc3\xa9tais \xc3\xa0 Paris instead of J'étais à Paris, etc..

when in linux terminal, I do

file -i textfr

I get

charset=utf-8

so I do not understand...

have you tried print f.read()? – Padraic Cunningham Dec 21 '14 at 18:34 — Padraic Cunningham, Dec 21 '14 at 18:34

score 11 · Answer 1 · answered Dec 21 '14 at 18:12

11

You need to specify the charset.

f = io.open('textfr', 'r', encoding='utf-8')

answered Dec 21 '14 at 18:12

Ignacio Vazquez-Abrams

776,304
153
1,341
1,358

Yes, it does. It must, by definition. – Ignacio Vazquez-Abrams Dec 21 '14 at 18:16
1

@Mustafa: When you say "I get `J'\xc3\xa9tais \xc3\xa0 Paris`" are you "get"-ting it by using the `print` statement to a terminal, or evaluating the contents of a variable in an interactive Python session? If you are using a terminal, how is it configured? (for *nix style hosts, what is your `TERM` env var set to) – Brian Cain Dec 21 '14 at 18:18
I get it when I enter f.read() in the spyder IDE, or in the python ide launched from the terminal of ubuntu 14.04 – Mostafa Dec 21 '14 at 18:32
What's the 'r'? – Denis G. Labrecque Jan 22 '21 at 22:59

Mazdak · Answer 2 · 2014-12-21T18:32:01.550

3

By default, files are read/written using the system default text encoding, as can be found in sys.getdefaultencoding() . On most machines, this is set to utf-8 . but some of machines like yours doesn't use utf-8 you can use a proper encoding for your file , or use utf-8 that is a universal encoding :

in python 3 :

with open('somefile.txt', 'rt', encoding='utf-8') as f:
         #do stuff

in python 2 you can use codecs.open():

import codecs
f=codecs.open ('somefile.txt', 'rt', encoding='utf-8').read()

edited Dec 21 '14 at 18:32

answered Dec 21 '14 at 18:18

Mazdak

105,000
18
159
188

1

`open()` in 2.x doesn't have an `encoding` argument. – Ignacio Vazquez-Abrams Dec 21 '14 at 18:20
@IgnacioVazquez-Abrams oh, yep ,i'll add this to answer – Mazdak Dec 21 '14 at 18:23

score 1 · Answer 3 · answered Dec 21 '14 at 18:15

1

use codecs instead of standard open so

import codecs
codecs.open('textfr','r', 'utf-8')

answered Dec 21 '14 at 18:15

sax

3,708
19
22

thanks, but it does not work...so maybe the problem is elsewhere – Mostafa Dec 21 '14 at 18:18

Open a text file with accents in python

3 Answers3

Linked