Encoding of Umlauts while importing into R tables

Question

I need to join two tables on the colum name. They come from different excel spreadsheets which I didn't create. I imported both sheets into R tables using the function read.xls from the gdata package.

Sometimes the names contains umlauts and other accents. They appear identical within the excel sheets but when I import them into R they are not the same. Hence my join doesn't join. I join using the sqldf function.

As an example : I see Lück in the two spreadsheets. In table1 of R this appears as L\374ck whereas in table2 it appears as L\303\274ck.

How can I best solve this problem? Is there a way to force an encoding when data is imported? Or should one try to force the comparison of strings in a different way?

What OS are you using? Do you know what OS the data were produced under? (OS = Operating System) — Serban Tanasa, Dec 28 '14 at 14:38

score 3 · Accepted Answer · answered Dec 28 '14 at 14:50

3

Since you stated you are not geting a TRUE on agstudy's code, I'm assuming you're using a Mac or a Linux Machine.

Try instead

x1=  "L\374ck"
x2 = "L\303\274ck"
iconv(x1,"latin1","UTF-8")
[1] "Lück"
identical(iconv(x1,"latin1","UTF-8"),x2)
[1] TRUE

Tested this on a Linux CentOS 7. I believe it should work on Macs too. If not look into

?iconv

for more details on encoding.

answered Dec 28 '14 at 14:50

Serban Tanasa

3,592
2
23
45

Your assumption is correct. I didn't realise that information was relevant. Well deduced! – Geoff Dec 28 '14 at 18:38

score 1 · Answer 2 · answered Dec 28 '14 at 14:03

1

It is an encoding problem. For example :

 x1=  "L\374ck"
 x2 = "L\303\274ck"
 identical(iconv(x2,"UTF-8","UTF-8"),x1)
 [1] "TRUE"

So you can try this for example before applying merge :

 df2name <- iconv(df2$name,'UTF-8','UTF-8')

answered Dec 28 '14 at 14:03

agstudy

119,832
17
199
261

When I run your example I get `[1] FALSE` – Geoff Dec 28 '14 at 14:20
@Geoff Seban answer should help. – agstudy Dec 28 '14 at 16:58

Encoding of Umlauts while importing into R tables

2 Answers2