4

a little bit about the background. I pull data from an API that supplies public transportation data. It returns the result in json format, which I process with the library 'jsonlite'.

 resp <- GET(url = url)


  resp_char <- rawToChar(resp$content)
  parsed <- fromJSON(resp_char, flatten = T)

  parsed.df <- do.call(what = "rbind", args = lapply(parsed[1], as.data.frame))

The problem is, in the result there are no special characters.

I am working on a Windows Server 2012 machine and my language settings in R look like this:

    > Sys.getlocale()
[1] "LC_COLLATE=German_Germany.1252;LC_CTYPE=German_Germany.1252;LC_MONETARY=German_Germany.1252;LC_NUMERIC=C;LC_TIME=German_Germany.1252"

Example:

    > df$direction
"U Alt-Mariendorf (Berlin)" 
"U Alt-Tegel (Berlin)" 
"U Alt-Mariendorf (Berlin)"              
"U Alt-Tegel (Berlin)" 
"Märkisches Viertel, Wilhelmsruher Damm"

The expected result for the fifth result is "Märkisches Viertel, Wilhelmsruher Damm"

After that I looked in the actual encoding.

> Encoding(df$direction)
   [1] "unknown" "unknown" "unknown" "unknown" "UTF-8"

In my opinion this looks good so far, but nevertheless I cannot see special characters.

I appreciate any suggestions and ideas on the subject.

Regards

ahLoco
  • 111
  • 6
  • have you tried this function ```iconv```? – amrrs Sep 08 '17 at 10:33
  • Yes I tried. That is the result > iconv(tmpStore2$direction,to ="UTF8") [1] "U Alt-Mariendorf (Berlin)" "U Alt-Tegel (Berlin)" "U Alt-Mariendorf (Berlin)" [4] "U Alt-Tegel (Berlin)" "Märkisches Viertel, Wilhelmsruher Damm" – ahLoco Sep 08 '17 at 10:52

2 Answers2

7

So finally I got it. Thanks to @parth, it has led me to the right answer. I used Encoding before my fromJSON statement and that worked for me.

  resp <- GET(url = url)

  resp_char <- rawToChar(resp$content)
  Encoding(resp_char) <- "UTF-8"
  parsed <- fromJSON(resp_char, flatten = T)

  parsed.df <- do.call(what = "rbind", args = lapply(parsed[1], as.data.frame))
ahLoco
  • 111
  • 6
0

Using the dataframe

df<-data.frame(direction=c("U Alt-Mariendorf (Berlin)","U Alt-Tegel (Berlin)","U Alt-Mariendorf (Berlin)","U Alt-Tegel (Berlin)","Märkisches Viertel, Wilhelmsruher Damm"), stringsAsFactors = FALSE)

> df
                                direction
1               U Alt-Mariendorf (Berlin)
2                    U Alt-Tegel (Berlin)
3               U Alt-Mariendorf (Berlin)
4                    U Alt-Tegel (Berlin)
5 Märkisches Viertel, Wilhelmsruher Damm

Now, just change the encoding of entire df$direction column as

Encoding(df$direction) <- "UTF-8"

which fixes the issue

> df
                               direction
1              U Alt-Mariendorf (Berlin)
2                   U Alt-Tegel (Berlin)
3              U Alt-Mariendorf (Berlin)
4                   U Alt-Tegel (Berlin)
5 Märkisches Viertel, Wilhelmsruher Damm
parth
  • 1,571
  • 15
  • 24
  • Unfortunately it is just working in your example. The difference ist, that my Dataframe already is UTF-8 encoded (see above). Your code is in Latin1 and then it works.. (converting my dataframe in latin1 and back to utf-8 does not work) – ahLoco Sep 08 '17 at 11:03