2

I try to parse data from a web API with jsonlite but for some reason the object it returns is a list.

It is said in the jsonlite package documentation that simplification process will automatically convert JSON list into a more specific R class but in my case it doesn't work.

It is like simplifyVector, simplifyDataFrame and simplifyMatrix function are disabled but each one is enabled by default.

What I would like is a dataframe to retrieve the $Name data (EAC, EFL, ELC, etc.).

I also try with the rjson library but still the same problem.

Any idea what could be wrong ?

Thank you,

Please find the code I use :

raw <- getURL("https://www.cryptocompare.com/api/data/coinlist")
library(jsonlite)
data <- fromJSON(txt=raw)

> class(data)
[1] "list"

> typeof(data)
[1] "list"

> str(data)
[...]
  ..$ EAC   :List of 13
  .. ..$ Id                 : chr "4437"
  .. ..$ Url                : chr "/coins/eac/overview"
  .. ..$ ImageUrl           : chr "/media/19690/eac.png"
  .. ..$ Name               : chr "EAC"
  .. ..$ CoinName           : chr "EarthCoin"
  .. ..$ FullName           : chr "EarthCoin (EAC)"
  .. ..$ Algorithm          : chr "Scrypt"
  .. ..$ ProofType          : chr "PoW"
  .. ..$ FullyPremined      : chr "0"
  .. ..$ TotalCoinSupply    : chr "13500000000"
  .. ..$ PreMinedValue      : chr "N/A"
  .. ..$ TotalCoinsFreeFloat: chr "N/A"
  .. ..$ SortOrder          : chr "100"
  ..$ EFL   :List of 13
  .. ..$ Id                 : chr "4438"
  .. ..$ Url                : chr "/coins/efl/overview"
  .. ..$ ImageUrl           : chr "/media/19692/efl.png"
  .. ..$ Name               : chr "EFL"
  .. ..$ CoinName           : chr "E-Gulden"
  .. ..$ FullName           : chr "E-Gulden (EFL)"
  .. ..$ Algorithm          : chr "Scrypt"
  .. ..$ ProofType          : chr "PoW"
  .. ..$ FullyPremined      : chr "0"
  .. ..$ TotalCoinSupply    : chr "21000000 "
  .. ..$ PreMinedValue      : chr "N/A"
  .. ..$ TotalCoinsFreeFloat: chr "N/A"
  .. ..$ SortOrder          : chr "101"
  ..$ ELC   :List of 13
  .. ..$ Id                 : chr "4439"
  .. ..$ Url                : chr "/coins/elc/overview"
  .. ..$ ImageUrl           : chr "/media/19694/elc.png"
  .. ..$ Name               : chr "ELC"
  .. ..$ CoinName           : chr "Elacoin"
  .. ..$ FullName           : chr "Elacoin (ELC)"
  .. ..$ Algorithm          : chr "Scrypt"
  .. ..$ ProofType          : chr "PoW"
  .. ..$ FullyPremined      : chr "0"
  .. ..$ TotalCoinSupply    : chr "75000000"
  .. ..$ PreMinedValue      : chr "N/A"
  .. ..$ TotalCoinsFreeFloat: chr "N/A"
  .. ..$ SortOrder          : chr "102"
  .. [list output truncated]
 $ Type        : int 100
NULL
Florent
  • 1,791
  • 22
  • 40
  • I imagine that `fromJSON()` has decided that this list is already in its simplest form. You have a nested list there. You'll probably have to do the rest of the work yourself. – Rich Scriven Mar 19 '16 at 18:16

2 Answers2

3

You showed the lower end of the structure, but the answer to the question regarding why a dataframe was not returned is seen at the top of the structure:

 # note: needed `require(RCurl)` to obtain getURL
> str(data)
List of 6
 $ Response    : chr "Success"
 $ Message     : chr "Coin list succesfully returned!"
 $ BaseImageUrl: chr "https://www.cryptocompare.com"
 $ BaseLinkUrl : chr "https://www.cryptocompare.com"
 $ Data        :List of 492
  ..$ BTC   :List of 13
  .. ..$ Id                 : chr "1182"
  .. ..$ Url                : chr "/coins/btc/overview"
  .. ..$ ImageUrl           : chr "/media/19633/btc.png"
  .. ..$ Name               : chr "BTC"
  .. ..$ CoinName           : chr "Bitcoin"
  .. ..$ FullName           : chr "Bitcoin (BTC)"
  .. ..$ Algorithm          : chr "SHA256"
 # ------snipped the many, many pages of output that followed---------

Furthermore the $Data node of that list has irregular lengths so coercing to a dataframe in one step might be difficult:

>  table( sapply(data$Data, length))

 12  13  14 
  2 478  12

After loading pkg:plyr which provides a useful function to rbind similar but not identical dataframes I'm able to contruct a useful starting point for furhter analysis:

require(plyr)
money <-   do.call(rbind.fill, lapply( data$Data, data.frame, stringsAsFactors=FALSE))
str(money)
#------------    
'data.frame':   492 obs. of  14 variables:

 $ Id                 : chr  "1182" "3808" "3807" "5038" ...
 $ Url                : chr  "/coins/btc/overview" "/coins/ltc/overview" "/coins/dash/overview" "/coins/xmr/overview" ...
 $ ImageUrl           : chr  "/media/19633/btc.png" "/media/19782/ltc.png" "/media/20626/dash.png" "/media/19969/xmr.png" ...
 $ Name               : chr  "BTC" "LTC" "DASH" "XMR" ...
 $ CoinName           : chr  "Bitcoin" "Litecoin" "DigitalCash" "Monero" ...
 $ FullName           : chr  "Bitcoin (BTC)" "Litecoin (LTC)" "DigitalCash (DASH)" "Monero (XMR)" ...
 $ Algorithm          : chr  "SHA256" "Scrypt" "X11" "CryptoNight" ...
 $ ProofType          : chr  "PoW" "PoW" "PoW/PoS" "PoW" ...
 $ FullyPremined      : chr  "0" "0" "0" "0" ...
 $ TotalCoinSupply    : chr  "21000000" "84000000" "22000000" "0" ...
 $ PreMinedValue      : chr  "N/A" "N/A" "N/A" "N/A" ...
 $ TotalCoinsFreeFloat: chr  "N/A" "N/A" "N/A" "N/A" ...
 $ SortOrder          : chr  "1" "3" "4" "5" ...
 $ TotalCoinsMined    : chr  NA NA NA NA ...

If you wanted to be able to access the rows by way of the abbreviations for those crypto-currencies, you could do:

rownames(money) <- names(data$Data)

Which now lets you do this:

> money[ "BTC", ]
      Id                 Url             ImageUrl Name CoinName
BTC 1182 /coins/btc/overview /media/19633/btc.png  BTC  Bitcoin
         FullName Algorithm ProofType FullyPremined TotalCoinSupply
BTC Bitcoin (BTC)    SHA256       PoW             0        21000000
    PreMinedValue TotalCoinsFreeFloat SortOrder TotalCoinsMined
BTC           N/A                 N/A         1            <NA>

Where before access would have been a bit more clunky:

> money[ money$Name=="BTC", ]
IRTFM
  • 258,963
  • 21
  • 364
  • 487
2

I reply to my own question as - already said in the comment section - returned object is already in it's simplest form. Probably that jsonlite cannot create data frame from multiple lists (lists imbrication).

The solution I have found is to use unlist and data.frame like this :

> df <- data.frame(unlist(data))
> class(df)
[1] "data.frame"
Florent
  • 1,791
  • 22
  • 40
  • 1
    I don't think that is a very useful result: `> str(df) 'data.frame': 6411 obs. of 1 variable: $ unlist.data.: Factor w/ 3674 levels " 27625814"," ClubCoin",..: 3436 2443 2833 2833 1005 57 518 2350 2266 2267 ...` – IRTFM Mar 19 '16 at 18:46