1

I am trying to save some county climate data onto a .csv and have saved some responses that I accessed, using the GET() function for API calls from the httr package. However, when I try to convert the content of the response into a dataframe using fromJSON(), I am constantly hit by a lexical error that there is an "invalid char in json text."

I have been looking through many similar issues on Stack Overflow and thought it might be somehow related to the content type being HTML. The validate() function from jsonlite also verifies that the output of rawToChar() on the content of my response is not in the JSON format. However I am not familiar enough with the httr or jsonlite packages to understand the exact issue I am encountering or how to resolve it.

Is there an issue with the call, the response, or something else? Any suggestions would be helpful.

Here is the relevant excerpt of the exact code I have been using (modified for a single week instead of a year):

test_county <- GET("https://nass.agroclimate.org/ws/data/daily.php?county=USAL01001&sdate=20220101&edate=20220108&csv")
df <- fromJSON(rawToChar(test_county$content), flatten = TRUE)

I was expecting this code to output a dataframe from the content of the response to my API call, but instead I received the subsequent error message:

Error: lexical error: invalid char in json text.
                                       date,nass_week,nass_week_year,c
                     (right here) ------^
Alex T.
  • 11
  • 1
  • 3
    The "&csv" in the URL tells you that the content returned is in CSV format, not JSON. So there's no need for GET or fromJSON, you can just read the CSV directly from the URL. – neilfws Jan 12 '23 at 00:15
  • @neilfws wow, I can NOT believe it was that simple. An oversight on my part. Thanks. – Alex T. Jan 12 '23 at 00:29

1 Answers1

3

We could directly read with read.csv

url <- "https://nass.agroclimate.org/ws/data/daily.php?county=USAL01001&sdate=20220101&edate=20220108&csv"
df <- read.csv(url)

-output

> str(df)
'data.frame':   8 obs. of  14 variables:
 $ date          : chr  "2022-01-01" "2022-01-02" "2022-01-03" "2022-01-04" ...
 $ nass_week     : int  52 52 1 1 1 1 1 1
 $ nass_week_year: int  2021 2021 2022 2022 2022 2022 2022 2022
 $ county        : chr  "USAL01001" "USAL01001" "USAL01001" "USAL01001" ...
 $ district      : int  140 140 140 140 140 140 140 140
 $ mint          : num  60.3 68.3 33.1 27.4 27.6 36.8 31.4 29.6
 $ avgt          : num  70.3 73.6 48.1 33.4 40.5 48.7 50.2 38.3
 $ maxt          : num  79.2 79.4 63 39.4 53.3 60.6 69.1 47
 $ ampt          : num  18.9 11 29.8 11.9 25.6 23.7 37.6 17.3
 $ totr          : num  0.003 0.028 0.905 0 0 0 0.183 0
 $ arid          : num  0 0 0 0 0 0 0 0
 $ gdd_b60       : num  9.8 13.9 0 0 0 0 0 0
 $ gdd_b50       : num  19.8 23.9 0 0 0 0 0.5 0
 $ gdd_b40       : num  29.8 33.9 8.1 0 0.5 8.7 10.2 0
akrun
  • 874,273
  • 37
  • 540
  • 662
  • Perfect - this is exactly the kind of solution I was looking for. Thank you! Oddly enough though this solution works on my personal computer but not on my work computer. – Alex T. Jan 12 '23 at 00:30
  • @AlexT. Can you check the R version etc in your work computer vs yours – akrun Jan 12 '23 at 00:51
  • Work is on 4.2.1 while personal is 4.2.2. Unfortunately not allowed to use 4.2.2 on work yet, but I wonder if that is part of the problem. – Alex T. Jan 12 '23 at 00:55
  • @AlexT. Can you try `read.csv(url(url))` here, the `url` wrapped is the function and not the object name – akrun Jan 12 '23 at 00:57