3

I am going through a similar issue as this question here:

I have a big JSON file on AWS S3 and am trying to access it via RStudio (EC2 instance from Louis Aslett's AMI). I have even tried moving from t2 to r4.xlarge with 30GB of memory but to no avail: I receive errors:

Error in writeBin(httr::content(r, as = "raw"), con = file) : long vectors not supported yet: connections.c:4147

If I use the free tier instance then it gives me the error:

Error in curl::curl_fetch_memory(url, handle = handle) : Failed writing body (0 != 16360)

It seems that the question that I mentioned in the reference has figured out a way to do it, though I am not able to follow it completely. Can someone please explain a little when they say that the directory needs to be something else than "home". How do you implement it? Because there is no permission to do that in Louis AMI. The question may be very basic but I am getting out of my wits here.

Cheers! A

Ashish Dang
  • 41
  • 1
  • 5

1 Answers1

0

"Can someone please explain a little when they say that the directory needs to be something else than "home". How do you implement it? Because there is no permission to do that in Louis AMI. The question may be very basic but I am getting out of my wits here."

I am sympathetic to you here as it is counterintuitive to a new linux user coming from windows IMHO, ironically I have seen two questions answering this closed as they are considered too basic for this advanced forum. But you are not alone, it sounds like the same problem from personal experience of the same error message with reading in data with the same AMI.

If you upload into a different drive on the instance then this can most likely be solved. As the Louis Aslett Rstudio AMI is based in this 8-10GB space then you will have to set your working directory outside this, the home directory. Not intuitively apparent from Rstudio server interface.

I would suggest to have a look at other directories (e.g. going up a few levels above home in the Rstudio directory selection box on the RHS in Rstudio or df command on the linux commandline). Then setwd() in another directory (e.g. xda or whatever has enough room) and try to read in again.

Joey
  • 137
  • 2
  • 13