Stack Overflow
Stack Overflow

Questions tagged [rcurl]

RCurl is an R package that provides an R-friendly wrapper for the libcurl C library.

RCurl is an R package that provides a wrapper for the libcurl C library (not the cURL command-line tool, ). RCurl greatly simplifies the process of interfacing R programs with web services and provides simple functions to perform common HTTP calls such as POST and GET. RCurl is cross-platform and installs necessary dependencies when it is added as an R package.

The core workhorse function in RCurl is curlPerform, which maps curl_easy_opt options almost verbatim. See listCurlOptions() for a list of arguments representing those options that can be passed to curlPerform (either directly or via other RCurl functions).

Users may also be interested in the package, which provides a further simplified wrapper around RCurl for common requests.

734 questions
6
votes
1 answer

Downloading a file after login using a https URL

I am trying to download an excel file, which I have the link to, but I am required to log in to the page before I can download the file. I have successfully passed the login page with rvest, rcurl and httr, but I am having an extremely difficult…
dmunslow
  • 149
  • 1
  • 7
6
votes
0 answers

R library(RCurl) behavior change on travis-ci

this line of code previously worked on travis-ci but now fails. works fine on appveyor RCurl::getURL( "ftp://ftp.cdc.gov/pub/data/yrbs/" , ftp.use.epsv = TRUE, dirlistonly = TRUE ) here's the text of the error- Error in function (type, msg,…
Anthony Damico
  • 5,779
  • 7
  • 46
  • 77
6
votes
1 answer

Cannot install 'RCurl' (Rcurl.h:4:23: fatal error: curl/curl.h: No such file or directory)

I've tried numerous methods to install RCurl, to no avail. I get this in my output: * installing *source* package 'RCurl' ... ** package 'RCurl' successfully unpacked and MD5 sums checked ** libs C:/Users/XXX/Documents/R/Rtools/mingw_64/bin/gcc …
Svencken
  • 479
  • 6
  • 14
6
votes
2 answers

Request URL failed/timeout in R

I'm trying to get a csv file from a url but it seems to be timing out after one minute. The csv file is being created at the time of the request so it needs a little more than a minute. I tried to increase the timeout but it didn't work, it still…
adjective_name
  • 453
  • 2
  • 8
  • 19
6
votes
1 answer

Send expression to website return dynamic result (picture)

I use http://www.regexper.com to view a picto representation regular expressions a lot. I would like a way to ideally: send a regular expression to the site open the site with that expression displayed For example let's use the regex:…
Tyler Rinker
  • 108,132
  • 65
  • 322
  • 519
6
votes
1 answer

Issues with RCurl crawler based on concurrent requests

The following is a script to reproduce the problems i'm facing when building a crawler with RCurl that performs concurrent requests. The objective is to download the content of several thousands of web sites in order to perform statistical…
marbel
  • 7,560
  • 6
  • 49
  • 68
6
votes
2 answers

httr GET function running out of space when downloading a large file

i'm trying to download a file that's 1.1 gigabytes with httr but i'm hitting the following error: x <- GET( extract.path ) Error in curlPerform(curl = handle$handle, .opts = curl_opts$values) : cannot allocate more space: 1728053248 bytes my C…
Anthony Damico
  • 5,779
  • 7
  • 46
  • 77
6
votes
2 answers

R, GET and GZ compression

I am building clients onto RESTful APIs. Some links let me download attachments (files) from the server, and in the best case these are .txt. I only mention the RESTful part since it means that I have to send some headers and potentially body with…
Alex Brown
  • 41,819
  • 10
  • 94
  • 108
6
votes
3 answers

Get response header

I would like to get response headers from GET or POST. My example is: library(httr) library(RCurl) url<-'http://www.omegahat.org/RCurl/philosophy.html' doc<-GET(url) names(doc) [1] "url" "handle" "status_code"…
Maciej
  • 3,255
  • 1
  • 28
  • 43
6
votes
1 answer

How to use R to download a zipped file from a SSL page that requires cookies

I am trying to download a file from an https page that requires an "I Agree" button be pushed and then stores a cookie. My apologies if this answer is obvious somewhere.. When I open up the web page directly in Chrome and click "I Agree" - the file…
Anthony Damico
  • 5,779
  • 7
  • 46
  • 77
6
votes
4 answers

How to Convert "space" into "%20" with R

Referring the title, I'm figuring how to convert space between words to be %20 . For example, > y <- "I Love You" How to make y = I%20Love%20You > y [1] "I%20Love%20You" Thanks a lot.
Kai Feng Chew
  • 779
  • 8
  • 24
5
votes
1 answer

How to do a SOAP request for EUR-Lex API with R?

How would you do a SOAP request for EUR-Lex's API using R? EUR-Lex is an EU database containing many legal acts. In the manual for their web services, they describe their SOAP system but not how to use R for it. I've tried for a while now to employ…
Will M
  • 692
  • 9
  • 20
5
votes
1 answer

Use R to fill html form and download the resulting file

I spent the day scouring the internet for examples of how to do this, however I'm still spinning in circles and could use a little direction. I am very new to html, have basic R coding experience, and minimal experience with any other coding…
shindig
  • 325
  • 2
  • 8
5
votes
0 answers

R: Connect to SFTP with RCurl

I'm trying to connect to a SFTP server. It is an encrypted server that uses FIPS mode. I am able to connect and perform file transfer through WinSCP, FileZilla, and through bash sftp commands with no problems. But, I cannot, access this same SFTP…
Dania
  • 305
  • 2
  • 10
5
votes
4 answers

installation of package ‘RCurl’ had non-zero exit status

I am trying to install RCurl on OS X 10.12.6 but get the following error: Error: package or namespace load failed for ‘RCurl’ in dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object…
Nick Malleson
  • 85
  • 1
  • 4
1 2 3
…
48 49