Stack Overflow
Stack Overflow

Questions tagged [rcurl]

RCurl is an R package that provides an R-friendly wrapper for the libcurl C library.

RCurl is an R package that provides a wrapper for the libcurl C library (not the cURL command-line tool, ). RCurl greatly simplifies the process of interfacing R programs with web services and provides simple functions to perform common HTTP calls such as POST and GET. RCurl is cross-platform and installs necessary dependencies when it is added as an R package.

The core workhorse function in RCurl is curlPerform, which maps curl_easy_opt options almost verbatim. See listCurlOptions() for a list of arguments representing those options that can be passed to curlPerform (either directly or via other RCurl functions).

Users may also be interested in the package, which provides a further simplified wrapper around RCurl for common requests.

734 questions
8
votes
2 answers

Create a C-level file handle in RCurl for writing downloaded files

In RCurl a function and a class CFILE is defined to work with C-level file handles. From the manual: The intent is to be able to pass these to libcurl as options so that it can read or write from or to the file. We can also do this with R…
antonio
  • 10,629
  • 13
  • 68
  • 136
8
votes
4 answers

"Non-zero exit status" error downloading XML and RCurl R packages

I'm having this problem: > install.packages("RCurl") Installing package(s) into ‘/home/username/R/x86_64-redhat-linux-gnu-library/2.13’ (as ‘lib’ is unspecified) trying URL 'http://lib.stat.cmu.edu/R/CRAN/src/contrib/RCurl_1.95-3.tar.gz' Content…
soosus
  • 1,211
  • 4
  • 18
  • 27
8
votes
1 answer

Using RCurl with SFTP

I'm attempting to use the ftpUpload in the RCurl package for the first time. The site I'm trying to access uses the sftp protocol. I've made sure to install the version of libcurl that includes the ability to make secure connections. SFTP is listed…
Statwonk
  • 713
  • 1
  • 8
  • 21
7
votes
1 answer

How to update RCurl and curl to the version on the host machine?

Initially I was trying to use devtools::install_github("EdwinTh/padr") where I got the following error: Error in curl::new_handle() : An unknown option was passed in to libcurl After some time I found out that devtools uses curl/RCurl which wraps…
Mamba
  • 1,183
  • 2
  • 13
  • 33
7
votes
2 answers

Login to .NET site using R

I am trying to login with my credentials to a .NET site but unable to get it working. My code is inspired from the below thread How to login and then download a file from aspx web pages with R library(RCurl) curl =…
Sushanta Deb
  • 529
  • 8
  • 20
7
votes
1 answer

R - Error when using getURL from curl after site was changed

I have been using getURL from curl (in R) to read from https://fantasy.premierleague.com/drf/bootstrap-static Example code: print(getURL("https://fantasy.premierleague.com/drf/bootstrap-static")) No problem until a few days ago. But now getting the…
mikecro
  • 101
  • 1
  • 1
  • 6
7
votes
1 answer

R Change IP Address programmatically

Currently changing user_agent by passing different strings to the html_session() method. Is there also a way to change your IP address on a timer when scraping a website?
tonyk
  • 348
  • 5
  • 22
7
votes
3 answers

Using an API to calculate distance between two airports (two columns) within R?

I was wondering whether there was a way to compare airport distances(IATA codes). There are some scripts but not is using R. So I tried that with with the API: developer.aero Example data: library(curl) # for curl post departure <-…
Michaer Macher
  • 81
  • 1
  • 4
7
votes
1 answer

Asynchronous POST Requests - R, using RCurl?

I am trying to make async requests to a REST API from R. The below curl command illustrates the parameters that I need to the pass to the api. I'm giving you guys the linux curl command as I'm hoping that will make it clear: curl -v -X POST…
hamel
  • 1,288
  • 1
  • 12
  • 16
7
votes
2 answers

Download a file keeping original filename when final link is hidden

I need to download a file, save it in a folder while keeping the original filename from the website. url <- "http://www.seg-social.es/prdi00/idcplg?IdcService=GET_FILE&dID=187112&dDocName=197533&allowInterrupt=1" From a web browser, if you click on…
Carlos Tafur Porras
  • 73
  • 4
7
votes
2 answers

RCurl and self-signed certificate issues

I am having problems getting RCurl function getURL to access an HTTPS URL on a server that is using a self-signed certificate. I'm running R 3.0.2 on Mac OS X 10.9.2. I have read the FAQ and the curl page on the subject. So this is where I stand: I…
asieira
  • 3,513
  • 3
  • 23
  • 23
7
votes
1 answer

How to capture RCurl verbose output

I have the following request library(RCurl) res=getURL("http://www.google.com/search?hl=en&lr=&ie=ISO-8859-1&q=RCurl&btnG=Search", .opts=list(verbose = TRUE) ) and would like to capture the verbose output of the call (i.e.,…
cryo111
  • 4,444
  • 1
  • 15
  • 37
7
votes
2 answers

Download .RData and .csv files from FTP using RCurl (or any other method)

I've uploaded a .RData file (created using save()) to an ftp server, and I'm trying to use getURL() to download that same file. For all the examples and posts I've read, I can't seem to get this to work. The .RData file was saved…
rbatt
  • 4,677
  • 4
  • 23
  • 41
7
votes
1 answer

log into a website to grab the data using RCurl

I wanted to login to the website using RCurl and grab the data from the web (The data cannot be seen without logging in.) I wanted to export this (for example)…
user1486507
  • 699
  • 2
  • 13
  • 24
6
votes
4 answers

Check if URL exists in R

I want to loop over a list of URLs and I want to find out if these URLs exist or not. RCurl provides the url.exists() function. However, the output doesn't seem to be right, because for example it says that amazon.com is not registered (it does so…
J. Doe
  • 169
  • 2
  • 6
1 2 3
…
48 49