Stack Overflow
Stack Overflow

Questions tagged [httr]

The R package 'httr' provides useful tools for working with HTTP connections. It is a simplified wrapper built on top of 'RCurl'.

The R package httr provides useful tools for working with HTTP connections. It is a simplified wrapper built on top of RCurl. It is much much less configurable but because it only attempts to encompass the most common operations it is also much much simpler.

Resources:

1019 questions
9
votes
1 answer

Error in curl::curl_fetch_memory(url, handle = handle) : Failure when receiving data from the peer

I'm trying to get lyrics from the chartlyrics API. I write an R function that works but not inside a loop. My script is: library(httr) library(RCurl) library(XML) df <- data.frame(artist = c('Led Zeppellin', 'Adele'), song = c('Rock´n roll',…
pjperez
  • 425
  • 4
  • 9
9
votes
3 answers

SSL connect error in httr / curl

I'm trying to access an open API with httr, and having no luck. Whenever I try: httr::GET("https://api.openaq.org/v1/countries") I get the following error: Error in curl::curl_fetch_memory(url, handle = handle) : SSL connect error However,…
AndrewMacDonald
  • 2,870
  • 1
  • 18
  • 31
8
votes
1 answer

How to get session token when authenticating to JSON REST API (in R)

I'm trying to access JSON data (in R) from a REST API. To authenticate myself, I need to use a POST method in https://dashboard.server.eu/login. The data that needs to be sent are email and password: library(httr) login <- list( email =…
user3341592
  • 1,419
  • 1
  • 17
  • 36
8
votes
1 answer

How to submit a form that seems to be handled by JavaScript using httr or rvest?

I'm trying to programatically search a website, but the submit button functionality seems to be primarily powered by JavaScript. I'm not overly familiar with how this works though, so I could be wrong. Here is the code I'm…
tblznbits
  • 6,602
  • 6
  • 36
  • 66
8
votes
2 answers

How to specify certificate, key and root certificate with httr for certificate based authentication?

I am trying to access data using httr library from server which expects certificate based authentication. I have certificate (cert.pem), key file (key.pem) and root certificate (caroot.pem) Following curl works. curl -H "userName:sriharsha@rpc.com"…
harsha
  • 933
  • 1
  • 12
  • 26
8
votes
2 answers

Identify a weblink in bold in R

The following script allows me to get to a website with several links with similar names. I want to get only one of them, which can be diferentiated from the others because it is printed in bold in the website. However, i could not find a way of…
Agus camacho
  • 868
  • 2
  • 9
  • 24
8
votes
2 answers

Ignore SSL errors in R httr connection

Updated: I am trying to access a corporate internal web API using: require(httr) url = 'https://my_server/api/search/query?q=stuff' httr::set_config( config(ssl_verifypeer = 0L) ) data <- httr::GET( url, use_proxy(url = "ipaddress:port"), verbose()…
KillerSnail
  • 3,321
  • 11
  • 46
  • 64
8
votes
2 answers

Using R to accept cookies to download a PDF file

I'm getting stuck on cookies when trying to download a PDF. For example, if I have a DOI for a PDF document on the Archaeology Data Service, it will resolve to this landing page with an embedded link in it to this pdf but which really redirects to…
Ben
  • 41,615
  • 18
  • 132
  • 227
8
votes
1 answer

Rcurl: url.exists returns false when url does exist

Trying to download information from a specific web page, and although it opens fine in any browser, RCurl says it does not exists: url.exists("http://www.transfermarkt.es/liga-mx-apertura/startseite/wettbewerb/MEXA") [1] FALSE Same results when…
edufierro
  • 107
  • 1
  • 6
8
votes
1 answer

Speed up API calls in R

I am querying Freebase to get the genre information for some 10000 movies. After reading How to optimise scraping with getURL() in R, I tried to execute the requests in parallel. However, I failed - see below. Besides parallelization, I also read…
majom
  • 7,863
  • 7
  • 55
  • 88
7
votes
1 answer

Authenticate at Github via Travis-CI using httr as well as locally (local works, remote doesn't)

I have an Rmd file that uses httr to access the Github-API. Locally, I can authenticate with Github just fine if I run the following in the R console before rendering the Rmd: myapp <- oauth_app("APP", key = "xyz", secret = "pqr") github_token <-…
Bryan Hanson
  • 6,055
  • 4
  • 41
  • 78
7
votes
1 answer

R: reading geotiff data straight from web url (httr::GET raw content)

I would like to create a RasterLayer from GeoTIFF data provided by a server. I'll query the server for this data using a httr::GET call (the data is provided on-demand, so in the application there won't be a url ending in .tif but a query…
WillemMaetens
  • 73
  • 7
7
votes
1 answer

R httr post-authentication download works in interactive mode but fails in function

the code below works fine in interactive mode but fails when used in a function. it's pretty simply two authentications POST commands followed by the data download. my goal is to get this working inside a function, not just in interactive…
Anthony Damico
  • 5,779
  • 7
  • 46
  • 77
7
votes
2 answers

Login to .NET site using R

I am trying to login with my credentials to a .NET site but unable to get it working. My code is inspired from the below thread How to login and then download a file from aspx web pages with R library(RCurl) curl =…
Sushanta Deb
  • 529
  • 8
  • 20
7
votes
1 answer

R Change IP Address programmatically

Currently changing user_agent by passing different strings to the html_session() method. Is there also a way to change your IP address on a timer when scraping a website?
tonyk
  • 348
  • 5
  • 22
1 2
3
…
67 68