0

I'm trying to log into bondora.com with using R httr POST request, because this site does not appear to be using authentication:

library(httr)
login <- "https://www.bondora.com/en/login"
pars <- list(
    username = "MyUserName",
    password = "MyPassword"
    )
POST(login, body = pars)

After logging in, the website directs the user back to landing page bondora.com/en/home, but instead if I parse the the POST request I get the same title for the page as the one from the login page:

library(XML)
test <- POST(login, body = pars)
test <- content(test, as = "text")
parsedHtml <- htmlParse(test, asText = TRUE)
xpathSApply(parsedHtml, "//title", xmlValue)
[1] "Join or log in|Loans and investing|Bondora"

I tried using the same techinque on some other sites and it appeared to be working quite fine, except for this site. The output from the POST command is as follows:

POST(login, body = pars)
   Response [https://www.bondora.com/en/login]
      Status: 200
      Content-type: text/html; charset=utf-8
   <!DOCTYPE HTML>
   <html xmlns="http://www.w3.org/1999/xhtml">

...

Are there some specific settings I should use for logging in on to bondora.com/en/login?

UPDATE 1 As per @hadley comment, I tired setting multipart TRUE and FALSE, but no help. Then I inspected the request via browser and added the same header:

login <- "https://www.bondora.com/en/authenticate"
pars <- list(
  username = "username",
  password = "password"
  )
headers <- list(
  "User-Agent" = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:28.0) Gecko/20100101 Firefox/28.0",
  "Referer" = "https://www.bondora.com/en/login?returnurl=/en/home",
  "Host" = "www.bondora.com",
  "Connection" = "keep-alive",
  "Accept-Language" = "en-US,en;q=0.5",
  "Accept-Encoding" = "gzip, deflate",
  "Accept" = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
  )
POST(login, body = pars, add_headers(.headers = character(headers)))
Error in character(headers) : invalid 'length' argument

It seems that I would need to specify the length argument as did the HTML Error 411 indicate. How should I do that? I also tried adding Content-Length = 9844 to the Request Header as it was in the Response Header, but no success either.

maivel
  • 35
  • 2
  • 8
  • Curious why you'd want to login from R? Perhaps your `pars` object needs to be as json first, so `POST(login, body = toJSON(pars))` – sckott Apr 21 '14 at 18:03
  • I want to log in to scrape some consumer credit data. I tried the toJSON, but still the same result. – maivel Apr 21 '14 at 19:10
  • 1
    If you look at the html, the form POSTs to "/en/authenticate", not "/en/login" – hadley Apr 21 '14 at 20:43
  • Thanks for the comment, @hadley! I tried with `POST("https://www.bondora.com/en/authenticate", body = pars)` and got Error 411 in return: `HTTP Error 411. The request must be chunked or have a content length.` What else might I be doing wrong? – maivel Apr 21 '14 at 23:33
  • You might need to experiment with multipart. It's also worth learning how to use a browser debugger so you can see exactly what your web browser is sending to the site. – hadley Apr 21 '14 at 23:44
  • @hadley, I added an update according to your comment to the original post – maivel Apr 22 '14 at 19:31
  • Replace `list()` with `add_header()` and then just do `POST(login, body = pars, headers()`. But I doubt those headers are the problem - it's more likely to be how the body is being sent. – hadley Apr 22 '14 at 19:57
  • @hadley, I tried with your suggestion and got back `HTTP Error 400. The request has an invalid header name.` Probably I'm doing headers wrong, but if you think this isn't the problem, would you have any suggestions on sending the `body`? Thanks – maivel Apr 22 '14 at 21:07
  • Try the dev version of httr - `verbose()` will now show you exactly what's getting sent in the body. – hadley Apr 23 '14 at 14:04
  • 3
    Did you ever get a solution to this? I'm having the same problem authenticating against IIS7 with httr POST – Colin Sep 08 '14 at 15:40

1 Answers1

2

I was able to solve this by upgrading httr_0.4 to httr_0.5

Colin
  • 930
  • 3
  • 19
  • 42