7

I wanted to login to the website using RCurl and grab the data from the web (The data cannot be seen without logging in.)

I wanted to export this (for example) "http://www.appannie.com/app/ios/instagram/ranking/history/chart_data/?s=2010-10-06&e=2012-06-04&c=143441&f=ranks&d=iphone" into R after I log in using RCurl. The issue is I cannot log in using RCurl. I haven't tried this before so mostly I referred to http://www.omegahat.org/RCurl/philosophy.html.

So here's what I tried. (here, 'me@gmail.com' is my user ID and '9999' is my Password - i just made it up.)

library(RJSONIO)
library(rjson)
library(RCurl)
appannie <- getURL("http://www.appannie.com/app/ios/instagram/ranking/history/chart_data/.json?s=2010-10-06&e=2012-06-04&c=143441&f=ranks&d=iphone, userpwd = me@gmail.com:9999", verbose = TRUE)

But this gave me the message below : 

About to connect() to www.appannie.com port 80 (#0)
*   Trying 69.167.138.64... * connected
* Connected to www.appannie.com (69.167.138.64) port 80 (#0)
> GET /app/ios/instagram/ranking/history/chart_data/?s=2010-10-06&e=2012-06-04&c=143441&f=ranks&d=iphone HTTP/1.1
Host: www.appannie.com
Accept: */*

< HTTP/1.1 403 FORBIDDEN
< Server: nginx/1.1.19
< Date: Fri, 01 Mar 2013 23:41:32 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Keep-Alive: timeout=10
< Vary: Accept-Encoding
< Vary: Cookie,Accept-Encoding
< 
* Connection #0 to host www.appannie.com left intact

So, I went back and read this http://www.omegahat.org/RCurl/philosophy.html again and didn't know what to do, so I tried this after I saw the similar question from stackoverflow.

getURL("http://www.appannie.com/app/ios/instagram/ranking/history/chart_data/?s=2010-10-06&e=2012-06-04&c=143441&f=ranks&d=iphone",.opts=list(userpwd=me@gmail.com:9999"))

But this gives me below output.

[1] ""

Can anyone give me a hint? (After a bunch of different trial, the website starts to send me warnings =(

user1486507
  • 699
  • 2
  • 13
  • 24
  • does this help `appannie <- getURL("http://www.appannie.com/app/ios/instagram/ranking/history/chart_data/.json?s=2010-10-06&e=2012-06-04&c=143441&f=ranks&d=iphone", userpwd = "me@gmail.com:9999", verbose = TRUE)` ? looked in ` ?getURL` – Eric Fail Mar 02 '13 at 00:50
  • 1
    Did you read their [blog post](http://blog.appannie.com/hourly-rankings-parent-publishers-data-export-analytics-api/) about the api allowing you to access data programmatically as being in private beta? Perhaps you will have more luck by signing up at support@appannie.com. Apparently, you can also download .csv data for your own apps. – Simon O'Hanlon Mar 02 '13 at 14:04
  • @Eric, thanks for the reply. btw, what is the difference between the first code that I used and the code that you put? (somehow it looks same to me..) – user1486507 Mar 04 '13 at 18:37
  • @Simon, thanks for letting me know their private beta program, which I didn't know before =) – user1486507 Mar 04 '13 at 18:38
  • @user1486507, I close the string (`"`) after the URL string and start a new string for `"me@gmail.com:9999"`. I'm not saying this will solve it, but it is really important that you read your code very carefully. – Eric Fail Mar 04 '13 at 18:55
  • Is there a redirect? What HTP response code do you get from server? – Mirko Ebert Jun 11 '13 at 09:55

1 Answers1

0

This is some sort of authentication issue not anything you did wrong with RCurl most likely.

You got through to the server but either your login was incorrect, it wasn't valid or the data is not available via the API.

http://en.wikipedia.org/wiki/HTTP_403

Dirk Calloway
  • 2,569
  • 4
  • 23
  • 34