I need to download a CSV file from Google insights programatically. Since it requires authentication, I used the clientLogin to get the session id.
How do I download the file by passing the session id as a cookie?
I tried using a new URLConnection object and set the cookie in setRequestParameter method hoping it would authenticate my login then, however it doesn't seem to be working. I have a feeling I shouldn't use two separate connections, is that true?
If so then how do I pass session id as parameter when i download the file? I also tried using the same connection this didn't work either. Please help.
try {
URL url1 = new URL("https://www.google.com/accounts/ClientLogin?accountType=GOOGLE&Email=*******.com&Passwd=*****&service=trendspro&source=test-test-v1");
URL url2 = new URL("http://www.google.com/insights/search/overviewReport?cat=0-7&geo=BR&cmpt=geo&content=1&export=1");
URLConnection conn = url1.openConnection();
// fake request coming from browser
conn.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.56 Safari/535.11");
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream(), "UTF-8"));
String f = in.readLine();
// obtaining the sid.
String sid=f.substring(4);
System.out.println(sid);
URLConnection conn2 = url2.openConnection();
conn2.setRequestProperty("Cookie", sid);
BufferedInputStream i= new BufferedInputStream(conn2.getInputStream());
FileOutputStream fos = new FileOutputStream("f:/testplans.csv");
BufferedOutputStream bout = new BufferedOutputStream(fos,1024);
byte data[] = new byte[1024];
while(i.read(data,0,1024)>=0) {
bout.write(data);
}
bout.close();
in.close();
}