5

Need help writing a script downloads data from google insight using c#

this is the download url and requires a login

http://www.google.com/insights/search/overviewReport?q=test&cmpt=q&content=1&export=2

how do i input my username and password? need some help im new to c#

newbie
  • 67
  • 1
  • 4

2 Answers2

7

To make this work you need to first authenticate in order to obtain a valid SID for a given google site which can be used to access data. Here's how you could achieve this:

class Program
{
    static void Main(string[] args)
    {
        using (var client = new WebClient())
        {
            // TODO: put your real email and password in the request string
            var response = client.DownloadString("https://www.google.com/accounts/ClientLogin?accountType=GOOGLE&Email=youraccount@gmail.com&Passwd=secret&service=trendspro&source=test-test-v1");
            // The SID is the first line in the response
            var sid = response.Split('\n')[0];
            client.Headers.Add("Cookie", sid);
            byte[] csv = client.DownloadData("http://www.google.com/insights/search/overviewReport?q=test&cmpt=q&content=1&export=2");

            // TODO: do something with the downloaded csv file:
            Console.WriteLine(Encoding.UTF8.GetString(csv));
            File.WriteAllBytes("report.csv", csv);
        }
    }
}
Darin Dimitrov
  • 1,023,142
  • 271
  • 3,287
  • 2,928
  • Perfect! But to get string use Encoding.Unicode.GetString(csv); instead of Encoding.UTF8.GetString(csv) – Dragouf Oct 21 '11 at 13:12
  • Hi, since today it seems like it is not working anymore. The Clientlogin method does not return enough information about the different cookies to create (in comparison to ServiceLogin) and we receive this message "You have reached your quota limit. Please try again later." – Dragouf Apr 10 '12 at 17:07
  • Fairly certain this does not work anymore as Google has deprecated it. – Jack Marchetti Mar 16 '14 at 22:05
1

Ok it changed since few days.

Now you have to pass auth and not SID.

So code is now :

class Program
{
    static void Main(string[] args)
    {
        using (var client = new WebClient())
        {
            // TODO: put your real email and password in the request string
            var response = client.DownloadString("https://www.google.com/accounts/ClientLogin?accountType=GOOGLE&Email=youraccount@gmail.com&Passwd=secret&service=trendspro&source=test-test-v1");
            // The Auth line
            var auth = response.Split('\n')[2];
            client.Headers.Add("Authorization", "GoogleLogin " + auth);
            byte[] csv = client.DownloadData("http://www.google.com/insights/search/overviewReport?q=test&cmpt=q&content=1&export=2");

            // TODO: do something with the downloaded csv file:
            Console.WriteLine(Encoding.UTF8.GetString(csv));
            File.WriteAllBytes("report.csv", csv);
        }
    }
}

And now it work again for me.

Dragouf
  • 4,676
  • 4
  • 48
  • 55
  • Do you know how to download data from google webmaster tool? Like crawl issue for each site? – Mithil Feb 14 '13 at 19:18