2

I'm fixing a module checking Google Keyword Ranking, each keywords for each 30s. Those day, Google block the request, the module is very important for our project.

I'm using HtmlAgilityPack in my module, the code is something like that:

HttpWebRequest req = WebRequest.Create(uri) as HttpWebRequest;                
req.Method = "GET";            
req.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6";

// read last cookie from disk, send it with request
CookieContainer cookies = ReadCookiesFromDisk(cookieFile); 
req.CookieContainer = cookies;

// request google, with url like: https://www.google.com/search?start=0&num=20&q=may+do+huyet+ap&hl=vi
resp = req.GetResponse() as HttpWebResponse;

// save new cookie to disk
CookieContainer newCookies = req.CookieContainer;
WriteCookiesToDisk(cookieFile, newCookies); // write cookie to disk using Serialize Object to BinaryFormatter

And as below is 2 function read/write cookie to binary file.

public void WriteCookiesToDisk(string file, CookieContainer cookieJar)
{
    using (Stream stream = File.Create(file))
    {               
        BinaryFormatter formatter = new BinaryFormatter();
        formatter.Serialize(stream, cookieJar);
    }
}

public CookieContainer ReadCookiesFromDisk(string file)
{
    try
    {
        using (Stream stream = File.Open(file, FileMode.Open))
        {
            BinaryFormatter formatter = new BinaryFormatter();
            return (CookieContainer)formatter.Deserialize(stream);
        }
    }
    catch (Exception e)
    {
        return new CookieContainer();
    }
}

After 1 day, Google block the request by captcha, and I found that:

  • If I change my IP, I can continue request. So, Google block by IP.

  • Using Opera with Enable cookie, Opera still search Google even when Google block my module with the same keywords, but if I disable cookie of Opera, Opera can not search Google like my module.

So, I'am trying to enable cookie like Opera (as my code on above), but Opera work, but my code does not work.

Could you please help me?

Minh Tuan Nguyen
  • 21
  • 2
  • Every 30s? that sounds a tad frequent. Im not surprised you got blocked. why so often? Im not sure cookies will save you – BugFinder Aug 13 '15 at 06:47
  • Generally SO can't help you with bypassing service provider restrictions because they are not known publicly and will change quickly if there is know hack around thus rendering answer obsolete almost immediately. If you are asking about general issues with your code you need to post complete sample along with errors you see. So far it is unclear how SO can help you. – Alexei Levenkov Aug 13 '15 at 06:49
  • Thanks, I add detail of 2 functions read/write cookies, I tested them, seem that they work. I'm trying to request Google like Opera does. At same time, same Internet connection, Opera Enable Cookie can google, but my module which send cookie with request can not do it. And the module is important for our Search Engine Optimization Process. – Minh Tuan Nguyen Aug 13 '15 at 07:24
  • Cookies are basically just a domain, a path, a name and a value, btw. All stuff you can store as plain text quite easily. – Nyerguds Apr 11 '16 at 16:01

0 Answers0