0

some of my website urls are duplicated. i need to know which of them are indexed by google i need some function in c# to know which of my url is indexed.

vakas
  • 1,799
  • 5
  • 23
  • 40

1 Answers1

3

In Google's search you can type: site:yourdomain

And it will show you the results. you can use the Google Custom Search API programmatically to do this. http://code.google.com/apis/customsearch/v1/overview.html

It returns JSON results that you can convert into C# objects using the DataContractSerializer.

You'll need to sign up for an API key if you go this route.

Edit As for Html Agility Pack, I have a blog post that shows how you can extract the links on a page

Finding links on a Web page

Shiv Kumar
  • 9,599
  • 2
  • 36
  • 38