0

I have been seeing very high number of dynamic URLs on Google Webmaster tools. There is no way Google search bot can find out these URLs unless (reasons I can think of)

  1. Google search bot can choose multiple options in my HTML form and click on the submit button
  2. Google search bot can read through Google Analytics stats and add those URLs to the search index.

Now based on my limited knowledge, both of the above should not be possible. One of my other sites has a +1 button and that could be another potential reason.

Any insights on this will be highly appreciated.

Pratik Thakkar
  • 395
  • 2
  • 4
  • 16

1 Answers1

0

The Google bot crawls your site and looks for URLs in the HTML. There's a link to each dynamic URL somewhere on your site...or linked to from another site.

  • does your site have a search function?
  • does your sitemap contain dynamic URLs?
  • are "anonymous" permissions in your CMS too loose?
  • Thanks for your inputs on this. Yes, I do have an advanced search function & XML sitemap but the sitemap only has static links. Also I am sure that no dynamic links exist on the website. CMS permissions are too strict and there cannot be any leakage there. That is where the whole confusion lies. – Pratik Thakkar Jun 12 '13 at 05:19
  • Check out your referral information. My guess is that a form POST URL is cached somewhere, probably on a different website. You should see a referral to the dynamic page from the offending site. –  Jun 12 '13 at 18:42
  • I could not find much referral traffic to these URLs. Also I am using the GET method & not POST. Will that make any difference to the way Google search bot would work? – Pratik Thakkar Jun 13 '13 at 08:58