Questions tagged [googlebot]

I have : RewriteRule ^Article/([^/]*)$ /article.php?newsid=$1 [L] Which means that the URL must be //example.com/Article/855563 but Google crawls //example.com/article.php?newsid=855563. Is there anything I can do to prevent this? Or to redirect…

I just updated my robots.txt file on a new site; Google Webmaster Tools reports it read my robots.txt 2 days before my last update. my last robots.txt had a "disallow: all" raw. Is there any way I can encourage Google to re-read my robots.txt as…