0

We have production and test environment as any other company. And I was thinking to put a robots.txt into the UAT root folder that Google web crawler would not do an unwarranted crawl on the uat pages.

But what I found out was surprising. I do not have robots.txt anywhere on my Uat box and still the none of the Uat sites show up in google search. Uat is not behind firewall so any outside computer can see the machine. Some how none of the Google searches are pulling any of the web site on my Uat box out.

Any ideas?

Lost
  • 12,007
  • 32
  • 121
  • 193
  • Google will only find your site if there are links coming into your system. Are there public links into your UAT environment? – PaulProgrammer May 10 '13 at 21:04
  • Yes there are public links in UAT environment. Those are the links that we give to our clients for testing. Anyone can browse to those links without authentication or white listing. – Lost May 10 '13 at 21:06
  • 1
    But, are they on another site that is crawled by google, or do you just send them in email? If they are published on a public website (e.g. your corporate home page), are those pages set to `nofollow` links? If the spider can't discover your service, it won't index it. – PaulProgrammer May 10 '13 at 22:04
  • I am sorry for the late reply. You are absolutely correct. My Uat site is not referenced anywhere so it is not showing up in any of the searches. thank you so much:)) – Lost May 30 '13 at 23:58

0 Answers0