How can I tell Google not to crawl a set of Urls

Question

How do I stop google to crawl to certain urls in my application?

For example: I want google to stop crawling all the URLs that starts with http://www.myhost-test.com/

What should I add in my robot.txt?

Did you try searching at all? – jrummell Jul 18 '12 at 13:51 — jrummell, Jul 18 '12 at 13:51
people who took down this question please specify reason ? – Kuldeep Dangi Feb 24 '16 at 09:47 — Kuldeep Dangi, Feb 24 '16 at 09:47

score 2 · Answer 1 · answered Jul 18 '12 at 13:50

2

The answer can be found directly here:

But it looks like you add "disallow" and your url.

answered Jul 18 '12 at 13:50

Shawn

1 Answers1