How to block this kind of URLs in robot.txt file?

Question

here are two set of conditions and I want to block all url with /search/jobdescription? and search/jobdescription/

1) http://<--sitename-->/search/jobdescription?id=0154613&css=a&act=a

2) http://<--sitename-->/search/jobdescription/id/0154613/css/a/act/a

How to block both occurances in robot.txt file.

Thanks

You may wish to revisit your old questions and accept answered where applicable. — lethalMango, May 10 '12 at 09:18

score 2 · Accepted Answer · answered May 10 '12 at 09:19

2

As simple as:

User-agent: *
Disallow: /search/jobdescription

answered May 10 '12 at 09:19

Aziz

score 0 · Answer 2 · answered May 10 '12 at 09:20

0

You can also use the X-Robots-Tag HTTP header to prevent indexing.

answered May 10 '12 at 09:20

Yohann

2 Answers2