Block certain Addresses from being Crawled Robots.txt

Question

I've been asked (in order to improve SEO results) to remove a certain web address from a Robots.txt.

I'm struggling (SEO is not my forte) to figure out how to block a certain domain, even though both addresses point to the same website.

For Example:

http://foo.example.com -- Web Address I need to stop from being Indexed
http://www.examplefoo.com -- Web Address I want to be indexed

From What I can tell, using a disallow / will only block from both (as they are the same site)

Any help?! Sorry if this is a silly question..

score 1 · Accepted Answer · edited Nov 13 '15 at 03:48

Step 1: If your first domain in up, you should redirect it to your second domain.

Sample (width htaccess):

RewriteCond %{HTTP_HOST} ^www.xxx.ir$ [NC,OR]

RewriteCond %{HTTP_HOST} ^xxx.ir$ [NC,OR]

RewriteCond %{HTTP_HOST} ^yyy.com$ [NC]

RewriteRule ^(.*)$ http://www.yyy.com/$1 [R=301,L]

Step 2: You should remove all urls (from first domain) by remove urls in webmaster tools (directory all)

good luck

Block certain Addresses from being Crawled Robots.txt

1 Answers1