Create a Global IIS 7 rewrite rule

Question

I have wildcard A records (* and *.*) for my domain pointing to my dev machine. Basically, any subdomain that is not www is pointed to my dev machine (except the root).

I never want my dev machine to be index or "followed" by search engines.

What I would like to do is simple set up a global URL Rewrite rule like so:

<rule name="Global robots.txt rewrite" stopProcessing="true">
    <match url="^robots\.txt" ignoreCase="true" />
    <action type="Rewrite" url="http://localhost/robots.txt" />
</rule>

The rule above will not work; though the following redirect rule does:

<rule name="GLobal robots.txt redirect" stopProcessing="true">
    <match url="^robots\.txt" ignoreCase="true" />
    <conditions>
        <add input="{HTTP_HOST}" pattern="^localhost$" negate="true" />
    </conditions>
    <action type="Redirect" url="http://localhost/robots.txt" />
</rule>

.. but I'm not sure if 301 redirecting to a robots.txt actually works for the search engines.

Any ideas on how to accomplish what I'm attempting?

score 3 · Answer 1 · edited Sep 23 '15 at 19:58

This looks like a really old post but I found this when searching for the answer. You can achive this by using a rule that looks like this:

<rule name="Robots Disallow" stopProcessing="true">
   <match url="robots.txt" />
   <conditions logicalGrouping="MatchAll" trackAllCaptures="false">
     <add input="{HTTP_HOST}" pattern="www" negate="true" />
   </conditions>
   <action type="Rewrite" url="robots_disallow.txt" />
</rule>

The robots_disallow.txt file has the following:

User-agent: *
Disallow: /

Create a Global IIS 7 rewrite rule

1 Answers1