ASP.NET HttpHandler mapping to Robots.txt without access to IIS

Question

Is there a simple way to map /robots.txt to a HttpHandler using only the web.config? I've tried all sorts of changes to the httpHandlers tag but none have made any difference. All the examples require modifications to IIS site properties, which I cannot access.

Is there a way using only the web.config to map:

<add verb="*" path="/robots.txt" type="Site.RobotsHandler" />

correctly?

I'm currently using .NET 3.5 and MVC2.

Which iis version are you using? – Kenneth Ito Apr 23 '12 at 23:29 — Kenneth Ito, Apr 23 '12 at 23:29

Anthony Shaw · Answer 1 · 2012-04-23T20:21:37.653

1

If you're using MVC, why not create a route to an Action and handle it there rather than creating another handler?

routes.MapRoute("robots.txt", "robots.txt", new { controller = "Home", action = "Robots" });

EDIT BASED ON COMMENT

routes.IgnoreRoute("{file}.txt", new { pathInfo = new RobotsIgnore() });

public class RobotsIgnore : IRouteConstraint
{
    public bool Match(HttpContextBase httpContext, Route route, string parameterName, RouteValueDictionary values, RouteDirection routeDirection)
    {
        return values[parameterName].ToString().ToLowerInvariant() != "robots.txt";
    }
}

edited Apr 23 '12 at 20:21

answered Apr 23 '12 at 19:46

Anthony Shaw

8,146
4
44
62

That was the first thing I tried, ... It doesn't appear to work for static files. You just get "The resource cannot be found." error, as though it were looking directly for robots.txt. – Rob Apr 23 '12 at 19:50
edited my answer based on your comment, see if that doesn't help. I've used very similar code to handle css files in the same manner you want to handle your robots.txt file. Basically you're telling the route processor to ignore all .txt files except robots.txt, and then make sure you still use the initial route I gave you and it should work – Anthony Shaw Apr 23 '12 at 19:59
No difference, what did you change? – Rob Apr 23 '12 at 20:04
I think the difference here is that CSS files are handled directly by asp.net and mvc, however txt files are handed over to the static file handler. – Rob Apr 23 '12 at 20:04
sorry, it never posted my change, only my comment. – Anthony Shaw Apr 23 '12 at 20:21
This still doesn't work for me, thanks for the help though. From everything I've researched, you need to be able to tell ISS to map txt files to ASPNET handler: http://msdn.microsoft.com/en-us/library/46c5ddfy.aspx - I can't do that unfortunately :( - the *.txt extension is handled before it even reaches .NET. – Rob Apr 23 '12 at 20:44

score 0 · Accepted Answer · answered Apr 23 '12 at 23:35

0

Based off some of the comments on other answers I'm guessing you're using iis6? As far as I know it's not possible to alter iis6 handler mappings via web config. .txt files will never reach asp.net without changing default mappings.

answered Apr 23 '12 at 23:35

Kenneth Ito

5,201
2
25
44

ASP.NET HttpHandler mapping to Robots.txt without access to IIS

2 Answers2