0

I keep getting BingBot trying to access the file below. There is no robots.txt file at that location (http://test.com/8329/test/documents). How can I stop this?

C:\Test\Website\8329\test\documentsrobots.txt

Exception

System.Web.HttpException: A public action method 'documentsrobots.txt' was not found on controller 'Tournaments.Controllers.EventController'.

Generated: Fri, 18 Oct 2013 17:24:45 GMT

System.Web.HttpException (0x80004005): A public action method 'documentsrobots.txt' was not found on controller 'Tournaments.Controllers.EventController'.
   at System.Web.Mvc.Controller.HandleUnknownAction(String actionName)
   at System.Web.Mvc.Controller.<>c__DisplayClass1d.<BeginExecuteCore>b__18(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`1.End()
   at System.Web.Mvc.Controller.EndExecuteCore(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`1.End()
   at System.Web.Mvc.Controller.EndExecute(IAsyncResult asyncResult)
   at System.Web.Mvc.Controller.System.Web.Mvc.Async.IAsyncController.EndExecute(IAsyncResult asyncResult)
   at System.Web.Mvc.MvcHandler.<>c__DisplayClass8.<BeginProcessRequest>b__3(IAsyncResult asyncResult)
   at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass4.<MakeVoidDelegate>b__3(IAsyncResult ar)
   at System.Web.Mvc.Async.AsyncResultWrapper.WrappedAsyncResult`1.End()
   at System.Web.Mvc.MvcHandler.EndProcessRequest(IAsyncResult asyncResult)
   at System.Web.Mvc.MvcHandler.System.Web.IHttpAsyncHandler.EndProcessRequest(IAsyncResult result)
   at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
Mike Flynn
  • 22,342
  • 54
  • 182
  • 341
  • I find it highly unlikely that BingBot is trying to access a robots.txt file that's in a subdirectory. More likely it found a bad link somewhere and keeps trying to hit it. Are you sure that your site doesn't have that link (or generate that link) somewhere? – Jim Mischel Oct 24 '13 at 21:17
  • No it doesnt, the user agent is bing bot. – Mike Flynn Nov 15 '13 at 19:42
  • Seems like your Web site shouldn't throw an exception when somebody tries to access a path that doesn't exist. Your app should detect the error and return a 404, Not Found. That would stop Bing from trying to hit it again. – Jim Mischel Nov 15 '13 at 22:11

0 Answers0