0

How to create a robots.txt file in a codeigniter project to hide a view page . where should i place this robots.txt file currently i have created file like this

User-agent: *
Disallow: /application/views/myviewpage.php

in side /public_html/folder/robots.txt (Where i place my .htaccess file). Is there any way to test this?

Cœur
  • 37,241
  • 25
  • 195
  • 267
Juice
  • 3,023
  • 6
  • 39
  • 66
  • You don't block the view file as that isn't directly accessible to the crawlers. You need to block the URL that is used to access your view. – Jeemusu Nov 04 '13 at 05:06
  • @Jeemusu should i give the complete url inside robots.txt file like this `http://www.server.com/folder/index.php/mycontroller/my_function` – Juice Nov 04 '13 at 05:52
  • You don't need to specify the full url. Just `/index.php/mycontroller/my_function` – Jeemusu Nov 04 '13 at 10:54

1 Answers1

0

The robots.txt file MUST be placed in the document root of the host. It won’t work in other locations.

If your host is example.com, it needs to be accessible at http://example.com/robots.txt.

If your host is www.example.com, it needs to be accessible at http://www.example.com/robots.txt.

unor
  • 92,415
  • 26
  • 211
  • 360