0
header("Content-Type: text/plain; charset=utf-8");
if ($_SERVER['SERVER_PORT'] == 443) {
echo "User-agent: *\n" ;
echo "Disallow: /\n" ;
} else {
echo "User-agent: *\n" ;
echo "Disallow: \n" ;
}

What does this code do in robots.php?

I found it on my server and it seems to block text from being indexed by the search engines

Audrius Meškauskas
  • 20,936
  • 12
  • 75
  • 93
user1925594
  • 1
  • This does not seem to do too much. It is a php file that responses as if it were a robots.txt file, setting up constraints .. User-agen:* and Disallow: /. This sets up to prevent anyone on port 443 to access your root directory. – user1760422 Dec 24 '12 at 00:06
  • http://www.inkplant.com/code/https-robots.php – ialexander Dec 24 '12 at 00:07

1 Answers1

0

When you read that page on port 443 (usually reserved for secure connection) e.g. https://yoursite.com/robots.php, returned content will be as follows:

User-agent: *
Disallow: /

The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.

Otherwise (page robots.php visited on any other port - http://yoursite.com/robots.php) returned content will be as follows:

User-agent: *
Disallow:

In this case robot can visit any page on the site.

Also header("Content-Type: text/plain; charset=utf-8"); displays a page content as regular plain text.

Tom
  • 26,212
  • 21
  • 100
  • 111