I have been using nmap to scan an IP address, and one part of the output is:
| http-robots.txt: 1 disallowed entry
What does this mean?
| http-robots.txt: 1 disallowed entry
Asked
Active
Viewed 4,427 times
1 Answers
2
It is an entry in the robots.txt file of the form Disallow: X. This means that at least some User-Agent has been instructed not to request these URIs. A list of them should be listed below the line you showed. You can read more about the robots.txt standard and format on robotstxt.org.
bonsaiviking
- 5,825
- 1
- 20
- 35