2

Is there an alternative way to tell Bing not to crawl/pass/click/enter a certain link such as google's rel=nofollow?

msnbot/2.0b keeps getting trapped in my bot trap. It does not obey anchor tag rel="nofollow".

Limits:

  • I can't add the link to robots.txt because the link is a dynamic parameter key i.e. anypage.php?xirh398. The key is unique for each session and changes every 5 minutes.
  • I can't use a meta tag beacause the trap is not a page.
  • I can't use a HTTP header because the bot will already have been banned by then.
  • I don't want to continously update whitelists.
tim
  • 2,530
  • 3
  • 26
  • 45
  • Well, whatever words I must put instead. Google doesn't enter the link and I don't bing to do it either. – tim Mar 11 '13 at 14:17
  • Checkout the four methods to [remove URLs from Bing index](http://www.bing.com/blogs/site_blogs/b/webmaster/archive/2009/06/08/how-to-remove-urls-from-our-index-expanded-edition.aspx). You can also use [Bing webmaster tools](http://www.bing.com/toolbox/webmaster) to specify how Bing should treat your website. – vulcan raven Apr 01 '13 at 13:33

1 Answers1

2

It does not obey anchor tag rel="nofollow".

nofollow means "Don't give weight to this link in search results". It doesn't mean "This is not a real link, don't follow it". (Yes, that is unintuitive).

If you want Good Bots to stay out of a part of your website, use a robots.txt to tell them about it.

Quentin
  • 914,110
  • 126
  • 1,211
  • 1,335