How to ignore some links in my website?

Question

I am working on a small php script and i have some links like this

*-phones-*.html

* are variables i want to disallow google to index this kind of links using robots.txt, it is possible ?

Answer 1

You're not disallowing anything. robots.txt is just a set of guidelines for webcrawlers, who can choose to follow them or not.

Rude crawlers should of course be IP banned. But you can't avoid that the webcrawler might come across that page. Anyway, you can add it to your robots.txt and googles webcrawler might obey.