6

I have a public page that is not supposed be possible for users to sign into. So I have a url that there is no link to and you have to enter manually and then sign in.

The url is multilanguage however, so it can be "/SV/Account/Logon" or "/EN/Account/Logon" etc etc. Can I disable this url to be indexed for all languages?

ajay_whiz
  • 17,573
  • 4
  • 36
  • 44
Oskar Kjellin
  • 21,280
  • 10
  • 54
  • 93

1 Answers1

3

This url might help you http://www.robotstxt.org/robotstxt.html. There will be no wildcard exclusion in your case you will have to add all the language specific login urls in robot.txt

Update

you can put Disallow: /folder/subfolder/ or Disallow: /folder/ in your robots.txt file

ajay_whiz
  • 17,573
  • 4
  • 36
  • 44
  • Thanks for answering. I read that and do not wan't to do this. For example other users might add more languages using the admin interface and they do not know how to edit the robots.txt. Perhaps I can edit it from my code when adding more languages – Oskar Kjellin Sep 03 '10 at 08:36
  • @Oskar you can also consider moving such pages under a single folder and just add that folder in the exceptions list in robots.txt – ajay_whiz Sep 03 '10 at 09:43
  • @ajay_whiz They are all under the same folder. But can I write folders? I thought the URLs where the only thing affecting this – Oskar Kjellin Sep 03 '10 at 09:55
  • 1
    @Oskar yes, you can put as `Disallow: /folder/subfolder/` or `Disallow: /folder/` – ajay_whiz Sep 03 '10 at 10:13
  • @ajay_whiz Thanks, please post an answer so I can accept it :) – Oskar Kjellin Sep 03 '10 at 10:47