0

I want to disallow with robots.txt from indexing my site available with all domain endings (co.uk, in, pl, de, nl ...etc), but allow only with com-ending.

I haven't physical access to htaccess file (affected is a blogger.com blog), otherwise i would just redirect all endings to the wishful one. But while i can't redirect, i want even to disallow all endings, beside of .com.

Sure, i could just create a list with all endings and disallow them one by one, but it seems to me, there must be a smarter solution for this issue, which i haven't found till now.

How to do that?

unor
  • 92,415
  • 26
  • 211
  • 360
Evgeniy
  • 2,337
  • 2
  • 28
  • 68
  • You realize `robots.txt` doesn't *actually prevent* viewings, but simply hints at the *relevant* bits of site information for robots to look at... right? – Qix - MONICA WAS MISTREATED Aug 10 '14 at 05:21
  • Yes, sure! While i can't prevent viewing through redirecting with htaccess, i want at least disallow indexing to prevent duplicated content from each domain ending. – Evgeniy Aug 10 '14 at 05:24
  • The prefered way would be `301 Moved Permanently` header for your main domain – Christian Gollhardt Aug 10 '14 at 06:30
  • @ Christian: redirects aren't possible at blogger.com. Beside of this, what concretely i achieve with 301 for the main domain, and where should it be pointed in your opinion? – Evgeniy Aug 10 '14 at 07:32

1 Answers1

1

Answer for all, who is looking for:

  1. The issue can't be solved with robots.txt.

  2. But Blogger's default template (and other well done templates) contains a rel="canonical" tag, which points to the URL with the "main" domain ending and in this manner figures out the issue with duplicated content from domains with all other domain endings.

Evgeniy
  • 2,337
  • 2
  • 28
  • 68