0

I am getting some hreflang and redirect errors on ahrefs.com because of utm campaigns and popup links that have anchors set up as "#" or "#!". I want to add some code on my robots.txt file in order to prevent google from crawling these urls so that those errors disappear.

For example the urls I want to disallow crawling are:

https://mywebsite.com/#
https://mywebsite.com/#!
https://mywebsite.com/about?utm_campaign=something

I tried this code but it isn't working:

Disallow: /*#!

Disallow: /*utm_campaign

zestzahir
  • 117
  • 1
  • 10

1 Answers1

0

Hash(#) is a sign of comment in robots.txt. Everything after # will be ignored in robots.txt

Also browsers and crawlers never send the part of the URL after the # to the server, so it's doesn't make sense to disallow it.

Boris Volkovich
  • 115
  • 6