Hiding Magento's Layered Navigation from Search Engines

Question

I’m hoping someone can help me with this problem I’ve been trying to solve for the past few days. I want to hide Magento’s Layered Navigation from the search engines entirely, but make it available to users. For SEO reasons, I don’t want to settle for NoFollowing all the links, or using noindex follow meta tags, or even blocking it entirely with Robots.txt. The most effective way of handling this would be only showing the layered Navigation to users with Cookies enabled, since Google doesn’t use cookies. The same effect could probably be achieved with JavaScript as well, but I’ve chosen the Cookie method.

So far I’ve managed to implement a crude piece of JS to check if cookies are enabled once the page has loaded (adapted from another thread on this forum). If cookies are enabled, it does nothing and layered nav displays, but if cookies are not enabled, I want to remove the “catalog.leftnav” block. I can’t for the life of me figure out how to do this from my JS script. All I’ve been able to achieve is removing the div element, or setting style.display to none etc., and while all of these techniques remove the links from the frontend, Google can still see them all. Here’s an example of the code I have so far in template/catalog/layer/filter.phtml

<div id="shop-by-filters">
  <ol>
    <?php foreach ($this->getItems() as $_item): ?>
    <li>
    <?php if ($_item->getCount() > 0): ?>
    <a href="<?php echo $this->urlEscape($_item->getUrl()) ?>" rel="nofollow"><?php echo $_item->getLabel() ?></a>
    <?php else: echo $_item->getLabel() ?>
    <?php endif; ?>
    <?php if ($this->shouldDisplayProductCount()): ?>
    (<?php echo $_item->getCount() ?>)
    <?php endif; ?>
    </li>
    <?php endforeach ?>
  </ol>
</div>

<script type="text/javascript">
if (navigator.cookieEnabled) {
    return true; 
} else if (navigator.cookieEnabled === undefined) {
    document.cookie = "testcookie";
    if (cookie_present("testcookie"))
        return true;   
} else {
var elem = document.getElementById('shop-by-filters');
elem.parentNode.removeChild(elem);
}
</script>

Can anyone help me with this, or is there a better way of going about it? Please keep in mind that I am still trying to get my head around Magento, so I might need some instructions if the implementation is complicated.

Thank you.

Brendon

Answer 1

I'm not sure if the Google robot will reliably parse your javascript.

You may be better off hiding the layered nav based on the current session with php.

<?php if (Mage::getSingleton('customer/session')): ?>
    ...your nav code...
<?php endif ?>

Answer 2

First of all, Javascript will do nothing to stop Google from indexing that content.

Why don't you want 'to settle for NoFollowing all the links'? That is exactly what NoFollow is for. You can also tell Google to not pay attention to the qualifiers/query strings in Webmaster Tools.

If for some reason you really wanted to hide that block from Google, edit the template and string compare $_SERVER['HTTP_USER_AGENT'] against Google's very public list of user agents here http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1061943

EDIT -- string compare

<?php if (stripos($_SERVER['HTTP_USER_AGENT'], 'Googlebot') !== false): ?>
<div id="shop-by-filters">
  <ol>
    <?php foreach ($this->getItems() as $_item): ?>
    <li>
    <?php if ($_item->getCount() > 0): ?>
    <a href="<?php echo $this->urlEscape($_item->getUrl()) ?>" rel="nofollow"><?php echo $_item->getLabel() ?></a>
    <?php else: echo $_item->getLabel() ?>
    <?php endif; ?>
    <?php if ($this->shouldDisplayProductCount()): ?>
    (<?php echo $_item->getCount() ?>)
    <?php endif; ?>
    </li>
    <?php endforeach ?>
  </ol>
</div>
<?php endif; ?>

Answer 3

It is a slick subject. We used this code to hide the layered navigation from Google, but we are not sure is it working...

<div id="filters-no-follow"></div>

<?php
function prepare_for_echo($string) {
$no_br = trim(preg_replace('/\s+/', ' ', $string));
$no_slashes = str_replace('\'', '\\\'', $no_br);
return $no_slashes;
}
?>

<script>
function please_enable_cookies() {
var f = document.getElementById('filters-no-follow');
f.innerHTML = '<div class="no-cookies-error">Enable cookies to choose filters.</div>';
}

function please_load_filters() {
var f = document.getElementById('filters-no-follow');
f.innerHTML = '<?php if ( !empty($filtersHtml) || !empty($stateHtml) ): ?>'
+ '\n<div class="block block-layered-nav">'
+ '\n    <div class="block-title">'
+ '\n        <strong><span><?php echo prepare_for_echo($this->__('Shop By')); ?></span></strong>'
+ '\n    </div>'
+ '\n    <div class="block-content">'
+ '\n        <?php echo prepare_for_echo($this->getStateHtml()); ?>'
+ '\n        <?php if ($this->canShowOptions()): ?>'
+ '\n        <p class="block-subtitle"><?php echo prepare_for_echo($this->__('Shopping Options')); ?></p>'
+ '\n        <dl id="narrow-by-list">'
+ '\n            <?php echo prepare_for_echo($filtersHtml); ?>'
+ '\n        </dl>'
+ '\n        <?php endif; ?>'
+ '\n    </div>'
+ '\n</div>'
+ '\n<?php endif; ?>';
}

function are_cookies_enabled()
{
    var cookieEnabled = (navigator.cookieEnabled) ? true : false;

    if (typeof navigator.cookieEnabled == "undefined" && !cookieEnabled)
    { 
        document.cookie="testcookie";
        cookieEnabled = (document.cookie.indexOf("testcookie") != -1) ? true : false;
    }
    return (cookieEnabled);
}

if(are_cookies_enabled()) {
please_load_filters();
} else {
please_enable_cookies();
}
</script>