Home » Magento19 December 2011

Magento: Show/Hide website to search engines by adjusting robots meta tag

The Robots META Tag is meant to provide users who cannot upload or control the /robots.txt file at their websites, with a last chance to keep their content out of search engine indexes and services.

The content=”robots-terms” is a comma separated list used in the Robots META Tag that may contain one or more of the following keywords without regard to case: noindex, nofollow, all, index and follow.

noindex = Page may not be indexed by a search service.
nofollow = Robots are not to follow links from this page.

You can find more about the robots meta tag over here: http://www.seoconsultants.com/meta-tags/robots/

There is an easy way in Magento to adjust robots meta tags. You can simply choose from admin settings.

Go to System -> Configuration -> Design -> HTML Head -> Default Robots

You have the following options to choose:

– index, follow
– noindex, follow
– index, nofollow
– noindex, nofollow

If you have a live site (production site) then choose ‘index, follow‘. If you have a test site (development site) then choose ‘noindex, nofollow‘.

Hope it helps. Thanks.

Magento

Get New Post by Email

Find me on

FacebookTwitterGoogle+LinkedInRSS Feed
  • John Ramsey

    Yes but how do you change say dynamically created pages from “sort by price” etc to NoIndex, Follow? How can I take a list of say 100 urls and change just those?