The Robots META Tag is meant to provide users who cannot upload or control the /robots.txt file at their websites, with a last chance to keep their content out of search engine indexes and services.

<meta name="robots" content="robots-terms">

The content=”robots-terms” is a comma separated list used in the Robots META Tag that may contain one or more of the following keywords without regard to case: noindex, nofollow, all, index and follow.

noindex = Page may not be indexed by a search service.
nofollow = Robots are not to follow links from this page.

You can find more about the robots meta tag over here: http://www.seoconsultants.com/meta-tags/robots/

There is an easy way in Magento to adjust robots meta tags. You can simply choose from admin settings.

Go to System -> Configuration -> Design -> HTML Head -> Default Robots

You have the following options to choose:

- index, follow
- noindex, follow
- index, nofollow
- noindex, nofollow

If you have a live site (production site) then choose ‘index, follow‘. If you have a test site (development site) then choose ‘noindex, nofollow‘.

Hope it helps. Thanks.

Get New Post by Email

Subscribe RSS Feed Follow @chapagain