Robots Meta Tag

Robots Meta Tag

Example Robots Meta Tag;

<meta name="robots" content="index, nofollow" />

I could use the above meta tag to tell Google to index the page but not to follow any links on the page, if for some reason, I did not want the page to appear in Google search results.

By default, Googlebot will index a page and follow links to it. So there’s no need to tag pages with content values of INDEX or FOLLOW. GOOGLE

There are various instructions you can make use of in your Robots Meta Tag, but remember Google by default WILL index and follow links, so you have NO need to include that as a command – you can leave the robots meta out completely – and probably should if you don’t have a clue.

Googlebot understands any combination of lowercase and uppercase. GOOGLE.

Valid values for Robots Meta Tag ”CONTENT” attribute are: “INDEX“, “NOINDEX“, “FOLLOW“, and  “NOFOLLOW“.

Example Usage:

  • META NAME=”ROBOTS” CONTENT=”NOINDEX, FOLLOW”
  • META NAME=”ROBOTS” CONTENT=”INDEX, NOFOLLOW”
  • META NAME=”ROBOTS” CONTENT=”NOINDEX, NOFOLLOW”
  • META NAME=”ROBOTS” CONTENT=”NOARCHIVE”
  • META NAME=”GOOGLEBOT” CONTENT=”NOSNIPPET”

Google will understand the following and interprets the following robots meta tag values:

  • NOINDEX – prevents the page from being included in the index.
  • NOFOLLOW – prevents Googlebot from following any links on the page. (Note that this is different from the link-level NOFOLLOW attribute, which prevents Googlebot from following an individual link.)
  • NOARCHIVE – prevents a cached copy of this page from being available in the search results.
  • NOSNIPPET – prevents a description from appearing below the page in the search results, as well as prevents caching of the page.
  • NOODP – blocks the Open Directory Project description of the page from being used in the description that appears below the page in the search results.
  • NONE – equivalent to “NOINDEX, NOFOLLOW”.

Robots META Tag Quick Reference

Terms Googlebot Slurp BingBot Teoma
NoIndex YES YES YES YES
NoFollow YES YES YES YES
NoArchive YES YES YES YES
NoSnippet YES NO NO NO
NoODP YES YES YES NO
NoYDIR NO YES NO NO
NoImageIndex YES NO NO NO
NoTranslate YES NO NO NO
Unavailable_After YES NO NO NO

I’ve included the robots meta tag in my tutorial as this IS one of only a few meta tags / HTML head elements I focus on when it comes to managing Googlebot and Bingbot. At a page level – it is a powerful way to control if your pages are returned in search results pages.

These meta tags go in the [HEAD] section of a [HTML] page and represent the only tags for Google I care about. Just about everything else you can put in the [HEAD] of your HTML document is quite unnecessary and maybe even pointless (for Google optimisation, anyway).

If you'd like us to dramatically improve your website & business, visit our "Services" page and then contact us for more information. And if you have any comments or questions about this article, contact us or Call +91-9915-337-448.

Do you want more customers?

Hey, I will make them more rewarding by improving your overall Returns on Investment. Fill the form below.