Meta Robots Tag
One other meta tag worth mentioning is the robots tag. This lets you specify that a particular page should NOT be indexed by a search engine. To keep spiders out, simply add this text between your head tags on each page you don't want indexed. The format is shown below (click on the picture if you want to copy and past the HTML for your own use):
You do NOT need to use variations of the meta robots tag to help your pages get indexed. They are unnecessary. By default, a crawler will try to index all your web pages and will try to follow links from one page to another.
Most major search engines support the meta robots tag. However, the robots.txt convention of blocking indexing is more efficient, as you don't need to add tags to each and every page. See the Search Engines Features page for more about the robots.txt file. If you use do a robots.txt file to block indexing, there is no need to also use meta robots tags.
The meta robots tag also has some extensions offered by particular search engines to prevent indexing of multimedia content. The article below talks about this in more depth and provides some links to help files. Search Engine Watch members should follow the link from the article to the members-only edition for extended help on the subject.
Was this information useful? What other tips would you like to read about in the future? Share your comments, feedback and experiences with us by commenting below!
Like our site? Be a member of Coolajax fan page to get daily posts updates!
You May Like This Posts!
» Best Free eBook Websites For Downloading Free eBooks
Tags:
robots tag, meta tag, use robots tag, HTML tutorial, robots.txt , crawler,block crawler indexing 
Total Read: 951 
Topics
Categories
Information
Connect
