Block search engines

What are we cooking today?
How to block Search Engines from indexing site.


Recipe

1. Add a robot.txt file with instructions to prevent all robots from indexing the page
   User-agent: * Disallow: /
2. I added a tag in the links to prevent from indexing too.
   Sample http://www.mylink.com rel="nofollow">My Link
   
3. I added a metatag :  
   
   
   Other details
   
   

   To exclude all robots from the entire server
   User-agent: * Disallow: /

   To allow all robots complete access
   User-agent: * Disallow:
   Or create an empty “/robots.txt” file.

   To exclude all robots from part of the server
   User-agent: * Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /private/

   To exclude a single robot
   User-agent: BadBot Disallow: /

   To allow a single robot
   User-agent: WebCrawler Disallow: User-agent: * Disallow: /

   More on robots --> http://www.robotstxt.org/