robots.txt: no indexing Posted on 08/12/2013 by Ramoonus To disallow any bot from crawling your site, add the following to robots.txt robotUser-Agent: * Disallow: / Share this:XMastodonLinkedInFacebook