why to disallow search in robots.txt code example Example 1: robots.txt disallow User-agent: * Disallow: / Example 2: robots txt disallow all User-Agent: * Disallow: /