how to read a robots.txt file code example Example 1: seo disallow User-agent: * Disallow: / Example 2: robots txt no index entire site User-agent: * Disallow: /