How to add `nofollow, noindex` all pages in robots.txt?

noindex and nofollow means you do not want your site to crawl in search engine.

so simply put code in robots.txt

User-agent: *
Disallow: /

it means noindex and nofollow.


There is a non-standard Noindex field, which Google (and likely no other consumer) supported as experimental feature.

Following the robots.txt specification, you can’t disallow indexing nor following links with robots.txt.

For a site that is still in development, has not been indexed yet, and doesn’t get backlinks from pages which may be crawled, using robots.txt should be sufficient:

# no bot may crawl 
User-agent: *
Disallow: /

If pages from the site are already indexed, and/or if other pages which may be crawled link to it, you have to use noindex, which can not only be specified in the HTML, but also as HTTP header:

X-Robots-Tag: noindex, nofollow