What does the --color=auto option for GNU grep mean?

The rules are the same as for ls, which does a better job documenting it in man ls. Quoting:

Using color to distinguish file types is disabled both by default and
with --color=never. With --color=auto, ls emits color codes only when
standard output is connected to a terminal. The LS_COLORS environment
variable can change the settings. Use the dircolors command to set it.

So it will make the command only add the color formatting when the output is going to a terminal and not, say, when it is going to a pipe where the program consuming the pipe might not handle the color formatting well.

Since grep is a GNU program another option might be having a look at the source code.

---

Internally grep tests the static int color_option for either 0,1 or 2.

• 0 never use colorized output
• 1 always use colors
• 2 only use colors when printing to a terminal

Now when you hand over --color=auto to grep as an argument on your CLI, it internally sets the variable color_option to 2.

If color_option equals 2 grep then further tests whether STDOUT is linked to a terminal or the user disabled colorized outpit via shell environment variables. If the former one is true and colorized output is permitted, grep then continues with evaluating which colors should be used and in the end finally prints out to your CLI in color.

---

source: (grep 2.21)

grep.c line 306, 2374, 2440
colorize-posix.c line 36
man isatty