How to use a special character as a normal one in Unix shells?

That very much depends on the shell. Check your shell manual for details.

Also note that some characters are only special in some contexts. For instance, in most shells,* and ? are only special in list contexts, in POSIX or csh-like shells, ~ is only special at the beginning of a word or following some characters like :. Same for = in zsh. In some shells, [ is only special when matched (with some restrictions) by a ].

In some shells like bash or yash, special characters like blank token delimiters also vary with the locale.

The quoting operators (to remove the special meaning of those characters) also vary greatly between shells.

Bourne-like shells

A summary for Bourne-like shells (that is the shells that have been known to be called sh on some system or another since the 80s):

Bourne shell

Special characters:

  • "'&|;()^`<>$, space, newline and tab are special in simple command lines when not quoted.
  • # (except in early version) is special at the beginning of a line or following an unquoted space, tab or &|()^<>;`.
  • { and } are only special in that they are shell keywords (so only words in command position).
  • *?[ are special as globbing operators, so only in list contexts. In the case of [, it's [...] that is the globbing operator, either [ or ] only need to be quoted to remove the special meaning.
  • = is special when in contexts where it's treated as an assignment operator. That is, in a simple command, for all words that do not follow an argument (except after set -k).

Quoting operators

  • \ quotes all special characters except newline (\<newline> is a way to continue a long logical line onto the next physical line, so that sequence is removed). Note that backticks add extra complexity as within them, \ is used first to escape the closing backtick and help the parser. Inside double quotes, \ may only be used to escape itself, ", $ and ` (\<newline> is still a line-continuation). Inside a here-document, same except for ". \ is the only way to escape characters inside here documents.
  • "..." double-quotes escape all characters but itself, \, $ and `.
  • '...' single quotes escape all characters but itself.

POSIX shells

POSIX shells behave mostly like the Bourne shell, except that:

  • ^ is no longer a special character
  • ~ is special in some contexts
  • { is allowed to be special so should be quoted.

ksh

like POSIX except that:

  • {string} is special if string contains an unquoted , (or .. in some cases and with some versions).
  • ksh93 has an additional special quoting operator: $'...' with complex rules. That operator is also found (with some variations) in bash, zsh, mksh and FreeBSD and busybox sh.
  • ksh93 also has a $"..." quoting operator that works like "..." except that the string is subject to localisation (could be configured so it translates to the language of the user). mksh ignores the $ in $"...".

bash

like ksh93 but:

  • in single-byte character locales, all blank (according to the locale) characters are considered as delimiters (like space or tab). In effect, that means you should quote all bytes with the 8th bit set in case they may be a blank character in some locale.
  • When csh history expansion is enabled like in interactive instances, ! is special in some contexts and double quotes do not always escape it. And ^ is special at the beginning of a command.

zsh

like ksh93 but:

  • same note as for bash for csh history expansion
  • = is special as the first character of a word (=ls expands to /bin/ls).
  • { and } can also open and close command groups when not delimited (as in {echo text} works like Bourne's { echo text;}).
  • except for [ alone, [ needs quoted even if not closed with a ].
  • With the extendedglob option enabled, #, ^ and ~ are globbing operators.
  • With the braceccl option, {non-empty-string} is special.
  • $"..." is not supported.
  • as a special quirk, ? is not special when following a % (even quoted or expanded) at the start of a word (to allow the %?name job specification)
  • a rcquotes option (not enabled by default) allows one to enter single quotes as '' inside single quotes à la rc (see below).

yash

like POSIX except that.

  • all blank characters are considered as delimiters.
  • With the brace-expand option, implements zsh-style brace expansion.

For all shells, there are some special contexts where quoting works differently. We've already mentioned here documents and backticks, but there's also [[...]] in ksh and a few other shells, POSIX $((...)), case constructs...

Also note that quoting can have other side-effects when it comes to expansions (with double-quotes), or when applied to here document delimiters. It also disables reserved words and affects alias expansion.

Summary

In Bourne-like shells, !#$^&*?[(){}<>~;'"`|=, SPC, TAB, NEWLINE and some bytes with the 8th bit set are or may be special (at least in some contexts).

To remove the special meaning so they be treated literally, you use quoting.

Use:

  • '...' to remove the special meaning of every character:

    printf '%s\n' '\/\/ Those $quoted$ strings are passed literally as
single arguments (without the enclosing quotes) to `printf`'

  • \ to remove the special meaning of one character only:

    printf '<%s>\n' foo bar\ baz #comment

    Above, only the space character preceded by a \ is passed literally to printf. The other ones are treated by the shell as token delimiters.

  • use "..." to quote characters while still allowing parameter expansion ($var, $#, ${foo#bar}...), arithmetic expansion ($((1+1)), also $[1+1] in some shells) and command substitution ($(...) or the old form `...`. Actually, most of the time, you do want to put those expansions inside double quotes in any case. You can use \ within "..." to remove the special meaning of the characters that are still special (but only them).

  • if the string contains ' character, you can still use '...' for the rest and use other quoting mechanisms that can quote ' like "'" or \' or (where available) $'\'':

    echo 'This is "tricky", isn'\''t it?'

  • Use the modern $(...) form of command substitution. Only use the old `...` for compatibility with the Bourne shell, that is to very old system, and only in variable assignments, as in don't use:

    echo "`echo "foo bar"`"

    Which won't work with the Bourne shell or AT&T versions of ksh. Or:

    echo "`echo \"foo bar\"`"

    Which will work with Bourne and AT&T ksh, but not with yash (2020 edit: only in version 2.41 and earlier though, it's since been changed in 2.42 / bug report / commit), but use:

    var=`echo "foo bar"`; echo "$var"

    which will work with all.

    Nesting them portably with double quotes is also impossible, so again, use variables. Also beware of the special backslash processing:

    var=`printf '%s\n' '\\'`

    Will store only one backslash inside $var, because there's an extra level of backslash processing (for \, `, and $ (and also " when quoted except in yash)) within backticks so you need either

    var=`printf '%s\n' '\\\\'`

    or

    var=`printf '%s\n' '\\\'

    instead.

Csh family

csh and tcsh have a significantly different syntax, though there is still a lot in common with the Bourne shell as they share a common heritage.

Special characters:

  • "'&|;()^`<>$, space, newline and tab are special everywhere when not quoted.
  • # (csh is the shell that introduced # as the comment leader) is special at the beginning of a script or following an unquoted space, tab or newline.
  • *?[ are special as globbing operators so in list contexts
  • {non-empty-string} is special (csh is the shell that introduced brace expansion).
  • ! and ^ are special as part of history expansion (again, a csh invention), and quoting rules are special.
  • ~ (tilde expansion also a csh invention) is special in some contexts.

Quoting operators

They are the same as for the Bourne shell, but the behaviour differs. tcsh behaves like csh from the syntax point of view, you'll find that many versions of csh have nasty bugs. Get the latest version of tcsh to get a roughly working version of csh.

  • \ escapes a single character except newline (same as for the Bourne shell). It's the only quoting operator that can escape !. \<newline> doesn't escape it but transforms it from a command separator to a token separator (like space)
  • "..." escapes all characters except itself, $, `, newline and !. Contrary to the Bourne shell, you can't use \ to escape $ and ` inside "...", but you can use \ to escape ! or newline (but not itself except when before a ! or newline). A literal ! is "\!" and a literal \! is "\\!".
  • '...' escapes all characters except itself, ! and newline. Like for double quotes, ! and newline can be escaped with backslash.
  • command substitution is only via the `...` syntax and can hardly be used reliably.
  • variable substitution is also pretty badly designed and error prone. A $var:q operator helps to write more reliable code involving variables.

Summary

Stay away from csh if you can. If you can't use:

  • single quotes to quote most characters. ! and newline still need a \.
  • \ can escape most characters
  • "..." can allow some expansions within it, but that's pretty buggy if they embed newline and/or backslash characters, best may be to use single quotes only and $var:q for variable expansion. You'll need to use loops if you want to join elements of an array reliably.

rc family

rc is the plan9 shell and like its descendants es and akanga has been ported to Unix and unix-likes. That's a shell with a much cleaner and better syntax and the one everyone would be using if we weren't stuck with Bourne-like shells for backward compatibility.

rc/akanga

Special characters

  • #;&|^$=`'{}()<>, SPC, TAB and NEWLINE are always special when not quoted.
  • *?[ are globbing operators.

Quoting operator

'...' is the only quoting operator. A litteral ' is written with '' within single quotes as in:

echo 'it''s so simple isn''t it?'

es

es could be seen as an experimental shell based on rc.

It has a few differences though. The one of interest for this Q/A is that \ is also a quoting operator (that quotes all special characters except newline) and can also be used to introduce escape sequences like \n for newline, \b for backslash...

fish

fish is a relative newcomer (circa 2005), is primarily intended for interactive use and also has a significantly different syntax from other shells.

special characters

  • "'\()$%{}^<>;&| always special when not quoted (note the % (for pid expansion) as a significant difference from other shells, and ` is not special)
  • # (comment) special when following unquoted space, tab, newline or ;&|^<>
  • *? (but not [...]) globbing operators

Quoting operators

  • \ quotes a single special character except newline, but beware it also doubles as a C escape sequence (\n, \b...) introducer. IOW, \n is not a quoted n but a newline.
  • "..." quotes everything but itself, $ and backslash and backslash can be used to escape those. \<newline> is a line continuation (removed) inside "...".
  • '...' quotes everything but itself and \, and you can use backslash to escape those.

1. Escape

Escape all of these characters with a \, like this (does not work on newlines/carriage returns):

$ echo Use a \"\\\" symbol to escape characters.
Use a "\" symbol to escape characters.

2. Double-quote

Enclose the whole text in "s, like this:

$ var=variables;echo "Enclose text in \"s. You can also use $var in them. `echo Surprise!!!`"
Enclose text in "s. You can also use variables in them. Surprise!!!

3. Single-quote

Same as double-quote, but there is no special token.

$ proof=proveit;echo 'This should not read "proveit": $proof'
This should not read "proveit": $proof

Tags:

Shell

Special Characters

Quoting

Recent Posts