Question about exercise 20.7 in the TeXbook
The exercise is about interpreting
\def\!!1#2![{!#]#!!2}
when [
, ]
and !
have category code 1, 2 and 6, respectively.
The paragraph before the quoted one reads
Incidentally, if you display this with
\tracingmacros=1
, TeX says\!!1#2[->{##]!!#2[ #1<-x #2<-[y]
The duplication of the category code 6 characters is evident in the top line: the expansion of \!
is shown to be
{##]!!#2
which means that the input stream will receive
{
1#
6]
2!
6[
1y
11]
2
and a left brace is expected.
That part of the answer refers to the output of \tracingmacros=1
. It also contains a mistake. (Edit: I got carried away by another discrepancy in the answer. The OP pointed out the correct interpretation; there is no mistake in this sentence.)
Exercise 20.7 on page 205, marked with a double “dangerous bend” symbol, asks:
Suppose that ‘
[
’, ‘]
’, and ‘!
’ have the respective catcodes 1, 2, and 6, as do ‘{
’, ‘}
’, and ‘#
’. See if you can guess what the following definition means:\def\!!1#2![{!#]#!!2}
What token list will result when ‘
\! x{[y]][z}
’ is expanded?
As mentioned on page 37, catcode 1 is “beginning of group” (like {
). Catcode 2 is “end of group” (like }
). Catcode 6 is parameter (like #
). So to answer the question first:
\def\!!1#2![{!#]#!!2}
┗━┳┛┗┫┗┫┗┫┃┃┃┗┫┃┗┫┗┫┃
┃ ┃ ┃ ┃┃┃┃ ┃┃ ┃ ┃┃
┗━━╋━╋━╋╋╋╋━╋╋━╋━╋╋━━━━━ \def primitive defines a macro.
┗━╋━╋╋╋╋━╋╋━╋━╋╋━━━━━ \! means that the command being defined is \! (like the \foo in \def\foo{bar} etc).
┗━╋╋╋╋━╋╋━╋━╋╋━━━━━ !1 is like #1 and means parameter 1, the first parameter passed to the macro when used.
┗╋╋╋━╋╋━╋━╋╋━━━━━ #2 means parameter 2, the second parameter passed to the macro when used.
┗╋╋━╋╋━╋━╋╋━━━━━ ! (as it's of catcode 6 and followed by a catcode 1 token) means, by the special rule mentioned after 20.5,
┃┃ ┃┃ ┃ ┃┃ that a token [ namely catcode 1, char 91 is inserted at end of both parameter text and replacement text.
┗╋━╋╋━╋━╋╋━━━━━ [ as it has catcode 1 is like { so this marks end of parameter text / start of replacement text of the macro
┗━╋╋━╋━╋╋━━━━━ { means the token { namely catcode 1, char 123
┗╋━╋━╋╋━━━━━ !# as it is two consecutive chars with catcode 6, means the token # namely catcode 6, char 35 -- HERE THE LATTER CHAR IS USED
┗━╋━╋╋━━━━━ ] means the token ] namely catcode 2, char 93
┗━╋╋━━━━━ #! as it is two consecutive chars with catcode 6, means the token ! namely catcode 6, char 33 -- HERE THE LATTER CHAR IS USED
┗╋━━━━━ !2 is like #2 and means parameter 2, the second parameter passed to the macro when used.
┗━━━━━ } marks the end of the replacement text and macro definition, but recall there's a { at end, by the special rule.
In case your browser doesn't display that properly:
When you invoke the expansion mentioned in the question, e.g. with
\catcode`[=1
\catcode`]=2
\catcode`!=6
\def\!!1#2![{!#]#!!2}
\tracingmacros=1
\message{Expansion gives \! x{[y]][z} ok?}
\end
You get the first parameter set to x
, then the second parameter set to [y]
(namely {[y]]
without the surrounding begin-group / end-group tokens), and the [
is consumed, and all this is replaced with the sequence of tokens {
, #
, ]
, !
, [y]
, [
. The log file shows
\!!1#2[->{##]!!#2[
!1<-x
#2<-[y]
(not #1<-x
as the exercise mentions: this is a bug either in TeX or in The TeXbook, not sure which one will DEK will consider it to be). Here, in this \tracingmacros
output:
“Category codes are not shown, but a character of category 6 always appears twice in succession.” — For example, the
##
and!!
that are shown, each representing a single catcode-6 token with char code that of#
and!
respectively.“A parameter token in the replacement text uses the character code of the final parameter in the parameter text.” — This refers to the fact that the output parameter in the first log line (on the right-hand side of the
->
in\!!1#2[->{##]!!#2[
) is shown as#2
rather than!2
. If we have\def\foo#1!2{#1#2}
then we see\foo #1!2->!1!2
, and if we have\def\foo!1#2{#1#2}
then we see\foo !1#2->#1#2
. That is, every parameter token shown in the replacement text by\tracingmacros
uses the last catcode-6 character that was used for a parameter in the parameter-text part of the macro definition. [Edit: This interpretation was pointed out by the OP; removed incorrect explanation that was here.]