Token counter - strange behaviour of math mode
The problem is not math mode but any braced expression where the first two tokens are the same. Take your example of "{22}":
At some point, you looked at everything before {22}
. Then the argument #1
of \TestNext
becomes 22
and therefore \ifx#1\finish
becomes \ifx22\finish
.
Here \ifx 22
is true, so \finish
is expanded, leading to the error.
You can fix this by comparing \CurrentToken
instead of using #1
directly.
You also need a helper macro storing your terminator:
\documentclass{article}
\def\finish{}
\def\finishmarker{\finish}
\newcount\tmp
\def\CountToken[#1,#2]{%
\def\TestedToken{#1}%
\tmp=0
\newcount#2%
\let\TotalOccurrence#2%
\let\next\TestNext\next}
\long\def\TestNext#1{%
\def\CurrentToken{#1}%
\ifx\CurrentToken\finishmarker
\def\next{\TotalOccurrence=\the\tmp\relax}%
\else
\ifx\CurrentToken\TestedToken
\advance\tmp by 1
\fi
\fi
\next}
\begin{document}
%-----------------------
\CountToken[i,\cnti]
Lorem ipsum dolor $\frac{22}{3}$ sit amet,
consectetur adipiscing elit.
\finish
%-----------------------
number of i's: \the\cnti
%-----------------------
\CountToken[f,\cntf]
Lorem ipsum dolor $\frac{22}{3}$ sit amet,
consectetur adipiscing elit.
\finish
%-----------------------
number of f's: \the\cntf
%-----------------------
\CountToken[\something,\cntsomething]
Lorem ipsum dolor $\frac{22}{3}$ sit amet,
consectetur\something adipiscing elit.\something
\finish
%-----------------------
number of \verb|\something|'s: \the\cntsomething
\end{document}
Another problem with the code is that it only counts "outer" tokens, meaning that tokens "hidden" inside of {} are ignored.
You do \def\CurrentToken{#1}
but then do \ifx#1\finish
instead of \ifx\CurrentToken\finish
. Wait! This wouldn't yield true if we're at the end!
Yes, if you do \def\finish{\finish}
. Of course you have to be careful that \finish
doesn't end up in some place where it's expanded.
The \tmp
counter is redundant: you have \TotalOccurrence
available, use it.
\documentclass{article}
\def\finish{\finish}
\def\CountToken[#1,#2]{%
\def\TestedToken{#1}%
\ifdefined#2#2=0 \else\newcount#2\fi
\let\TotalOccurrence#2%
\let\next\TestNext\next
}
\long\def\TestNext#1{%
\def\CurrentToken{#1}%
\ifx\CurrentToken\finish
\let\next\relax
\else
\ifx\CurrentToken\TestedToken
\advance\TotalOccurrence by 1
\fi
\fi
\next
}
\begin{document}
%-----------------------
\CountToken[i,\cnti]
Lorem ipsum dolor $\frac{21}{3}$ sit amet,
consectetur adipiscing elit.
\finish
%-----------------------
number of i's: \the\cnti
%-----------------------
\CountToken[f,\cntf]
Lorem ipsum dolor $\frac{21}{3}$ sit amet,
consectetur adipiscing elit.
\finish
%-----------------------
number of f's: \the\cntf
%-----------------------
\CountToken[\something,\cntsomething]
Lorem ipsum dolor $\frac{21}{3}$ sit amet,
consectetur\something adipiscing elit.\something
\finish
%-----------------------
number of \verb|\something|'s: \the\cntsomething
\end{document}
Just for completeness, here's an expl3
version:
\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\NewDocumentCommand{\CountToken}{mm+m}
{% #1 = token to check, #2 = counter, #3 = text to count in
\int_zero_new:N #2
\tl_map_inline:nn { #3 }
{
\tl_if_eq:nnT { #1 } { ##1 } { \int_incr:N #2 }
}
}
\ExplSyntaxOff
\begin{document}
%-----------------------
\CountToken{i}{\cnti}{
Lorem ipsum dolor $\frac{21}{3}$ sit amet,
consectetur adipiscing elit.
}
%-----------------------
number of i's: \the\cnti
%-----------------------
\CountToken{f}{\cntf}{
Lorem ipsum dolor $\frac{21}{3}$ sit amet,
consectetur adipiscing elit.
}
%-----------------------
number of f's: \the\cntf
%-----------------------
\CountToken{\something}{\cntsomething}{
Lorem ipsum dolor $\frac{21}{3}$ sit amet,
consectetur\something adipiscing elit.\something
}
%-----------------------
number of \verb|\something|'s: \the\cntsomething
\end{document}
A similar problem was solved in https://tex.stackexchange.com/a/525246/4427 (also counting tokens or sets of tokens inside braces).
Let's look at the code in more detail.
The macro \CountToken
is defined to have three mandatory arguments, the last of which can contain also \par
tokens (because of the +
prefix).
The command first sets the counter to zero (argument #2
) or allocates a new one if not yet existing (the effect is similar to \ifdefined#2#2=0 \else\newcount#2\fi
).
Next we map the third argument: a token list can be seen as a list of items (spaces between items are ignored, though), where braced groups count as a single item. Each item is passed to the code specified as the second argument, in this case
\tl_if_eq:nnT { #1 } { ##1 } { \int_incr:N #2 }
Here ##1
stands for the current item in the mapping. The T
stands for True: the third argument is only executed if the two token lists given as the first two arguments are equal.
It's essentially the same code as above, but the details of building the loop and checking equality are already provided by standard functions.