Appending a character to a token list using its charcode
\lowercase is a good way to do this, with any TeX.
\def\prepend#1#2{% toks, charcode
\begingroup
\lccode`9=#2\relax
\lowercase{%
\edef\0{\endgroup
#1={9\the#1}}%
\0}}
Assumes the toks register is not \0.
You can use \char_generate:nn { <charcode> } { <catcode> }:
\input expl3-generic
\ExplSyntaxOn
\cs_new_eq:NN \toks_use:N \tex_the:D
\cs_new_protected:Npn \prepend #1 #2
{
\if:w \exp_not:N #1 #1
\use:x { #1 = { \char_generate:nn {#2} { 12 } \toks_use:N #1 } }
\else:
\tl_put_left:Nx #1 { \char_generate:nn {#2} { 12 } }
\fi:
}
\ExplSyntaxOff
\newtoks\test
\test={bc}
\prepend\test{97}
\showthe\test % should print abc
\def\test{bc}
\prepend\test{97}
\show\test % should print abc
\bye
The terminal output will be:
> abc.
l.16 \showthe\test
% should print abc
?
> \test=macro:
->abc.
l.21 \show\test
% should print abc again
?
How \char_generate:nn generates the characters depends on which engine is in use. In LuaTeX it uses tex.cprint(<catcode>, utf8_char(<charcode>)), in a similar way to Henri's answer, but with a possible <catcode> setting. In XeTeX it uses \Ucharcat <charcode> <catcode>.
In the other engines supported by expl3 (pdftex, ε-pTeX, and ε-upTeX) there is no way to actually generate characters in expansion-only contexts (a key feature of \char_generate:nn), so expl3 pre-generates these characters using the same approach as in egreg's answer, and then \char_generate:nn just uses the characters when asked for.
As in egreg's answer, you can't generate characters of some catcodes—namely 0, 5, 9, 14, and 15—because they don't produce tokens (they disappear when TeX is scanning the input, so they don't exist at the macro-expansion level). Also, the expl3 implementation does not allow the generation of space characters for consistency across engines, because the Lua version doesn't allow so. However since you want a Knuth TeX version, space chars are also allowed.
The code below is an adaptation of the expl3 code for \char_generate:nn modified to work in Knuth TeX. The code is basically the same, except that a few more complications are necessary mainly due to the lack of \unexpanded, which allows you to have single parameter tokens in a macro, and allows you to easily append stuff to a macro without the need of a toks register. Other than that, it's the same thing.
The code first defines a temporary toks register which contains the null character (^^@) with the different possible catcodes, separated by \or:
\or ^^@% 1
\or ^^@% 2
\or ^^@% 3
\or ^^@% 4
\or % 5 Invalid
\or ^^@^^@% 6 Has to be doubled for a later `\def`
\or ^^@% 7
\or ^^@% 8
\or % 9 Invalid
\or ^^@% 10
\or ^^@% 11
\or ^^@% 12
\or ^^@% 13
then it loops through all 256 character codes and sets the \lccode of the null character to #1 and then uses the \lowercase trick in egreg's answer:
\begingroup
\lccode0=#1
\lccode32=#1
\edef\x{\endgroup
\gdef\expandafter\noexpand
\csname c__char_\romannumeral#1_tl\endcsname{\the\tmptoks}}%
\lowercase\expandafter{\x}
which for a character code, say 97, results in:
\gdef\c__char_xcvii_tl{\or a\or a\or a\or a\or \or aa\or a\or a\or \or a\or a\or a\or a}
then given a character code <charcode> you can access that token list with \csname c__char_\romannumeral<charcode>_tl\endcsname, and then with `\ifcase\fi you have the requested character.
The \chargenerate macro first checks (in \generateaux) if the arguments are in a valid range (catcode between 1 and 13, except 5 and 9, and charcode between 0 and 255, though with Knuth TeX you might need to change that to 127), and then calls \generateauxi with the arguments, which then uses the \ifcase test above (with a few more bits and pieces for expansion control) to leave the requested character.
Running the code below with tex I get:
% Auxiliaries
\long\def\gobbletoqstop#1\qstop{}
\long\def\firstofone#1{#1}
\chardef\expend=0
% Expandable error message
\begingroup
\long\xdef\expandableerror#1{%
\noexpand\expandafter\noexpand\expandafter\noexpand\expandafter
\noexpand\gobbletoqstop\noexpand\firstofone
{\csname Error! \endcsname#1}\noexpand\qstop}
\endgroup
% Append stuff to a toks register
\def\toksputright#1{%
\begingroup
\def\toksputtoks{#1}%
\afterassignment\toksputrightaux
\toks0=}
\def\toksputrightaux{%
\edef\x{\endgroup
\toksputtoks={\the\toksputtoks\the\toks0}}%
\x}
% Set up constant token lists
\newtoks\tmptoks
\begingroup
\tmptoks{ \noexpand\or}%
\catcode0=1
\toksputright\tmptoks{^^@\iffalse}}%
\catcode0=2
\toksputright\tmptoks{{\fi\noexpand\or^^@}%
\begingroup
\def\noop{}%
\edef\x{\expandafter\noop\the\tmptoks}%
\expandafter\endgroup
\expandafter\tmptoks\expandafter{\x}%
\catcode0=3 \toksputright\tmptoks{\or^^@}%
\catcode0=4 \toksputright\tmptoks{\or^^@}%
\catcode0=5 \toksputright\tmptoks{\or}%
\catcode0=6 \toksputright\tmptoks{\or^^@^^@}%
\catcode0=7 \toksputright\tmptoks{\or^^@}%
\catcode0=8 \toksputright\tmptoks{\or^^@}%
\catcode0=9 \toksputright\tmptoks{\or}%
\catcode0=10 \toksputright\tmptoks\expandafter{\firstofone{\or}^^@}%
\catcode0=11 \toksputright\tmptoks{\or ^^@}%
\catcode0=12 \toksputright\tmptoks{\or^^@}%
\catcode0=13 \toksputright\tmptoks{\or^^@}%
\def\chartmp#1;{%
\begingroup
\lccode0=#1
\lccode32=#1
\edef\x{\endgroup
\gdef\expandafter\noexpand
\csname c__chargen_\romannumeral#1_tl\endcsname{\the\tmptoks}}%
\lowercase\expandafter{\x}}%
\let^^L\relax
\catcode`^^L=12
\count0=0
\loop
\expandafter\chartmp\number\count0;
\advance\count0 by 1
\ifnum\count0<256 \repeat
\endgroup
% Main definition
\def\chargenerate#1#2{%
\romannumeral\expandafter\generateaux
\number#1\expandafter;\number#2;}
% Check for invalid input
\def\generateaux#1;#2;{%
\ifnum0%
\ifnum#1=0 1\fi
\ifnum#2=10 1\fi
=11
\expandableerror{Cannot generate null char as a space.}%
\else
\ifodd0%
\ifnum#2< 1 1\fi
\ifnum#2= 5 1\fi
\ifnum#2= 9 1\fi
\ifnum#2>13 1\fi\space
\expandableerror{Invalid catcode for char generation.}%
\else
\ifodd0%
\ifnum#1< 0 1\fi
\ifnum#1>"FF 1\fi\space
\expandableerror{Charcode requested out of engine range.}%
\else
\generateauxi{#1}{#2}%
\fi
\fi
\fi
\expend}
% Actual char generation
\def\generateauxi#1#2#3\expend{%
#3%
\iffalse{\fi
\expandafter\expandafter
\expandafter\expend
\expandafter\expandafter
\ifcase#2%
\csname c__chargen_\romannumeral#1_tl\endcsname
\or}
\fi}
% Testing
\def\empty{}
\begingroup
\lccode`\~=`a
\lowercase{\endgroup
\gdef ~{\ active character a}%
}
\def\test#1{%
\edef\x{%
\ifnum#1=2 {\iffalse}\fi\space\noexpand\meaning\fi % add { if a is a }
\chargenerate{97}{#1}%
\ifnum#1=6 \chargenerate{97}{#1}\fi% add another # if a is a #
\ifnum#1=1 \iffalse{\fi\space\noexpand\meaning}\fi % if a is a {, add a }
}%
\ifx\x\empty
#1: ERROR
\else
#1: \expandafter\meaning\x
\fi\par}
\tt\scrollmode
\count2=0
\loop
\test{\the\count2 }%
\advance\count2 by 1
\ifnum\count2<16
\repeat
\bye
\newtoks\test
\def\prepend#1#2{%
\ifcase\catcode#2\relax
% 0, do nothing
\or
% 1, do nothing
\or
% 2, do nothing
\or
\prependaux#1{#2}{$}% 3
\or
\prependaux#1{#2}{&}% 4
\or
% 5, do nothing
\or
\prependaux#1{#2}{##}% 6
\or
\prependaux#1{#2}{^}% 7
\or
\prependaux#1{#2}{_}% 8
\or
% 9, do nothing
\or
\prependaux#1{#2}{ }% 10
\or
\prependaux#1{#2}{a}% 11
\or
\prependaux#1{#2}{?}% 12
\or
\prependaux#1{#2}{~}% 13
% 14 or 15, do nothing
\fi
}
\def\prependaux#1#2#3{%
\begingroup\lccode`#3=#2\relax
\lowercase{\endgroup\toks0={#3}}%
#1\expandafter{\the\toks\expandafter0\the#1}%
}
\test={bc}
\prepend\test{97}
\message{\number`?}
\catcode`?=3
\prepend\test{63}
\the\test$
\prepend\test{`\#}
\showthe\test
\bye
You can't add characters with category codes 0, 1, 2, 5, 9, 14 or 15.
As you see, I prepended a “strange” category code 3 character and the code \the\test$ prints a math formula.
Restriction: #1 cannot be \toks0.