Using a loop in \halign's preamble

As Henri Menke wrote in a comment, your preamble has to be expandable (and every token that should be expanded has to be preceded by \span). So how do we build a fully expandable loop only using plain TeX? We use \romannumeral. If \count1 is some integer, then we can use \the\count1 000 to get 1000*\count1. This is expressed as a roman numeral by exactly \count1 times the letter m. \romannumeral is expandable, so we can use this in our preamble. Then our macro is repeated, consuming one m at every iteration, until the ms run out:

\count1=4
\def\subpre#1{
  \span\if m#1
    & ##
    \span\expandafter
    \span\expandafter\subpre
  \fi
}
\def\gobble#1{}
\def\preamb{
  ##
  \span\expandafter
  \span\expandafter\expandafter
  \expandafter\span\expandafter\expandafter\expandafter\subpre
  \expandafter\gobble\romannumeral\the\count1 000x
  \cr
}
\halign{\span\preamb
  first& line&val&x \cr
}
\bye

To understand all the \expandafters in \preamb, try to think as if you were TeX: In \preamb, first we add ## to the preamble. Then there is \span, so the next token is expanded. It is \expandafter, so we skip one token and expand the next, which (again) is \expandafter. This repeats 6 times, expanding the first, second, fourth, fifth, seventh and eighth \expandafter in \preamb until finally expanding \romannumeral\the\count1 000x. This expands to \romannumeral4000x and therefore mmmmx. Without the \expandafters and \romannumeral expanded, we are left with the tokens

\span\expandafter
\span\expandafter\subpre\gobble mmmmx

Again, we have \span, so we expand this. The remaining \expandafters end up expanding \gobble which gobbles the first m. This is the equivalent to \advance \columns by -1 in your original code. So what remains is

\span\subpre mmmx

which finally starts expanding \subpre.

On the other hand, it is often much easier to not restrict yourself to expandable constructs. Instead you can construct the preamble first (e.g.in a token list), and only insert the finished preamble into \halign: (based on David Carlisle's answer to your original question)

\newcount\columns
\newtoks\preambtoks
\def\myhalign{%
  \afterassignment\xmyhalign
  \columns=%
}
\def\xmyhalign{\afterassignment\xxmyhalign\let\tmp=}
\def\subpre{
  \preambtoks\expandafter{\the\preambtoks & ##}
  \advance\columns by -1 \ifnum\columns>0 \subpre \fi
}
\def\xxmyhalign{%
  \preambtoks{##}%
  \advance \columns by -1
  \ifnum\columns>0 \subpre \fi
  \halign\bgroup\span\the\preambtoks\cr
}

\myhalign3{
first line & second column \cr
1& 2&3 \cr
}
\bye

The primitive \span in the context of a \halign preamble, performs a single expansion step.

You need some more sophisticated code if you want to allocate a variable number of columns.

Here I define \preamb to take as argument a column template (in the example, \hfil#\hfil); this template is repeated as many times as the current value of \columns (assumed to be positive).

The trick is to use \span to expand \romannumeral-`@, that does recursive expansion until finding an unexpandable token.

The \subpre macro is called \columns-1 times and each time the third argument contains the preamble-so-far augmented by a further column.

% syntactic sugar
\long\def\firstoftwo#1#2{#1}
\long\def\secondoftwo#1#2{#2}

% the code

\newcount\columns

\def\preamb#1{#1\span\romannumeral-`@\subpre\columns{#1}{}}

\def\subpre#1#2#3{%
  \ifnum\numexpr#1-1=0
    \expandafter\firstoftwo
  \else
    \expandafter\secondoftwo
  \fi
  {#3\cr}%
  {\expandafter\subpre\expandafter{\the\numexpr#1-1}{#2}{#3&#2}}%
}

% the test

\tabskip=10pt
\columns=4
\halign{\span\preamb{\hfil#\hfil}%
  1&2&3&4\cr
  AAA&BB&CCCC&DDDD\cr
  u&v&w&x\cr
}

\bye