Why does this regexpatch command only work once, not twice?

Using l3regex directly instead of \regexpatchcmd it works:

\documentclass{article}
\usepackage{xparse}
\ExplSyntaxOn
\NewDocumentCommand \addinnertoouter { m }
  {
    \regex_replace_once:nnNTF
      { \c{outercmd} \{ (.*) \} \Z}
      { \c{outercmd} \cB\{ \1 \c{inner} \cB\{ #1 \cE\} \cE\} }
      \test
      { }
      { error }
  }
\ExplSyntaxOff
\begin{document}
\ttfamily

\def\test{\outercmd{\inner{abc}}}
\meaning\test

\addinnertoouter{def}
\meaning\test

\addinnertoouter{ghi}
\meaning\test
\end{document}

(\outer is a (very special) TeX primitive, so it's better if you use something else.)

The issue is that l3regex apparently makes the added def tokens catcode 12 (looks like a bug to me; I'll summon a wizard to check), and when \regexpatchcmd tries to retokenize them they become catcode 11, and \regexpatchcmd thinks (correctly) that the macro cannot be patched. If you don't mind that the inserted tokens be catcode 12 (in this case it probably won't matter), then \regex_replace_once:nnNTF will suit your needs.

---

Beware that the argument of your \addinnertoouter is still a regular expression, so if you do \addinnertoouter{\textit{jkl}} the output will not be what you expect. You can use the \u feature of l3regex to add an arbitrary token list (without catcode changes, so it will also work with \regexpatchcmd):

\documentclass{article}
\usepackage{regexpatch}
\ExplSyntaxOn
\tl_new:N \l_bers_tmpa_tl
\NewDocumentCommand \addinnertoouter { m }
  {
    \tl_set:Nn \l_bers_tmpa_tl {#1}
    \regexpatchcmd
      \test
      { \c{outercmd} \{ (.*) \} \Z}
      { \c{outercmd} \cB\{ \1 \c{inner} \cB\{ \u{l_bers_tmpa_tl} \cE\} \cE\} }
      { }
      { error }
  }
\ExplSyntaxOff
\begin{document}
\ttfamily

\def\test{\outercmd{\inner{abc}}}
\meaning\test

\addinnertoouter{def}
\meaning\test

\addinnertoouter{ghi}
\meaning\test

\addinnertoouter{\textit{jkl}}
\meaning\test
\end{document}

Phelype Oleinik already described the problem with catcode changes in \regexpatchcmd, so I'm skipping the explanation part here ... The problem can also be fixed by replacing the line

{ \c{outer} \cB\{ \1 \c{inner} \cB\{ #1 \cE\} \cE\} }

by

{ \c{outer} \cB\{ \1 \c{inner} \cB\{ \cL(#1) \cE\} \cE\} }

in your original code, correctly giving

You can use the \u feature of l3regex:

\documentclass{article}
\usepackage{regexpatch}
\newcommand{\addinnertoouter}[1]{%
  \def\berstemp{#1}%
  \regexpatchcmd{\test}
    { \c{outer} \{ (.*) \} \Z}
    { \c{outer} \cB\{ \1 \c{inner} \cB\{ \u{berstemp} \cE\} \cE\} }
    {}{error}
}
\begin{document}
\ttfamily

\def\test{\outer{\inner{abc}}}
\meaning\test

\addinnertoouter{def}
\meaning\test

\addinnertoouter{ghi}
\meaning\test

\end{document}