Symbols with optional parameter in glossaries with \newglossary

Method 1: Specific Glossary

\defglsentryfmt has an optional argument that indicates which glossary this format should govern. If omitted, main is assumed, so you also need to do the same for the new glossary:

\defglsentryfmt[symbolslist]{%
  \let\orgglsarg\glsarg
  \ifdefempty\glsinsert
  {}%
  {%
    \let\glsarg\glsinsert
    \let\glsinsert\relax
  }%
  \glsgenentryfmt
  \let\glsarg\orgglsarg
}

Method 2: All Glossaries

Alternatively, you can just redefine \glsentryfmt to apply to all glossaries. (Individual glossaries may be overridden using \defglsentryfmt, which \newacronym does automatically.)

\documentclass{scrbook}
%
\usepackage[acronym,toc]{glossaries}
%this line is new
\newglossary[slg]{symbolslist}{syi}{syg}{List of Symbols}
\makeglossaries
\glsnoexpandfields

\newcommand*{\glsarg}{i}

\newglossaryentry{BetragVektor}{
     name=\ensuremath{|\overline{u_i}|},
     text=|\overline{u_{\glsarg}}|,
    description={ABC}}
\newglossaryentry{s:tP}{
    name=\ensuremath{T_p\mathcal M},
    text=T_\glsarg\mathcal M,
    description={},
    type=symbolslist %in own list
}
% modify the entry's format

\renewcommand*{\glsentryfmt}{%
  \let\orgglsarg\glsarg
  \ifdefempty\glsinsert
  {}%
  {%
    \let\glsarg\glsinsert
    \let\glsinsert\relax
  }%
  \glsgenentryfmt
  \let\glsarg\orgglsarg
}

\begin{document}

$\gls{BetragVektor}[1]$

$\gls{BetragVektor}$

$\gls{BetragVektor}[1]$

$\gls{BetragVektor}[2]$

$\gls{BetragVektor}[]$

$\gls{s:tP}[1]$

$\gls{s:tP}$

$\gls{s:tP}[1]$

$\gls{s:tP}[2]$

$\gls{s:tP}[]$

\printglossaries
\end{document}

This produces:

image of result

Method 3: Specific Category (glossaries-extra)

If you want to use the extension package glossaries-extra, this is a better method:

\documentclass{scrbook}
%
\usepackage[abbreviations]{glossaries-extra}

\newglossary[slg]{symbolslist}{syi}{syg}{List of Symbols}
\makeglossaries
\glsnoexpandfields

\newcommand*{\glsdefaultarg}{i}
\newcommand*{\glsarg}{\glsdefaultarg}

\newglossaryentry{BetragVektor}{
    category=arg,% requires an argument
     name=\ensuremath{|\overline{u_i}|},
     text=|\overline{u_{\glsarg}}|,
    description={ABC}}
\newglossaryentry{s:tP}{
    category=arg,% requires an argument
    name=\ensuremath{T_p\mathcal M},
    text=T_\glsarg\mathcal M,
    description={},
    type=symbolslist %in own list
}
% modify the entry's format

\preto\glsentryfmt{%
  \glsifcategory{\glslabel}{arg}% if category set to "arg"
  {%
    \ifdefempty\glsinsert
    {\let\glsarg\glsdefaultarg}%
    {%
      \let\glsarg\glsinsert
      \let\glsinsert\empty
    }%
  }%
  {}%
}

\begin{document}

$\gls{BetragVektor}[1]$

$\gls{BetragVektor}$

$\gls{BetragVektor}[1]$

$\gls{BetragVektor}[2]$

$\gls{BetragVektor}[]$

$\gls{s:tP}[1]$

$\gls{s:tP}$

$\gls{s:tP}[1]$

$\gls{s:tP}[2]$

$\gls{s:tP}[]$

\printglossaries
\end{document}

Now you just need to tag which entries require an argument by using category=arg in the entry definition. You can also store the default argument, if it should be different for each entry. For example:

\documentclass{scrbook}
%
\usepackage[abbreviations]{glossaries-extra}

\newglossary[slg]{symbolslist}{syi}{syg}{List of Symbols}
\makeglossaries
\glsnoexpandfields

\newcommand*{\glsarg}{}

\newglossaryentry{BetragVektor}{
    category=arg,% requires an argument
    user1={i},
     name=\ensuremath{|\overline{u_i}|},
     text=|\overline{u_{\glsarg}}|,
    description={ABC}}
\newglossaryentry{s:tP}{
    category=arg,% requires an argument
    user1={p},
    name=\ensuremath{T_p\mathcal M},
    text=T_\glsarg\mathcal M,
    description={},
    type=symbolslist %in own list
}
% modify the entry's format

\preto\glsentryfmt{%
  \glsifcategory{\glslabel}{arg}% if category set to "arg"
  {%
    \ifdefempty\glsinsert
    {\glsfieldfetch{\glslabel}{useri}{\glsarg}}%
    {%
      \let\glsarg\glsinsert
      \let\glsinsert\empty
    }%
  }%
  {}%
}

\begin{document}

$\gls{BetragVektor}[1]$

$\gls{BetragVektor}$

$\gls{BetragVektor}[1]$

$\gls{BetragVektor}[2]$

$\gls{BetragVektor}[]$

$\gls{s:tP}[1]$

$\gls{s:tP}$

$\gls{s:tP}[1]$

$\gls{s:tP}[2]$

$\gls{s:tP}[]$

\printglossaries
\end{document}

Method 4: Specific Entry (glossaries-extra)

Here's another method: use \glsxtrfmt for the cases where a parameter is required. This deviates from the concept of an optional parameter, but it's worth considering as it doesn't alter the normal display format.

A designated field (by default user1, but may be changed by redefining \GlsXtrFmtField) stores the name (without the leading backslash) of a command that takes a single mandatory argument that's associated with the formatting of the given entry. Then \glsxtrfmt[options]{label}{parameter} applies that command to parameter (encapsulated by \glslink to index and, if enabled, hyperlink to the glossary).

For example, define:

\newcommand{\vectorlength}[1]{|\overline{u_{#1}}|}
\newcommand{\tP}[1]{T_#1\mathcal{M}}

Then the entries can be defined as:

\newglossaryentry{BetragVektor}{
 user1={vectorlength},
 name=\ensuremath{\vectorlength{i}},
 description={ABC}}

\newglossaryentry{s:tP}{
 user1={tP},
 name=\ensuremath{\tP{p}},
 description={},
 type=symbolslist
}

So now \gls{BetragVektor} uses the value of the first/text field (as normal, which here is obtained from the name field) but \glsxtrfmt{BetragVektor}{1} is essentially like \glslink{BetragVektor}{\vectorlength{1}}

Modified MWE:

\documentclass{scrbook}

\usepackage[automake]{glossaries-extra}

\newglossary[slg]{symbolslist}{syi}{syg}{List of Symbols}

\makeglossaries
\glsnoexpandfields

\newcommand{\BetragVektor}[1]{|\overline{u_{#1}}|}
\newcommand{\tP}[1]{T_#1\mathcal{M}}

\newglossaryentry{BetragVektor}{
 user1={BetragVektor},
 name=\ensuremath{\BetragVektor{i}},
 description={ABC}}

\newglossaryentry{s:tP}{
 user1={tP},
 name=\ensuremath{\tP{p}},
 description={},
 type=symbolslist %in own list
}

\begin{document}

$\glsxtrfmt{BetragVektor}{1}$

$\gls{BetragVektor}$

$\glsxtrfmt{BetragVektor}{1}$

$\glsxtrfmt{BetragVektor}{2}$

$\gls{BetragVektor}$

$\glsxtrfmt{s:tP}{1}$

$\gls{s:tP}$

$\glsxtrfmt{s:tP}{1}$

$\glsxtrfmt{s:tP}{2}$

$\gls{s:tP}$

\printglossaries

\end{document}

There are some more examples in Section 3.1 "Functions" of glossaries-extra and bib2gls: An Introductory Guide (that section is general to glossaries-extra regardless of whether or not you're using bib2gls).


¹ It's a little more complicated: \glsxtrfmt[options]{label}{text} effectively does \glslink[default options,options]{label}{\glsxtrfmtdisplay{csname}{text}{}} where csname is the control sequence name obtained from the designated field. (If the field is unset csname defaults to @firstofone.)

The starred version \glsxtrfmt*[options]{label}{text}[insert] is effectively \glslink[default options,options]{label}{\glsxtrfmtdisplay{csname}{text}{insert}}.