New primitives and grammar for replacing horrible \expandafter

You need to separate out the choice of the : and _ syntax of L3 from the functionality.

The following uses a much smaller expansion library than xparse (just 7 lines of code) but it implements the n (no expansion) x (full \edef expansion) and o (expand once \expandafter) expansion types.

Rather than define variant macro names with :nnx suffixes, just explicitly prefix the use with a \use command that specifies how to expand the arguments (which is also the basis of xparse syntax).

Note how with o, \argb just expands once to bbb {\arga} before \cmd is called but with x, \argb fully expands to bbb {aa}.

\documentclass{article}
\usepackage[T1]{fontenc}

\def\use#1{\xuse#1z}
\def\xuse#1{\csname use#1\endcsname}
\def\usez#1{#1}
\long\def\usen#1z#2#3{\xuse#1z{#2{#3}}}
\long\def\useo#1z#2#3{\expandafter\useohelp\expandafter{#3}{#1}{#2}}
\long\def\useohelp#1#2#3{\xuse#2z{#3{#1}}}
\protected\long\def\usex#1z#2#3{\edef\usetmp{#3}\expandafter\useohelp\expandafter{\usetmp}{#1}{#2}}

\begin{document}



\def\cmd#1#2{{\ttfamily\par\def\a{#1}\#1:\meaning\a \quad\#2:\def\a{#2}\meaning\a\par}}
\def\arga{aa}
\def\argb{bbb {\arga}}



        \cmd{\arga}{\argb}  % normal
\use{nn}\cmd{\arga}{\argb}  % normal
\use{on}\cmd{\arga}{\argb}  % expand #1 once 
\use{no}\cmd{\arga}{\argb}  % expand #2 once
\use{oo}\cmd{\arga}{\argb}  % expand #1 and #2 once
\use{xx}\cmd{\arga}{\argb}  % fully (\edef) expand #1 and #2

\end{document}

As noted in comments it isn't at all clear whether you intended your () syntax to correspond to o or x but either way It is really better handled at the macro layer rather than a change of the engine expansion logic.

It's easy with expl3:

\input expl3-generic

\def\arga{what}
\def\argb{how}

\ExplSyntaxOn
\cs_new:Npx \zh_cmda: { \arga\argb }

\cs_new:Npn \zh_cmdb:nn #1 #2
 {
  \str_if_eq:nnTF { #1 } { what }
   { \msg_term:n{\#1~is~`what'} }
   { \msg_term:n{\#1~isn't~`what'} }
  \str_if_eq:nnTF { #2 } { how }
   { \msg_term:n{\#2~is~`how'} }
   { \msg_term:n{\#2~isn't~`how'} }
 }
\cs_generate_variant:Nn \zh_cmdb:nn { no }
\cs_generate_variant:Nn \zh_cmdb:nn { on }
\cs_generate_variant:Nn \zh_cmdb:nn { oo }

\msg_term:n
 {
  The ~ replacement ~ text ~ of ~ \exp_not:N \zh_cmda: ~ is ~
  `\token_get_replacement_spec:N \zh_cmda:'
 }
\zh_cmdb:nn { \arga } { \argb }
\zh_cmdb:no { \arga } { \argb }
\zh_cmdb:on { \arga } { \argb }
\zh_cmdb:oo { \arga } { \argb }

\bye

The terminal session will show

*************************************************
* The replacement text of \zh_cmda: is `whathow'
*************************************************
*************************************************
* #1 isn't `what'
*************************************************
*************************************************
* #2 isn't `how'
*************************************************
*************************************************
* #1 isn't `what'
*************************************************
*************************************************
* #2 is `how'
*************************************************
*************************************************
* #1 is `what'
*************************************************
*************************************************
* #2 isn't `how'
*************************************************
*************************************************
* #1 is `what'
*************************************************
*************************************************
* #2 is `how'
*************************************************

Note that the result is what's expected: the argument are expanded once when o is used, making the \str_if_eq:nnTF test succeed.

Do you need to use expl3 syntax in your code? Not at all! Just define a suitable wrapper. With \exp_args_generate:n you can define all the variants you need.

\input expl3-generic

\ExplSyntaxOn

\cs_new:Npn \use #1 { \cs_if_exist_use:cF { exp_args:N#1 } { \ERROR } }
\exp_args_generate:n { nn, on, ee }

\ExplSyntaxOff

\def\cmd#1#2{{\tt\par\def\a{#1}\#1:\meaning\a \quad\#2:\def\a{#2}\meaning\a\par}}
\def\arga{aa}
\def\argb{bbb {\arga}}



        \cmd{\arga}{\argb}  % normal
\use{nn}\cmd{\arga}{\argb}  % normal
\use{on}\cmd{\arga}{\argb}  % expand #1 once 
\use{no}\cmd{\arga}{\argb}  % expand #2 once
\use{oo}\cmd{\arga}{\argb}  % expand #1 and #2 once
\use{xx}\cmd{\arga}{\argb}  % fully (\edef) expand #1 and #2
\use{ee}\cmd{\arga}{\argb}  % fully (\edef) expand #1 and #2

\bye

The last is the new e-type expansion, which is similar to x-expansion, but is fully expandable itself. If a variant is not defined, you get an error message. Of course \use{nn} is the same as nothing.

The advantage is that you can define variants for as many arguments as you need. So if \macro has three arguments, you can use

\use{noe}\macro{<first>}{<second>}{<third>}

and accomplish the desired expansions before \macro is actually called, provided you add noe to the list of known variants.