High-order recursive macro to define several similar macro at once does infinite loop

You need to expand away the \fi

\RequirePackage{amsmath}

\makeatletter

\newcommand{\defset}[1]{\expandafter\newcommand\csname #1\endcsname{\mathbb{#1}}}
\def\defsets#1{\defnext\defset #1@}
\def\defnext#1#2{\ifx @#2%
    \expandafter\@gobble
   \else
    #1{#2}%
    \expandafter\@firstofone
   \fi    
    {\defnext#1}}
\defsets{RDNZQ{Zzz}C}


\show\Z
\show\Zzz


\newcommand*{\defop}[1]{\expandafter\DeclareMathOperator\csname #1\endcsname{#1}}

\defnext\defop {Vect}{Spec}@

{\let\protect\show\Vect}

\stop

Produces

LaTeX2e <2016/03/31>
Babel <3.9q> and hyphenation patterns for 81 language(s) loaded.
> \Z=\long macro:
->\mathbb {Z}.
l.15 \show\Z

? 
> \Zzz=\long macro:
->\mathbb {Zzz}.
l.16 \show\Zzz

? 
> \Vect =\long macro:
->\qopname \newmcodes@ o{Vect}.
\Vect ->\protect \Vect  

l.23 {\let\protect\show\Vect
                            }
? 

Note you need to re-brace #2 as {#2} to support multiple token arguments.

You cannot skip more than one token by one \expandafter. There are more solutions of your problem. For example, you can set the used def-method by \let first:

\def\defset#1{\expandafter\def\csname#1\endcsname{\mathbb{#1}}}

\def\defsets#1{\let\defnextA=\defset \defnext #1@}
\def\defnext#1{\ifx @#1\else
    \defnextA{#1}%
    \expandafter\defnext
  \fi}

\defsets{RDNZQC}

Second: you can repeat the def-method as #1 but with one more \expandafter:

\def\defset#1{\expandafter\def\csname#1\endcsname{\mathbb{#1}}}

\def\defsets#1{\defnext\defset #1@}
\def\defnext#1#2{\ifx @#2\else
    #1{#2}%
    \expandafter\defnext\expandafter#1%
  \fi}

\defsets{RDNZQC}

The problem is of course in \expandafter\defnext#1 that tries to expand whatever is replaced for #1 instead of the intended \fi.

If you hate code repetition, you'll surely like this, where \defsets and \defops are basically one-liners.

\documentclass{article}
\usepackage{amsmath}
\usepackage{xparse}

\ExplSyntaxOn
\NewDocumentCommand{\defsets}{m}
 {
  \tl_map_inline:nn { #1 }
   {
    \cs_new_protected:cpn { ##1 } { \mathbb{##1} }
   }
 }
\NewDocumentCommand{\defops}{m}
 {
  \clist_map_inline:nn { #1 } 
   {
    \galex_declaremathoperator:cn { ##1 } { ##1 }
   }
 }
\cs_set_eq:NN \galex_declaremathoperator:Nn \DeclareMathOperator
\cs_generate_variant:Nn \galex_declaremathoperator:Nn { c }
\ExplSyntaxOff

\defsets{RDNZQC}
\defops{Vect,Spec}

\begin{document}

\texttt{\meaning\R}

\texttt{\meaning\Q}

\texttt{\meaning\Spec}

\texttt{\expandafter\meaning\csname Spec \endcsname}

\end{document}

Only one \expandafter to show that \Spec does the right thing. None in the coding part.

I could have used

\exp_args:Nc \DeclareMathOperator { ##1 } { ##1 }

instead of defining \galex_declaremathoperator:Nn and a variant thereof. But this way all is cleaner and in line with recommendations.

The same is obtained with \@tfor:

\makeatletter
%\let\@xp\expandafter % already done by amsmath
\newcommand{\defsets}[1]{%
  \@tfor\next:=#1\do{%
    \@xp\newcommand\csname\next\@xp\endcsname\@xp{\@xp\mathbb\@xp{\next}}%
  }%
}
\newcommand{\defops}[1]{%
  \@tfor\next:=#1\do{%
    \@xp\DeclareMathOperator\csname\next\@xp\endcsname\@xp{\next}%
  }%
}
\makeatother

\defsets{RDNZQC}
\defops{{Vect}{Spec}}

The \@tfor function is essentially the abstract version of the \slowRomannumeral trick in your variant: it consumes one item at a time from the token list passed between := and \do, doing \def\next{<item>} (\next can be any token). The main difference is exactly this: we need to expand \next if we don't want that it is incorporated in the definition of \N and so on. Therefore the long chain of \expandafter's is needed. An item in this context is a single token or a braced group.

The \tl_map_inline:nn method is much better, because the current item is available “literally” as #1 (so in the body of a definition it must be called ##1).