Manually controlled size of delimiters in macros

The package mathtools already provides the necessary features; you just have to add the possibility of using numbers instead of the commands \big and siblings.

\documentclass{article}
\usepackage{mathtools}

\DeclarePairedDelimiterX{\innset}[2]{\lbrace}{\rbrace}{%
  #1\;\delimsize\vert\;#2}

\newcommand{\set}[3][0]{%
  \ifcase#1\relax
    \innset{#2}{#3}\or         % 0
    \innset[\big]{#2}{#3}\or   % 1
    \innset[\Big]{#2}{#3}\or   % 2
    \innset[\bigg]{#2}{#3}\or  % 3
    \innset[\Bigg]{#2}{#3}     % 4
  \else
    \innset*{#2}{#3}
  \fi}

\begin{document}
\begin{gather*}
\set{A}{\frac{B}{C}}\\
\set[0]{A}{\frac{B}{C}}\\
\set[1]{A}{\frac{B}{C}}\\
\set[2]{A}{\frac{B}{C}}\\
\set[3]{A}{\frac{B}{C}}\\
\set[4]{A}{\frac{B}{C}}\\
\set[-1]{A}{\frac{B}{C}}
\end{gather*}
\end{document}

With a number not in the range 0–4, the version with \left, \middle and \right would be used.

If you are prepared to take something like Big as an argument then a simple solution to your request is to use \csname ...\endcsname to generate command names as needed, e.g.,

\newcommand\set[3]{\csname #3l\endcsname\{#1%
                   \,\csname#3\endcsname\vert\,%
                   #2\csname #3r\endcsname\}}

If you want to generate the needed commands via a numerical argument consider using \ifcase.