Is it possible to use subcaption inside a minipage?

Yes, it is possible, just use \captionsetup{type=figure} inside your minipage to tell the (sub)caption package to treat this minipage as figure, at least from captions point of view. Afterwards use ordinary captioning commands like \caption and \subcaptionbox, e.g.:

\documentclass{article}
\usepackage[demo]{graphicx}
\usepackage{caption}
\usepackage{subcaption}
\captionsetup[subfigure]{labelformat = parens, labelsep = space, font = small}
\begin{document}
  \begin{minipage}{\linewidth}
    \captionsetup{type=figure} % -- This line added
    \centering
    \subcaptionbox{A frictionless ball in a semi-circle. \label{semicircle}}
      [.5\linewidth]{\includegraphics{semicircle}}
    \subcaptionbox{Close up of the forces on the frictionless ball.
      \label{exam2triangle}}
      [.5\linewidth]{\includegraphics{exam2triangle}}
    \caption[Frictionless Ball on a Semi-Circle]{A frictionless ball
      oscillating on a semi-circle.}
    \label{exam2}
  \end{minipage}
\end{document}

Normally I wouldn't suggest this, but this seems to be a case in which the H specifier (from, for example, the float package) might be handy to suppress flotation:

\documentclass{article}
\usepackage{graphicx}
\usepackage{caption}
\usepackage[most]{tcolorbox}
\usepackage{subcaption}
\usepackage{float}
\captionsetup[subfigure]{labelformat = parens, labelsep = space, font = small}

\begin{document}

\begin{tcolorbox}
\begin{figure}[H]
  \begin{minipage}{\linewidth}
    \centering
    \subcaptionbox{A frictionless ball in a semi-circle.}
    {\includegraphics[height=3cm]{example-image-a}}\quad
    \subcaptionbox{Close up of the forces on the frictionless ball.}
     {\includegraphics[height=2cm]{example-image-b}}
    \caption[Frictionless Ball on a Semi-Circle]{A frictionless ball
      oscillating on a semi-circle.}
    \label{exam2}
  \end{minipage}
\end{figure}
\end{tcolorbox}

\end{document}