Tikz: Rotating a circle

See if

\documentclass[border=1cm]{standalone}
\usepackage{tikz}
\begin{document}

\begin{tikzpicture}
\def\iangle{atan(4/5)}
\coordinate (O) at (0, 0);
\coordinate (A) at (5, 4);
\draw (O)--coordinate (mid) (A);
%
    \node[draw, circle, minimum size=2cm, rotate=\iangle, yshift=1cm] at (mid) {};
\end{tikzpicture}
\end{document}

is what you looking for :)

If there is no content (eg. text) that should be rotated, you can use a calculated border anchor to position the circle node. Then there is no need to know the size of this node.

\documentclass[border=1cm]{standalone}
\usepackage{tikz}
\usetikzlibrary{calc}
\begin{document}
\begin{tikzpicture}
\coordinate (O) at (0,0);
\coordinate (A) at (5,4);
\draw 
  let \p1=(O), \p2=(A),
      \n1={atan2(\y2-\y1,\x2-\x1)},
      \n2={ifthenelse(abs(\n1)<=90,\n1-90,\n1+90}
  in  (O)--(A)
  node[pos=.5,draw,fill=yellow,minimum width=2cm,circle,anchor={\n2},outer sep=0pt]{}
;
\end{tikzpicture}
\end{document}

---

Transformations in scopes do not apply to nodes automatically. But you can enable this transformations for shapes too: add transform shape to the scope options.

\documentclass[border=1cm]{standalone}
\usepackage{tikz}
\begin{document}

\begin{tikzpicture}
\def\iangle{atan(4/5)}
\coordinate (O) at (0, 0);
\coordinate (A) at (5, 4);
\draw (O)--coordinate[pos=0.5] (mid) (A);

\begin{scope}[rotate=\iangle,transform shape]
    \node[draw, circle, minimum size=2cm,yshift=1cm] at (mid) {};
\end{scope}
\end{tikzpicture}
\end{document}