Circles inside a rectangle

It is perfectly possible to nest loops. However, nesting tikzpicture environments is known to be hazardous. Although it sometimes works, it should be avoided. Basically, it is expected to break.

What I would do is to draw the circles first and the fit the outer box around them. For example:

\documentclass[border=5pt,tikz,multi]{standalone}
\usetikzlibrary{fit,positioning}
\begin{document}
\begin{tikzpicture}[ultra thick]
  \coordinate (c0) at (0,0);
  \foreach \i [count=\j, evaluate=\j as \k using \j-1, evaluate=\j as \m using { int(mod(\j,5))==0 ? "" : "draw" }, evaluate=\j as \n using { \j>10 ? "20mm" : "10mm" }] in {1,...,11}
  \node (c\j) [right=7.5pt of c\k |- c0, circle, anchor=north west, minimum size=\n, \m] {};
  \node [fit=(c1) (c11), draw, rounded corners=15pt, inner xsep=5mm, minimum height=30mm] {};
\end{tikzpicture}
\end{document}

If you prefer, you can simply draw the circles one-by-one and then draw the box in the same way. Just name the leftmost and rightmost so that you can say fit=(<name 1>) (<name 2>) and all should be well.

In this particular case, you could, if you really wanted, nest the tikzpictures. However, (5,1) is at x=5cm, well to the left of the leftmost border of the box which is at x=10cm. So to put it in the box, you'd need to put it in the box.

You also need to put the tikzpicture inside a node. For example:

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{fit,positioning}
\begin{document}
\begin{tikzpicture}
  \draw [rounded corners=15pt] (10,0) rectangle ++(15,3);
  \node at (12.5,1.5) {% BREAKAGE EXPECTED !!
    \begin{tikzpicture}% DON"T TRY THIS AT HOME !!
      \draw circle (1cm);
    \end{tikzpicture}
  };
\end{tikzpicture}
\end{document}

But there is not much point in courting disaster when it would be much easier to write

\documentclass{article}
\usepackage{tikz}
\usetikzlibrary{fit,positioning}
\begin{document}
\begin{tikzpicture}
  \draw [rounded corners=15pt] (10,0) rectangle ++(15,3);
  \draw (12.5,1.5) circle (1cm);
\end{tikzpicture}
\end{document}

which produces the same result without the attendant risks and much more easily.

One of many possible solution (considering the first @cfr comment):

\documentclass[border=11mm,
               tikz]{standalone}
\usetikzlibrary{calc}

    \begin{document}
\begin{tikzpicture}[
box/.style = {draw, rounded corners=15pt,
              minimum width=150mm, minimum height=30mm}
                    ]
\node (a)   [box] {};
\foreach \i in {1,...,4}
    \draw ($(a.west)+(1.1*\i,0.5)$) circle (5mm);
\foreach \i in {1,...,4}
    \draw ($(a.west)+(5+1.1*\i,0.5)$) circle (5mm);
\draw ($(a.east)-(2,0)$) circle (10mm);
\end{tikzpicture}
\end{document}