Do identical starting conditions always lead to identical outcomes?

There are several layers to this, so I get to have fun uncovering them.

The first layer is simple Netwonian Mechanics. If we assume Netwonian mechanics applies, and that the universe only consists of this box and its contents, and the contents of the box are set up exactly the same every time, then the resulting positions of the balls as time goes on is deterministic. It will be exactly the same, every single time.

However, it gets more interesting. Newtonian mechanics can be chaotic. A chaotic system is sensitive to initial conditions. A slight perturbation of the setup can yield drastically different results. Perhaps you put one of the balls in the wrong place: off by 0.5mm. This can cause the collisions to occur differently, and lead to drastically different results. A classic example of this is the double pendulum. In many regions, its motion is very sensitive to initial conditions. In this sense the box is unpredictable but deterministic. There's only one way the balls can move, but it's impossible to predict because properly predicting it would require infinitely precise measurements, and we don't have any way of measuring things like that.

Which brings us to widening our universe. Up to this point, we only considered a universe containing this box and this box alone. But there are outside influences on real world boxes. For example, there are gravitational forces being applied. Literally speaking, the position of Jupiter could affect the position of these balls colliding around by subtly changing the velocities of the balls.

Of course, what I just said sounds like astrology, so I should back off a bit. In practical scenarios, Jupiter is not going to noticeably affect the results. In a truly chaotic system, all inputs matter, but in our practical box, forces like friction are eventually going to make the system highly predictable. There's no need to go to a fortune teller to find the alignment of the planets before doing this experiment in real life!

But we are good at making experiments which are closer and closer to these ideal chaotic environments. So we can ask ourselves what happens as we push this to the extreme. What happens when we make an experiment so refined that Jupiter is having an effect. Well, we also start seeing other effects: quantum effects. Quantum effects will perturb the setup, just like failing to perfectly set up all of the balls, or failing to account for the gravitational effects of Jupiter. These effects are tiny, so in any practical situation, you will not observe them. However, they are there. And what's interesting about them is that, to our best knowledge, they are truly random. We know of no way to predict the effects of quantum interactions at a particle by particle level. As far as we can tell, their effects are truly nondeterministic, and so your box is nondeterministic too.

But, taking a step back, if you look at the sum total of many trillions of quantum interactions occurring each and every second, the results are statistically predictable. If you take the laws of quantum mechanics, and apply them to incredibly large non-coherent bodies (like a billiard ball or a box), you find that the equations simplify out to Newtonian mechanics (more or less). So unless you carefully craft your box and balls with the expressed intent of detecting the nondeterministic effects of quantum mechanics, you will find that the balls behave very much deterministically (although if you build a chaotic system, they may still not be predictable).

In Newtonian Mechanics there is no randomness involved once you know all the initial data. In fact, let $M$ be the phase space of a classical system. The points of $M$ are pairs $(q,p)$ of coordinates and momenta.

The time evolution of the system is described by ordinary differential equations on $M$, and once you know the initial conditions, by the existence and uniqueness theorem for solutions of ODE's you get the result that there is a single path in phase space corresponding to the sequence of states parametrized by time, in other words, a single association

$$t\mapsto (q(t),p(t)).$$

The trouble is that if the system has a huge number of particles the problem becomes extremely hard to tackle. Because of this one studies systems like this with Statistical Mechanics and then start dealing with means and so forth. But if you knew all the initial data and could solve the equations of Classical Mechanics (existence is guaranteed, but it might be very hard as I said), there is a unique path of evolution with no randomness whatsoever.

EDIT: Let's tackle this from a different point of view. In Quantum Mechanics a system is described by a state space $\mathcal{E}$. The elements of $\mathcal{E}$ are vectors called state vectors which we denoted like $|\varphi\rangle$. It turns out that if you know the state of the system, meaning that you know what state vector describes it, you still don't have full information about the system.

One example: consider a single particle with spin $1/2$. The spin can be either up or down. If the spin of the particle is up, the state of the system is $|\uparrow\rangle$ and if it is down the state is $|\downarrow\rangle$. These states are simple to understand, but that's not all. The most general state is $|\varphi\rangle = a |\uparrow\rangle + b |\downarrow\rangle$ and in this state everything you can say is that there is a probability of $|a|^2$ that when the spin is measured it will be up and $|b|^2$ that when the spin is measured it will be down.

And that is not all. Even in the state $|\uparrow\rangle$ you can't know the $x$ and $y$ components of spin. You just know the $z$ component is $1/2$. All you can get are probabilities.

So in QM even if you know the state, you don't know it all. There is randomness that is part of the theory.

Nevertheless the evolution is deterministic. Given one initial state, there is precisely a single evolution. But that's not the point. The system will evolve to some other state like these I've examplified, and in the state there will be inacessible information about the system. Again, deterministic evolution guarantees that you can evolve an initial state in a unique manner, but even if you know the state, you can't know it all.

Classical Mechanics isn't like that. In a Classical System you can know both position and momentum exactly in each state. Every observable is a function of position and momentum, hence you know any physical quantity if you know the state. Together with the fact that the evolution in time is unique, if you know the initial state you know it all.

Again you need to know exactly: (i) the initial conditions, (ii) the solution to the equations of motion. It is guaranteed to exist, not guaranteed to be easy to know.

I want to emphasize something that is implied in the other answers but not said explicitly:

Your problem is not as clearly defined as you may think.

Why?

• If your question is about real life, then because perfectly precise measuring is impossible, the problem makes no sense: you can never know for real if the balls are in the same positions as in the previous experiment, nor if they behaved in the same way. Unless your question is statistical, but then as mentioned above you would need to define your problem more precisely (tolerances for measurements, for instance; but then, as mentioned by Cort Ammon, we know of systems which are chaotic, which means that you cannot prescribe certain precision for the initial state and expect the outcome to be within the same precision).

• If the question is about physics, then you are putting it in some theoretical framework, and each has its own answer (in Newtonian Mechanics the answer would be yes, in Quantum Mechanics the answer would be no). To correlate the theory with real life you would need experiment, and then you go back to the first bullet.