When does symmetry in an optimization problem imply that all variables are equal at optimality?

The Monthly article "Do symmetric problems have symmetric solutions?" by William Waterhouse discusses this issue. For global optimality one really needs strong global constraints on the objective function, such as convexity (as in Igor's answer), and there's nothing one can say otherwise. However, local results hold in considerably greater generality. Waterhouse calls this the Purkiss Principle: in a symmetric optimization problem, symmetric points tend to be either local maxima or local minima.

Specifically, suppose we are optimizing a smooth function $f$ on a manifold $M$. The symmetry can be expressed by saying some finite group $G$ acts on $M$ and preserves $f$. If $x$ is a point in $M$ that is fixed by $G$, we would like to understand the behavior of $f$ near $x$. To analyze it, we can study the action of $G$ on the tangent space $T_x M$. The key hypothesis is that $T_x M$ should be a nontrivial irreducible representation of $G$. In that case, $x$ is automatically a critical point for $f$, and if it is a nondegenerate critical point, then it must be a local maximum or minimum (i.e., not a saddle point). In fact, even more is true: the Hessian matrix will have only one eigenvalue, which is either zero (degenerate), positive (local minimum), or negative (local maximum).

This is one of those results where finding the right statement is the real issue, and once the statement has been found it takes just a few lines to prove it. In his article, Waterhouse builds up to this formulation in several steps, and he shows how it encompasses various more concrete cases and applications. He also gives a wonderful historical overview.

If the representation on the tangent space is reducible, then the Purkiss Principle can fail. If we break the representation into a direct sum of irreducible subrepresentations, then the Hessian matrix for $f$ will have an eigenvalue for each irreducible (with multiplicity equal to the dimension of the irreducible), and there's no reason why they should all have the same sign. However, this decomposition is nevertheless very useful for doing local calculations, because even if $M$ is high-dimensional, $T_x M$ may decompose into just a few irreducibles.

So the upshot is this: if you are applying the second derivative test to a symmetric optimization problem, then representation theory will tell you what the symmetry implies, and irreducibility leads to the simplest possible answer.

A bit too long for a comment: Let's consider a optimization problem of the form $$ \min_x f(x)\quad \text{s.t.}\quad x\in C. $$

If we now consider "symmetry" a bit more abstract by saying that you have a group $G$ acting on the set $C$ such that the objective is invariant under the group action, i.e. for $g\in G$ we have $f(gx) = f(x)$ then you see that the set of minimizers is also invariant under the group action.

The most common condition for symmetry in my experience is convexity: if the feasible region is symmetric and convex, and the objective function is convex, it is immediate that argmin has all variables equal (or invariant by whatever your symmetry group is).