How can the diameter of the universe be so big, if nothing can go faster than light?

Is it possible to explain this seeming contradiction in a way that appeals to reason and doesn't require a pencil and paper?

I think so, but it does require some imagination/visualization (and perhaps a better explanation than I can provide).

How could two particles never exceeding 186,000 miles per second ever end up farther than 27.544 light years apart?

A short answer would be "space is expanding, which increases the distance between the particles, without them ever having to travel faster than the speed of light".

Think of two particles in a small universe (a.k.a. the early universe) moving away from each other at the speed of light. If the universe didn't expand, some time ($t$) later they would just be $2ct$ apart from each other. In an expanding universe, they will now be separated by a distance $$ 2d = 2ct + 2v_{space}(t)t $$ where $2v_{space}(t)$ is the speed at which the distance between them expanded. If you then try and calculate the average speed of a single particle $(d/t)$ you get $$ v_{avg} = \frac{d}{t} = c + v_{space}(t) $$ which is greater than the speed of light. Crucially, relativity hasn't broken - the particles always traveled at $c$, it was the space that expanded.

A realistic/intuitive analogy is difficult because these kinds of effects simply don't happen on our every-day scale. A thought experiment you could think about is: Two cars travelling at maximum speed, in opposite directions along a very long and straight road. What happens to the distance between them if the road stretches?

or is it just the kind of thing one doesn't understand unless they are a physicist?

Absolutely not, although having the time to think about these kinds of things and seeing the maths helps a lot.

Lets assume you have to explain the truth without any complications to your friend.

Firstly, give your friend some introduction and draw some analogy between physics terms and some nicer terms.

"Space is like a piece of cloth, a fabric which can stretch. There are some godly powers, forces which keep stretching the space fabric (the cloth) continuously. Light always moves along the fabric of space like a super fast worm."

Now ask your friend to be the god by holding two diagonally opposite corners of a cloth and ask him to stretch it quickly (faster than the worm). After stretching, tell him that light still continues to move along the fabric at the speed of light being oblivious to all the stretching magic which happened.

Ask him if the distance between his two hands has increased. Also, ask how far the worm has travelled during the process...

As Yashas said, the standard "math-free" explanation is that the space in between the particles stretches, so that the distance between the particles is growing faster than the speed of light. Now we have to explain why this doesn't contradict your statement, "the particles never exceed 186,000 miles per second".

The problem is we need to specify what this speed is relative to. In special relativity, you can compute the speed of any object relative to any other object (e.g., the speeds of the two particles relative to each other). However, this is only possible since spacetime in special relativity is uniform. Every point is the same, so anything at any two points can be compared.

In general relativity, spacetime is no longer homogeneous, and as a result it is impossible to compare vectors, such as velocities, at two different points. You can say that a particle has just moved past you at some speed, but it's meaningless to calculate the relative velocity of two distant particles.

As an analogy, suppose we stand at two different spots on the equator, both pointing North. If we wanted to compare the directions we're pointing, we have to meet in the same spot. If we meet on the equator, our directions will agree. If we both march up to the North pole and compare, they'll disagree! This happens because the Earth's surface is curved. For identical reasons, it is impossible to compare distant velocities in curved spacetime. There's no unique way to do it.

As such, the fact that the distance between them grows faster than $c$ doesn't contradict relativity. Nothing is ever traveling faster than $c$ according to a local observer.