Why do physicists believe that there exist magnetic monopoles?

Aqwis, it would help in the future if you mentioned something about your background because it helps to know what level to aim at in the answer. I'll assume you know E&M at an undergraduate level. If you don't then some of this explanation probably won't make much sense.

Part one goes back to Dirac. In E&M we need to specify a vector potential $A_\mu$. Classically the electric and magnetic fields suffice, but when quantum mechanics is included you need $A_\mu$. The vector potential is only defined up to gauge transformations $A_\mu \rightarrow g(x)(A_\mu + \frac{i}{e} \partial_\mu ) g^{-1}(x)$ where $g(x)=\exp(i \alpha(x))$. The group involved in these gauge transformations is the real line (that is the space of possible values of $\alpha$) if electric charge is not quantized, but if charge is quantized, as all evidence points to experimentally, then the group is compact, that is it is topologically a circle, $S^1$. So to specify a gauge field we specify an element of $S^1$ at every point in spacetime. Now suppose we don't know for sure what goes on inside some region (because we don't know physics at short distances). Surround this region with a sphere. We can define our gauge transformation at every point outside this region, but now we have to specify it on two-spheres which cannot be contracted to a point. At a fixed radial distance the total space of angles plus the gauge transformation can be a simple product, $S^2 \times S^1$ but it turns out there are other possibilities. In particular you can make what is called a principal fibre bundle where the $S^1$ twists in a certain way as you move around the $S^2$. These are characterized by an integer $n$, and a short calculation which you can find various places in the literature shows that the integer $n$ is nothing but the magnetic monopole charge of the configuration you have defined. So charge quantization leads to the ability to define configurations which are magnetic monopoles. So far there is no guarantee that there are finite energy objects which correspond to these fields. To figure out if they are finite energy we need to know what goes on all the way down to the origin inside our region.

Part two is that in essentially all models that try to unify the Standard Model you find that there are in fact magnetic monopoles of finite energy. In grand unified theories this goes back to work of 't Hooft and Polyakov. It also turns out to be true in Kaluza-Klein theory and in string theory.

So there are three compelling reasons to expect that magnetic monopoles exist. The first is the beauty of a deep symmetry of Maxwell's equations called electric-magnetic duality, the second is that electric charge appears to be quantized experimentally and this allows you to define configurations with quantized magnetic monopole charge, and the third is that when you look into the interior of these objects in essentially all unified theories you find that the monopoles have finite energy.

The symmetry between the exchange of the electric and magnetic fields would become more complete. Indeed, this is not a proof that the magnetic monopoles exist. Moreover, the symmetry is broken, for example because the mass and charges of the two types of charged particles substantially differ.

But the actual state-of-the-art reasons why magnetic monopoles have to exist were explained e.g. in our discussion with Carlo Rovelli below my answer to this question:

Can Maxwell's equations be derived from Coulomb's Law and Special Relativity?
Can Maxwell's equations be derived from Coulomb's law and special relativity?

In any consistent theory of quantum gravity, one may pair-create at least black holes that have a nonzero magnetic flux through the event horizon; the two black holes have the opposite value, of course. There exists an instanton solution that guarantees that such a process has a nonzero probability. This solution can't be "banned" or "denied" without a rather severe violation of the laws of locality. It implies that there exist magnetically charged monopole states - light black holes - that are as heavy as the Planck scale (the lightest black holes worth the name). But they can be lighter.

In grand unified theories (GUT), one may write down completely regular, smooth solutions that put the old singular solution by Dirac for the "Dirac monopole" explained in another answer on a firmer ground. Such Dirac monopoles have to be as heavy as the GUT scale which is lighter than the Planck scale. All the intuition in this and previous paragraphs is field-theoretical in character but it is confirmed by our only theory whose reach goes beyond field theory, namely by string theory.

I would bet that these arguments may still sound too mathematical to you but they are extremely good science. The true logic behind physics is based on mathematics and the mathematics is beautiful for those who constantly adopt Nature's sense of beauty as more knowledge is being collected.

I'd be interested to know who told you such a thing. I gather that the existence of magnetic monopoles is a natural consequence of many (most? all?) grand unified theories, so that's probably what whoever it is is talking about. But there are quite a few physicists out there who are more swayed by experimental evidence.

If you combine the GUTs that predict the existence of monopoles with non-inflationary hot big bang theory, you predict that monopoles should have been produced in huge numbers in the early Universe, leading to the puzzle of why we don't see them around today. Inflation was originally devised at least in part to solve this problem: if the monopoles were produced before inflation, then they would have been "inflated away" to unobservably low density today (i.e., less than one in the observable Universe).