Why is gravity so hard to unify with the other 3 fundamental forces?

If you go back to the origins, the difficulty in merging gravity with the other forces mostly stems from general relativity being a purely geometric theory -- again, that's in its original form -- and all the other forces being quantum, by which I mostly mean they are conveyed by well-defined force particles. The photon as the particle that conveys the electromagnetic field is the simplest example, but the idea carries over very well to both the weak and strong forces.

General relativity in contrast works very, very well without even invoking such concepts, or for that matter particles in general. As Einstein formulated it, GR really, truly is all about curved spaces.

By analogy it was subsequently assumed -- I think sometime back in the 1960s or perhaps 1950s? -- that gravity must also have a quantum form, but it's always been an assumption, not an absolute proof, a sort of "it worked here, and here, and here... so surely it also works just as well for the last force, gravity?"

But it's a bit tough to bridge such a huge gap. It's reasonably easy to provide a general description of gravity as a universally attractive force, although even there you quickly get into odd infinity problems not seen with other forces. But if you do that... what happened to all that part about the space being curved? The simplest possible all-attractive quantum force model would simply keep space as a rigid framework and do everything pretty much as with the electromagnetic force, only with a single type of charge (mass).

So, you can do one (curved space), or you can do the other (universal graviton-mediated quantum attraction), but it's not trivial to do both. And no matter how you do it, the other forces don't directly bend space, which keeps gravity pretty unique. Consequently, the details never seem to work out quite right, and many books have been written about why that might be.

A detailed account of the subtleties and technicalities that highlight the problem, is a research topic by itself!

An attempt is made here to provide some brief discussion on epistemological arguments, without mathematical jargon or detailed phenomenology, which I am sure some would wish to complement or improve. I would welcome it.

There are a number of reasons which, when combined, can help us understand why we don’t see how to proceed in the endeavour of finding a good theory of quantum gravity and hence unify it with the other forces of nature. The main reason is the physically different structures of the gravitational force and the other forces of nature. When it comes to gravity, even the notion of quantum fluctuations of the fields is already problematic. While for the other forces of nature quantum fluctuations have meaningful interpretation and are relatively "easy" to calculate.

Another possible reason is, the tools we are using and the philosophy we hold on the notion of quantisation of fields, and that we try to push this philosophy to include gravity. This has been realised in the “old” approaches, in which attempts were made to construct perturbatively renormalizable theories of quantum gravity. They have all suffered from one or another shortcoming.

A successful quantum field theory allows us to do calculations, and extract sensible results which then can be tested against experiments. In the two main approaches to quantum gravity, string theory and loop quantum gravity, such testable calculations have not been possible to access experimentally, due to the large amount of energies required. So the results of these theories remain at best, at the present time, just theoretical speculations. But both these theories are still being developed and by no means they are, at this stage, completed theories of nature.