Are fundamental forces conservative?

The notion of "conservative forces" is not in any way fundamental. What's fundamental is that we're able to assign a number to the state of a system, and that number is conserved.

The relatively uninteresting notion of a "conservative force" can be applied only to a force that can be expressed as a vector field that depends only on position. That means it's meaningful for Newtonian gravity and for electrostatics, but not for any other force that could be considered fundamental. Re the nuclear forces, see Do strong and weak interactions have classical force fields as their limits? .

General relativity has local conservation of energy-momentum, which is expressed by the fact that the stress-energy tensor has a zero divergence. A mass-energy scalar or energy-momentum vector isn't something that can be defined globally in GR for an arbitrary spacetime.

WP says (Gugg, where was the link from?):

However, general relativity is non-conservative, as seen in the anomalous precession of Mercury's orbit. However, general relativity can be shown to conserve a stress-energy-momentum pseudotensor.

The first sentence is wrong, because Mercury's anomalous precession can be described in terms of a test particle moving in a Schwarzschild metric. The Schwarzschild metric has a timelike Killing vector, so there is a conserved energy-momentum vector for test particles.

The second sentence is also misleading, since it doesn't make the global/local distinction. What's conserved locally isn't a pseudotensor, it's a tensor (the energy-momentum vector). Globally, there are various pseudotensors that can be defined, and the fact that they're pseudotensors rather than tensors means that they're fundamentally not well-defined quantities -- they require some specially chosen system of coordinates.