What exactly makes a force conservative?
All fundamental forces are conservatives and I would say that this is a postulate. Fundamental physics is constructed in such way that there is a quantity called energy which can be assigned to every possible state. If any fundamental process seems to violate conservation of energy we nowadays believe that there are some states, processes or even interactions that we are missing to take into account. Once we are able to take into account every state and interaction, the system and its interactions are conservative.
On the other hand, at macroscopic level, most of times we are not able to describe the system in terms of fundamental forces. We need to replace the zillions of coupled equations describing the dynamics of the system by a single equation or force, which we shall call effective, and which can describe the macroscopic results we observe. However, in this process we may miss many of the states and processes occurring such that we are no longer able to keep track of the mechanical energy balance. Energy balance would fail unless we consider other forms of energy, such as heat, which is also an effective quantity. A classic example is friction. We are not able to describe two macroscopic surfaces interacting in terms of every microscopic particle participating in the process. So we forget about it and assume there is an effective force called friction. Mechanical energy balance fails and we need to assume that the effective missing energy is present in the form of heat. That is why friction is non conservative. Another example is that of an time varying potential. It is only non conservative because we are effectively replacing a large, with many particles and closed system by one small, with few particles under external interaction. There is something that we are not able to keep track whose effect is the same as of a time varying potential.
As you know, energy is always conserved.
When we talk of a force being non-conservative, it means that the force is operating within a system from which energy is allowed to escape.
Perhaps the most common example of this is a system where work is being done in the presence of friction. We talk of work being useful or not and that defines a parameter of our system. Non-useful work, in the form of work that generates heat instead of mechanical action, falls outside of our system, so it is "lost". Thus, that force is non-conservative.
In short, non-conservative forces are an emergent property of systems that define a boundary for energy transfers under consideration. Conservative forces transfer energy within the system. Non-conservative forces transfer energy out of (or, indeed, into) the system.
This might seem like a restatement of the definition of conservative and non-conservative forces but I think it sheds some light on what exactly makes a force conservative or otherwise.
One first must realize that forces are mediated by fields (well, more precisely by the exchange of virtual particles, but let's keep it all classical for the sake of simplicity and the fact that I don't understand QFT ;)). Now, it might have been presented to you that fields are just mathematical fictions used to do calculations but they are just as real as anything else. They themselves contain momentum, energy, angular momentum, everything.
Now, the action of a force on a particle is really just the interaction of the particle with the field. It might happen that the nature of the interaction between the field and the particle is of such a nature that when the particle goes in a complete loop, the energy that the field has transferred to the particle is exactly zero. In such a case, we call the field to be conservative. If the particle comes back to the position in which it earlier was but during the process, the field has taken away some energy from it or has given some energy to it then we call that force-field to be non-conservative.