Are laws of nature absolute or are they just very good approximations?

Every model in physics is an approximation to some degree. The concept of emergence is relevant here, systems in nature can combine and interact to produce new systems which have properties that the smaller systems from which they are built do not. A good example and the one you have highlighted is the "breaking down" of Newtonian mechanics at small physical scales at which point quantum mechanics becomes a better model.

The term "law of nature" does not mean some rule that has to hold everywhere and for everything with perfect precision; rather it refers to a pattern that holds good over a wide range of circumstances and with good enough precision to earn the title. Sometimes it is used for patterns that are very wide-ranging indeed, such as the law of conservation of momentum for isolated systems, but even a law like that one is hard to state in a clear and unambiguous way at the largest and smallest scales. At the scale of large parts of the whole cosmos it is hard to get to the regime called "asymptotic" in general relativity where ideas such as total momentum become well-defined. At the small scale called Planck scale it is expected that high-energy processes are relevant and all of our physical ideas are tentative. So statements called "laws of nature" sometimes run out of validity not because the statement is directly wrong, but because it is not speaking the right language---the terms in which it is stated are no longer the right ones to describe the phenomena that are observed. This limitation probably comes in for everything we ever state in science. We are always in a process of learning how our terms and concepts have to be broadened or made more rich in order to describe new areas more fully and accurately.

Mathematics is a never-ending quest, and it is probably true that science too is a never-ending quest, though we cannot be sure about that. The "laws of nature" as we may try to state them at our current state of knowledge are valuable insights but not the final word.

Or is it just a handy approximation that happens to work extremely well for everyday sized objects/forces?

Yes.

Suppose I have some arbitrarily precise measuring devices capable of taking measurements down to the Planck length/second/force. And I set up an experiment where I apply a force to an object to test its acceleration, removing all outside interference. Would F=ma hold true?

Quantum mechanics says you can't do this. Specifically, the Uncertainty Principle.

At the particle length, all sorts of non-Newtonian behavior happens. The classic electron double slit experiment, for example, is not consistent with particles being tiny balls with a position and a momentum value. They can also "tunnel" through "solid" objects, which is a problem for classical collision mechanics.