Why do physicists trust black hole physics?

At first many people didn't care much for black holes. But later people showed that they were pretty unavoidable features of the theory of general relativity and that theory made other quite precise predictions that were tested and found good.

So when you are told that black holes are required if you have GR and GR looks like the best game in town then it becomes less bothersome.

But there is more. Having a detailed classical theory of black holes gives limits on the sizes of neutron stars, and we see neutrons stars. So you can look for neutron stars, look for evidence of their mass and now if you see one that is too large you can disprove GR. So people look. And GR wins again.

And eventually we start to see objects that behave like we expect a black hole to behave. So it makes sense to refer to things as black holes. Because they are enough like them that theories about black holes can work.

You still make a line between what has been observed and what hasn't. And Hawking radiation is on the wrong side. But if someone talks about Hawking radiation with certainty they are probably trying to explicate a known theory's predictions rather than an experimentally confirmed fact. But it is always important to distinguish between new results and known results, so the apparent certainty is probably an attempt to say "I am not saying something new" and it just comes off badly.

...why do we trust black hole physics? ... (physics which is derived by combining quantum mechanics and GR such as Hawking Radiation, things relating to the Information Paradox, etc. )

Formally, there isn't quite a reason to because we've not observed these things yet. But that's also perfectly okay as well because that is how science sometimes works: we have a theory that makes predictions, then we test those predictions.

We have some theoretical framework, general relativity, that does pretty well in all tests thus far. Since black holes are a solution to the Einstein field equations (EFE) and we have evidence of black holes (see below), then there is an expectation that physics around/near the event horizon described by GR also holds.

So we don't really know that an event horizon actually exists, but we believe it does because the solutions to the EFE show it should and we've observed BHs. Hopefully the Event Horizon Telescope will show us soon the event horizon of Sgr A*.

We also don't know that BHs undergo Hawking radiation, but since it also follows from applying QM (also fairly well tested) sufficiently far from the BH that quantum gravitational effects should be negligible and one can use QFT in curved spacetime, one can reasonably expect a BH to undergo Hawking radiation as well. Hopefully Fermi-LAT can detect evaporating primordial black holes during its lifetime (currently supported until 2018)

...I'd like to understand why in the absence of direct evidence ...

Note that we do have evidence of black holes. The image below is a map of the orbits of some of the 28 stars tracked over a period of 16 years at the interior of our galaxy. The orbits are less than 100 AU (about the orbit of Pluto) and are zipping along around 1000 km/s (compared to Pluto's 4 km/s).

(source)

We know that the orbital velocity of an object is related to the attractor mass; since these 28 stars are all orbiting the same point, the attractor is pretty much the dead-center of the galaxy (indeed it is stationary to ~1 km/s). A 2009 study, Gillessen et al (ApJ 692), has shown that the stars require the interior object's mass to be about 4 million solar masses, $\sim8\times10^{39}\,\rm g$ . Observations of Sgr A* itself suggest its radius is less than 1 AU. Such a massive object in such a small space is beyond any reasonable doubt, a supermassive black hole.

About evidence supporting the existence of Event Horizons in these very compact objects, here are some news from the well known Cygnus X-1, one of the most studied compact objects and the most promising candidate for a stellar collapse black hole:

... evidence of just such an event horizon may have been detected in 1992 using ultraviolet (UV) observations with the High Speed Photometer on the Hubble Space Telescope. As self-luminous clumps of matter spiral into a black hole, their radiation will be emitted in a series of pulses that are subject to gravitational redshift as the material approaches the horizon. That is, the wavelengths of the radiation will steadily increase, as predicted by General Relativity. Matter hitting a solid, compact object would emit a final burst of energy, whereas material passing through an event horizon would not. Two such "dying pulse trains" were observed, which is consistent with the existence of a black hole.

Source? just good ol' wiki... https://en.wikipedia.org/wiki/Cygnus_X-1#Compact_object

As for the rest of your questions, they all can be answered by the following. Physical theories have a property called their validity range. Basically, this validity range corresponds to the range in the values of the magnitudes involved in the theory in which we are sure the theory is correct. In general, because of two things: 1) the correctness has been empirically established; 2) the more fundamental theories actually predict that the corrections or deviations also predicted by them become negligible at these scales (or, if we don't have the theories yet, some other theoretical considerations provide the arguments).

In this sense, that's why we trust Newtonanin gravity for basic orbital calculations, that's why we trust Newtonian mechanics for basic, say, fluid dynamics, etc.

Quantum Field Theory (QFT) in curved backgrounds is a theory that assumes that we are working in the domain in which gravity is well described by classical GR (like in the vicinities of the event horizon of a stellar mass or supermassive black hole). Under that assumption, we do QFT in this framework. There should be nothing suspicious about that insofar we don't do unjustified extrapolations to the regime in which we know that our initial assumption is untenable (like, e.g., in the singularity at the center of the black hole). Finally, since both theories are believed to work just fine in that regime (i.e., drastic modifications of their core principles do not occur; there may be some minor modifications, e.g., we actually use a general reformulation of QFT for a general curved background, this is covered by the algebraic approach), then we trust in the conclusions. Of course, these conclusions have to be checked against experiment. And we are actually trying to do just this. The thing is that it's not that easy to do!