Side effects of using large resistances

There are many drawbacks to both low and high values alike.

The ideal values will fall in between very large and very small for most applications.

A larger resistor of same type will, for example, create more noise (by itself and through small induced noise currents) than a smaller one, though that may not always be important to you.

A smaller resistor will drain more current and create more losses, as you have surmised yourself.

A larger resistor will create a higher error with the same leakage current. If your feedback pin in the middle of your resistors leaks 1 μA when the resistor feeding that leak is 1 MOhm, that will translate to an error of 1V, while a 10k resistor will translate to an error of 10mV.

Of course, if the leakage is in the order of several nA or less, you might not care much about the error a 1 MOhm resistor creates. But you might, depending on what exactly you are designing.

Smaller resistors in feedback systems, e.g. with inverting amplifiers using op-amps, may cause errors on the incoming signal if the incoming signal is relatively weak.

It's all checks and balances, and if that's not enough information at this point, you might want to ask a more direct question about specifically what you are doing. With schematics and that.

In addition to the issues that @Asmyldof mentions, when using high resistances in the megaohms (and especially at 10M and more) environmental contamination such as dust, skin oils, soldering flux residue etc can easily reduce the effective resistance in unpredictable and time-varying ways.

In addition to other answers, also consider thermal noise. As your resistance goes up, so does the noise. If you want very accurate measurements, this may be an issue.