Does the Content Security Policy header provide a false sense of security if a page is served over unencrypted HTTP?

Your understanding is correct.

Anything that isn't authenticated, encrypted, and integrity-checked is insecure. Full stop. You can make the attacker work a little harder for things, but at the end of the day the attacker can get between the user and the server, and after that all bets are off.

Something like injecting ads already requires being in a MITM position, so yeah, they can just strip out your CSP headers. A mass attack that isn't customized at all might not be smart enough to do that yet, but it won't take long. End-to-end security, such as TLS, is the only way to go.

For what it's worth, if you absolutely must use unsecured connections and want some protections in your authentication (i.e. resistant to anything except a MITM), take a look at digest auth. It never sends the actual password, or anything password-equivalent, over the wire. It's also got at least some resistance to replay attacks. You could also implement something in JS, using asymmetric crypto or SRP or similar. That offers the advantage of even being able to transmit the data encrypted, though you're still vulnerable to an attacker just injecting JS that steals the plain-text data as soon as your JS decrypts it.

At the end of the day, though, that's more effort than TLS, for significantly less security. There very few situations where you absolutely can't use TLS. Even self-signed certs can be imported into a browser or have an exception added for that one site, and then the user will be safer than otherwise, as the attacker would need to present their own cert (which does not yet have an exception) and that will trigger another warning in the browser.

Does it then give the developer a false sense of security ....

You understanding of the issue is correct. But if this gives the developer a false sense of security depends on the knowledge of the developer.

A properly educated developer should know that with a successful MITM attack everything can be changed in the traffic. This is not only restricted to modifying the CSP or for ad injection but in fact the whole contents and code on the page can be replaced with something completely different. That is instead of nice fluffy kittens it could serve some ugly malware.