Why doesn't the HTTP/2 spec require TLS?

There is no technical reason to limit HTTP/2 to TLS. Communication without TLS has its technical use, no matter if this is unencrypted traffic or if the traffic is encrypted by other means (VPN etc).

Restricting HTTP/2 to TLS in the standard would bind the use of the HTTP/2 protocol to the use of TLS for political(*) reasons only. Such bindings for non-technical reasons are usually avoided: If you look for example at the RFC for HTTP/1.1 they explicitly keep the transport layer open, i.e. acknowledge that HTTP/1.1 is usually used on top of TCP/IP but might be used on top of other protocols (RFC2616, section 1.4).

Thus while one might think that there was some evil lobbying I think that simply the majority found that standards should be a place for technical details but not for political(*) statements.

An interesting mailing thread in this context is Mandatory encryption is theater on the IETF HTTP WG mailing list in 2013 which highlights the diversity of opinions among the technical users. And is also visible from this thread that this is not a lobbying from some ISP or similar but that there are technical reasons to not bind HTTP/2 too much to TLS, because TLS is known to be not the optimal solution for the diversity of authentication, encryption and privacy problems: in some cases you want to have better protection than TLS can offer and in other cases you don't need the protection but the overhead of TLS is bothering you.

(*) To make more clear what I consider political: It's less geo-politics or company-politics but mostly personal opinions influenced by these larger politics. This leads to arguments based on the personal view of how the world should work and not based on technical reasons. Sometimes this political arguments are even blind to technical arguments because they don't fit into the personal view of the world. This includes the argumentation that all use cases require privacy (which TLS does not offer fully anyway), that small systems without large resources should just grow up or not use HTTP, that caching is an unimportant thing which does not need to be considered (i.e. we have lots of bandwidth and don't care if others don't) etc.

Mandatory encryption presents at least one thing that non-encrypted communication doesn't. Truly encrypted communication, end-to-end (at least http over SSL) is impossible to cache and requires more bandwidth. Requiring SSL/TLS would limit the case for non-sensitive information from being cached by some intermediate proxy server.

Encryption also has a cost. It adds some overhead, and adds some processing power. But then so do many features of any protocol. So it's a fair question to ask why encryption is special, and is optional.

There's some practical reasons to not require TLS as well. Unless you want to train users to skip through scary self-signed, or expired certificates, TLS requires you to obtain a signed certificate for each endpoint, and maintain it since they expire. There's millions of devices in homes and businesses that communicate over http. Baby monitors, washing machines, routers, phones, etc all commonly have http interfaces. Requiring TLS would mean these devices couldn't adopt http 2, and would have to stay on http 1.1.

There's other concerns too, but I think it highly involves what's valued. Stephen calls this "political", but this seems like an oversimplification over a standards argument. Political can mean a lot of things. Sometimes it means conflicting interests. Sometimes it means "geo-political", and sometimes it's about political ideology. And sometimes it means... I don't even know what. Throwing all that into one bucket seems a bit reductive in my view.