Is there actually a cave entrance on every screen?

No, there isn't a secret entrance on every screen. Additionally, not all of them are accessible right from the get-go. Some require items that you collect through game play, like bombs.

An overworld map, as well as all hidden locations, and where to find what (like free rupees, potion shops, gambling spots etc) can be found here.