Can't see second azure sql database in SSMS object explorer

So for me, it turned out that when I was connecting using SSMS, I had set the database to connect to as my first database by accident - meaning that was all I could see.

On connect, go to Options, then check the database you're connecting to. If you're using multiple users to connect to that server like I was, my admin user ended up also being forced to connect to a single database instead of < default >.