Where can I see all Xbox live games I own?

You can view your purchase and download history for your account on the Microsoft website. You can filter that list down to Arcade Games, or Full Games to view just the games you have got, or you can view the full list which includes things like apps and DLC.

All given links now point to a landing page saying you should download the Xbox app on PC or your phone. Since the recent update of the Xbox app on iOS and Android I am able to see them indeed. There is no such section online.

After some searching however, I found a possibility on the Microsoft website (not Xbox) that shows your transaction history: https://account.microsoft.com/billing/orders/?period=AllTime&type=All&filterChangeCount=0

I'm not sure how far this will go back in time, since I only started buying digital games 2 years ago. It is not a full library of your games, but at least you should be able to see if you own a specific game or not.