Completely remove a subscription from Management Portal

Visit the portal. In the upper-right corner, you should see a link for Billing.

This will take you to a list of your subscriptions.

Select your subscription. Then, on the right side, you'll see a few options, and one toward the bottom should be 'Cancel Subscription.'

According to the Azure support it is (currently) not possible to delete canceled subscriptions.

Greetings from Microsoft Azure. I reviewed your request and would like to mention that there is unfortunately no option to remove the disabled subscription from the Azure portal. This is by design to enable customer’s view the subscriptions purchased by them right from the day the Azure account was created.

I consider this a bad design choice but that probably is just me.