Disable Core Extensions in Magento 1.x

Create a file in app/etc/modules called Zzz.xml (so it loaded last) with this content

<?xml version="1.0"?> 
<config>
    <modules>
        <Mage_Downloadable>
            <active>false</active> 
        </Mage_Downloadable>
        <Mage_Authorizenet>
            <active>false</active> 
        </Mage_Authorizenet>
        <!-- all other modules here -->
    </modules>
</config>

There is a chance that you won't be able to disable some of them because there are other modules depending on some of them.

For example if you want to disable Mage_Usa you will have to disable Mage_XmlConnect also.

[EDIT] Here is what you can disable:

• Mage_Usa (only if you disable Mage_GoogleCheckout and Mage_XmlConnect)
• Mage_Authorizenet
• Mage_Downloadable
• Mage_GiftMessage
• Mage_GoogleCheckout cannot be disabled because of this: http://www.magentocommerce.com/bug-tracking/issue/?issue=14359
• Mage_Paygate (Only if you disable Mage_Paypal, Mage_PaypalUk and Mage_Authorizenet)
• Mage_Poll
• Mage_Weee (only if you disable Mage_XmlConnect also)
• Phoenix_Moneybookers

Assuming you are using 1.7+, you can safely disable Mage_Poll and Phoenix_Moneybookers. Mage_Poll executes code on each page so I usually disable that if I don't need it.

Mage_Authorizenet, Mage_GoogleCheckout and Mage_Paygate should be safe to disable but per the comments some silly bugs may be in there. I would not care about them much, as long as their payment methods are inactive they are basically disabled.

Mage_GiftMessage can be turned off via admin settings, so I would not bother with it either.

As for Mage_Usa, Mage_Weee and Mage_Downloadable, do not disable unless you have some hours to spend on testing and fixing. I tried to disable Mage_Usa at some point (Magento 1.5) and it broke most of the shipping modules.

Despite being late with an answer I would like to answer the question from @snh_nl which modules you can disable: Simply all of them, except Mage_Core.

But to disable tight couple modules you need to install another module which takes care that nothing will break. Therefore I've developed: https://github.com/Zookal/magento-mock

Zookal Mock: Transparent auto-detecting of disabled core modules and extensions and providing mock objects for not breaking Magento. Nothing to configure. No class rewrites. Only one observer. Works out of the box. You can even physically remove the files!

One thing to consider: It does not work on command line.