How do I pass a function as a Lightning component attribute?

The solution is to declare an event instead of an attribute when you want to provide a callback. Below is myButton.cmp rewritten to use events.

pressEvent.evt

<aura:event type="COMPONENT" />

myButton.cmp

<aura:component>
    <aura:attribute name="label" required="true" type="String" />
    <aura:attribute name="class" required="false" type="String" />

    <!-- Declare an "instance" of pressEvent -->
    <aura:registerEvent name="press" type="c:pressEvent"/>

    <input type="button" value="{!v.label}" class="{!v.class}" onclick="{!c.onClick}" />
</aura:component>

myButtonController.js

({
    onClick : function(component, event, helper) {
        helper.trackButtonClick(component, event);

        var event = component.getEvent('press');
        event.fire();
    }
})

This wasn't clear from the documentation, but there are two ways to handle an event. The first is to use the <aura:handler /> markup. To handle the press event of myButton, you could do this:

<aura:handler name="press" event="c:pressEvent" action="{!c.onOk}"/>

I would avoid this approach because if you have more than one button, the press event for all of them would be handled by the same action c.onOk.

The second way to handle an event is to pass a callback as an attribute in the <c:myButton /> markup. I didn't see this in the Salesforce docs but found it in an unrelated question. This is exactly what I needed. Each instance of myButton gets a unique callback.

myButtonConsumer.cmp

<aura:component>
    <!-- No aura:handler necessary. Just pass the callback as an attribute -->
    <!--<aura:handler name="press" event="c:pressEvent" action="{!c.onOk}"/>-->

    <c:myButton label="OK" class="slds-button" press="{!c.onOk}" />
    <c:myButton label="Cancel" class="slds-button" press="{!c.onCancel}" />
</aura:component>

myButtonConsumerController.js

({
    onOk: function(component, event) {
        alert('OK pressed!');
    },

    onCancel: function(component, event) {
        alert('Cancel pressed!');
    }
})

Another solution will be using <aura:method>. The downside will be that you are not able to 'bind' a value from the parent markup (you have to call the method from a parent's JavaScript Helper) but maybe this technique is interesting for you anyway:

MyComp.cmp:

<aura:method name="doSth" action="{!c.onDoSth}">
    <aura:attribute name="callerComponent" type="Aura.Component" />
    <aura:attribute name="callback" type="Object" />
</aura:method>

MyCompController.js:

({
    onDoSth:function(component, event, helper) {
       var params = event.getParams().arguments;
       var callerComponent = params.callerComponent;
       var callback = params.callback;
       [do sth]
       if(!$A.util.isUndefinedOrNull(callback)) {
          callback(callerComponent, 'message from my comp');
       }
    }
})

ParentComponent.cmp:

<aura:component>
   <aura:handler name="init" value="{!this}" action="{!c.onInit}" />
   <c:MyComp aura:id="myComponent" />
</aura:component>

ParentComponentController.js:

({
   onInit:function(component, event, helper) {
      helper.callMyComponent(component);
   }
})

ParentComponentHelper.js:

({
    callMyComponent:function(component) {
        component.find("myComponent").doSth(component, myInternalCallback).bind(this);
    },

    myInternalCallback:function(component, returnMessage) {
        //back from other component
    }
})