How to map a custom protocol to an application on the Mac?

This question is a decade old(!) but a Google search brought me here so I wanted to mention something I just discovered.

Platypus is an open source tool that allows you to create standalone "Applications" from a shell script or other scripting language. Although it's really just a script wrapper, it does enable some cool things like dialog boxes and menu bar items.

Critically, it even enables you to register your "app" as a handler for your own custom URL scheme. From the docs:

Register as URI scheme handler makes the app register as a handler for URI schemes. These can be either standard URI schemes such as http or a custom URI schemes of your choice (e.g. myscheme://). If your app is the default handler for a URI scheme, it will launch open every time a URL matching the scheme is opened. The URL is then passed to the script as an argument.

Setup is dead simple. Just provide your script, enter your desired scheme name on the advanced settings page and then click to build the app (it's all automated). Everything after the scheme and forward slashes will be passed as the argument to your script.

For example, you could use the following bash script as a handler for the "speak://" protocol.

#!/usr/bin/env bash
# The 'say' command on macOS will speak the provided text through the speaker
say $1

You could invoke it by entering speak://say-something-funny into your browser or by using the open command on the command line:

$ open "speak://hello-from-the-command-line"

On Macs this is easy to do with AppleScript. The most detailed description is in this article, Launch Scripts from Webpage Links on Mac. I'd read that page since it includes a full walk-through and a full working example to download.

Basically, you make an event handler in a script:

on open location this_URL
    display dialog "I'm doing something with this URL: " & return & this_URL
end open location

Then save that as an Application. Then in the Finder use Show Package Contents to edit the Info.plist. You add some properties to the app to register it as a handler for your protocol.

<key>CFBundleURLTypes</key>
<array>
    <dict>
        <key>CFBundleURLName</key>
        <string>Cliff's handler</string>
        <key>CFBundleURLSchemes</key>
        <array>
            <string>abcdef</string>
        </array>
    </dict>
</array>

As of today, the best way I found to solve this problem on a Mac with the least overhead is using the command line duti which allows me to define in a very simple text file all my associations:

brew install duti

You will need two things. First bundle ids of the Apps you want to associate:

mdls -name kMDItemCFBundleIdentifier /Applications/MacVim.app

Second the UTI of the file type, Apple provides a list, but you can also explore the supported UTI by your app like this:

mdls -name kMDItemContentTypeTree /Applications/MacVim.app

Now make a text file somewhere in your system where you associate bundle ids with UTI:

# ~/.default-apps.duti
#
# bundle id       UTI                  role
com.apple.Safari  public.html          all
org.vim.MacVim    txmt
org.vim.MacVim    public.ruby-script

Notice that I can associate a URL handler like txmt and also file types like Ruby scripts.

In that file you I keep track of all my app preferences and reproduce them immediately after a complete fresh install or when getting an account on other Mac just running:

duti ~/.default-apps.duti

I've not had occasion to use it, but some time ago I bookmarked OS X URL handler to open links to local files which is exactly what you're looking for.

The important part of the linked procedure is adding an appropriate CFBundleURLTypes to your application's Info.plist that describes the scheme. The example given there looks like this:

<key>CFBundleURLTypes</key>
<array>
    <dict>
        <key>CFBundleURLName</key>
        <string>Local File</string>
        <key>CFBundleURLSchemes</key>
        <array>
            <string>local</string>
        </array>
    </dict>
</array>