How do I output a config value in a Sphinx .rst file?

For the substitution of links extlinks is fine, for including arbitrary config values as asked in your question you can use rst_epilog for substitutions (or rst_prolog for text, that should be added on top of your .rst files):

In your conf.py:

my_config_value = 42
rst_epilog = '.. |my_conf_val| replace:: %d' % my_config_value

In your .rst source:

My config value is |my_conf_val|!

In your output:

My config value is 42!

Ah hah!

Take a look at the sphinx.ext.extlinks module.

So I have code in my conf.py that does this:

extlinks = {'api_url' : (settings.BASE_URL + '%s', settings.BASE_URL)}

And in my .rst file, I have this:

:api_url:`/myapp/api/application/`

which produces the nicely formatted link as such:

http://localhost:8000/myapp/api/application/