Why are there no alternatives to TeX, or, why is TeX still used?

(My apologies in advance for any "too long; didn't read" sentiments -- if you only read one thing out of this, make it this link: Best Way to Start Using LaTeX/TeX?)

My Non-Expert Background:

• I started using LaTeX around 2002 to help engineering students at my university write their M.S. theses and PhD dissertations.
• For several years, I made basic modifications to an older thesis style file dating to the early 1990s, and made two or three attempts at writing one from scratch.
• I finally succeeded with a version I started in 2008, and made a lot of changes to it recently to isolate my users from the style's inner structure by default. The new style covers every feature of the old style, adds quite a bit more automation for our odd page headings in the ToC and other lists, and has around 50% fewer lines of code.

Things I've learned:

• Writing LaTeX documents is programming as soon as you get to even a moderate level of document requirements.
• It can be programmed badly, or programmed efficiently.
• How badly or efficiently you program it is strongly influenced by obsessive-compulsive tendencies, curiosity, and critical thinking.

As for specific issues posed:

most error messages have a mysterious origin

I can see where you're coming from there. You certainly don't get a clean Pythonic stack trace. There's no easy debugging tools (ConTeXt may have better support, though). Debugging is generally a process of producing minimal working examples, which either lets you identify where the problem went away as you started commenting and removing code, or at least scopes the problem down enough to where you can get expert advice more easily.

Really, this reminds me of how we'd debug Fortran code back in the day with 'echo checks' when we didn't have a good debugger (or else didn't know how to use one).

I have to read a paper on font naming schemes to find how to apply Helvetica?

I'm assuming this is partially hyperbole, but it does speak to the varying levels of documentation available. Some documentation is out of date, some is incomplete. Googling 'latex font' returns a pretty unpredictable range of answers, including some that are either just wrong, or outdated, or incomplete.

Googling 'latex font selection' returned other pages, including this Wikibooks page, but it's not quite what you were looking for unless you wanted to use XeTeX (which was created, in part, to handle Unicode and arbitrary fonts more easily).

I did eventually search this site for font information, and found How do I change a font document wide? -- it didn't give a complete example, but has all the pieces there. A MWE for Helvetica by default is

\documentclass{article}
\usepackage{helvet}
\renewcommand\familydefault{\sfdefault}
\begin{document}
Hello, world.
\end{document}

where even are functions? (with a diversion into OCD territory)

I'm not sure how you define functions versus macros as such, but I can certainly define blocks of reusable code and call them. Those blocks can take varying numbers of arguments, and act accordingly. They can effectively provide default values for arguments. LaTeX's global namespace can be a hindrance, but is generally worked around by tagging local variables with a special prefix or similar.

Example: our signature sheet for the thesis committee needs one line per committee member. Some committee members have a particular title (chair or cochair), others have no particular title. Each line should also include a spot for the member to write the approval date. At one time, we had a relative ton of copied and pasted code that only differed in whether we were typesetting the name for \memberone, \membertwo, etc. and whether the person had a specific title.

Now, we have a single function (or macro, I guess) that takes two arguments, and can typeset any member's signature line.

\newcommand{\nameanddateline}[2]{% irrelevant spacing and other commands removed
  \eqparbox{name}{\hrulefill} \quad \parbox{.75in}{\hrulefill} \\ 
  \eqparbox{name}{#1\ifthenelse{\equal{#2}{}}{\relax}{, #2}}
  \quad \makebox[.75in][c]{\hfill{}Date\hfill{}} \\ 
}

Called as \nameanddateline{\committeechair}{\chairtitle} we get a signature line with the chair's title added. Called as \nameanddateline{\memberone}{}, we get a signature line with no title at all. In a version I'll be posting in the next few days, I even got rid of the \memberN parts in favor of an array of names (students would use \renewcommand{\committeemembers}{Member 1, Member 2, ...}, and I can iterate over the comma-separated values in \committeemembers.

LaTeX won't force you to write efficient or maintainable code or markup. I don't claim mine is top-tier. My Python code would possibly make you cry in anguish. But it's like any other language in regards to it being up to the programmer to decide how clean they want things to be.

no serious opportunity for actual programming (as opposed to braindead markup)

I'm not sure what you'd call braindead markup. There's enough programming capability in it for me to get my documents typeset in a relatively efficient fashion. I can define my own commands, I can loop over elements of a simple array defined as comma-separated values, and I have if/then/else structures. For my needs, it's worked out fine. I'd certainly prefer this to writing VBA or fighting master documents and other issues in MS Word.

There's a lot of stale information out there. There are a lot of bad habits ingrained in long-time users who haven't kept up with new packages, or haven't thought through how to write their documents more like code, and less like explicit markup. Case in point: I've got a guy who has 1700+ \bf commands in his dissertation, each of which throws errors in newer memoir classes by default. All of those commands are for defining vectors. If he had ever read any of the intermediate material that said you could define a \bvec{} command to do the vector formatting, he'd only have to change \newcommand\bvec[1]{\bf #1} to \newcommand\bvec[1]{\bfseries #1} once.

Similarly, a collaborator brought some odd theorem spacing to my attention. He had been working around it by including a \vspace* command to remove the extra space. I thought, "this can't possibly be the way it's designed to work" and my OCD kicked in. Within an hour or so, I had found a much cleaner solution using the \qedhere command. It's well-documented at Wikibooks, but you'd have to go look for it rather than simply implement the first fix that comes to mind.

I'm disheartened by things like this, but not shocked. Maybe my standards are too low -- just a few years ago, I had a student ask me why her professor's Fortran finite element code wouldn't compile on her system, but worked fine on his. I started writing Fortran in 1989, and I've adapted code written in the early 70's. I didn't recognize the notation on the lines throwing errors. One of the other faculty looked at and said, "Oh, that's Hollerith notation." That notation was deprecated in the Fortran 66 standard, and deleted entirely in Fortran 77. The original professor was still using it in 2004 or so. Why? Because it still compiled for him. There's a similar effect in some long-time LaTeX users.

a confusingly fragmented codebase (which binary, distribution, library should I use, and how do they interact?).

Thankfully, the fragmentation is more in theory than in practice. At least in my experience, LaTeX code generally is distribution-neutral. I've had to modify one command in my thesis style to handle two different versions of the memoir class (one from 2005 in PCTeX, and the current one on CTAN). But all I had to do was to check if a particular very simple command was defined, and if not, define it myself.

CTAN, I think, is pretty good about having clean, updated packages. Instructions or packages you get from a random Google search can obviously vary quite a lot.

The alternatives to TeX exist: they are WYSIWYG word processors and desktop publishing programs. And in most commercial areas they are adopted far more widely.

TeX is still used in many academic and other fields because

• its output is beautiful
• it's free and open source
• it's gained considerable inertia in the 30 years since it's been in use, being part of the academic tradition

I'm not sure what you mean by a lack of "serious programming opportunities". Functions and data structures are implemented as macros, yes, but I think that most of the power TeX users would call themselves programmers.

You might find some of these questions helpful in your quest for understanding the TeX way:

• What is TeX used for?
• Why should I use LaTeX?
• What are good learning resources for a LaTeX beginner?

And as for putting section headings in Helvetica, it isn't that bad, but you do have to understand a few things about LaTeX and its history. TeX predates PostScript, PDF, TrueType, and most electronic typesetting standards (as well as Unicode which is a whole new set of issues). So its font mechanism is a little archaic. This has been dramatically improved with the XeTeX engine, which works natively with TrueType fonts and Unicode.

Here is a sample file, which is similar to the solution that you linked to in that it uses the sectsty package. This package is probably on your local TeX distribution anyway; no need to download or install. Unlike the example you linked to you have to run this through xelatex instead of pdflatex. Again, that executable is also probably in your local distribution too. The advantage is the the font specification is more sane.

\documentclass{article}
\title{Helvectica section headings}
% TEX.SE \url{}
\usepackage{lipsum}

\usepackage{fontspec}
\newfontfamily{\phv}{Helvetica}
\usepackage{sectsty}
\allsectionsfont{\phv}

\begin{document}
\maketitle

\section{First section}

\lipsum[1-3]

\section{Second section}

\lipsum[4-5]

\end{document}

Why it is that, within the problem domain of programmatic document formatting, TeX is the only serious player?

It's not obvious to me what you mean by this, but on the face of it, the presupposition isn't true. Based on Java, there are two exceedingly capable tools for generating PDFs: Open Office has a completely programmable back-end, whilst i-Text, a Java library for creating PDFs, allows almost complete coverage of PDF features, and is more capable in this respect than can be done with Tex engines (try putting a digital signature on your Latex'd letters). Both of these support a method of generating documents similar to HTML::Template where the document template and code are kept separate, which many regard as cleaner and more maintainable than how Latex documents are often written. FWIW, recent versions of Word allow VBA to access .NET libraries, allowing a quite high level of programmatic sophistication in document creation as well.

The strength of the Tex family doesn't lie, IMO, in its feature list, although with respect to microtypography, none of the above systems are competitive with Luatex, but rather in the ethos of document preparation and styling. Knuth's creation was extraordinarily rich. I have lost count of the number of ascii notations for mathematics whose authors believe it to be better than Knuth's: none in fact are. Knuth did not attempt to give the most general model, but just one sufficient for mathematicians writing for publication, but his model has weathered well and has been hugely influential on later designs, which generally have learned only some of the lessons on offer.

It takes time to become competent with any of the Tex systems, but once you have, it is a very natural way of preparing texts. Tex is really only hard when you want fine control over output, and then all systems are hard. The semantics of macros are often complained about and they are tricky, but it is not easy to find a computational mechanism that fits so smoothly into text.

So there are alternatives, and the Tex family has deep and distinctive strengths.