Can this proof of existence of a Hamel basis using transfinite recursion be shortened/simplified?

You are essentially correct. The simplest way of thinking about it, putting aside the full formality of transfinite recursion argument (which is really just cluttering the beauty of these proofs), is to go about this whole thing in the most obvious way you'd expect.

Choose some vector, then choose a vector from the complement of the span of the previous one; then choose some vector from the complement of the span of the first two, and so on, by transfinite recursion. At the end, the recursion has to stop because the vector space is just a set, and you will be left with a basis.

Fleshing this idea a little bit, let $V$ be a vector space, and $C$ a choice function from subsets of $V$. We define by transfinite recursion a $\subseteq$-increasing sequence of sets, $B_\alpha$ all of which are linearly independent.

Suppose that for all $\gamma<\alpha$, $B_\gamma$ was defined. Then $B_{<\alpha}=\bigcup_{\gamma<\alpha}B_\gamma$, is a linearly independent set (as the increasing union of linearly independent sets is a linearly independent set). If $F$ is undefined on $V\setminus\langle B_{<\alpha}\rangle$, then it is empty and $B_{<\alpha}$ is a linearly independent set which spans $V$, therefore it is a basis for $V$; otherwise let $B_\alpha=B_{<\alpha}\cup\{F(V\setminus\langle B_{<\alpha}\rangle)\}$.

We argue that there is some step in the construction where the recursion has to halt, otherwise we found an injection from $\sf Ord$ into $V$, which is impossible¹ since $V$ is a set, and $\sf Ord$ is a proper class.

If you'd like, you can full formalize this proof by defining $f$ and arguing there is a unique function etc. etc., but the essence is in the proof above, and it's much clearer too.

---

¹ As a set theorist, $V$ denotes the universe. So writing that there is no injection from $\sf Ord$ into $V$ made me feel strange from the inside.

Here's the much simpler argument I had in mind last night, when I realised what a fool I'd been. $\newcommand{\powerset}{\mathscr{P}}$ $\newcommand{\sspan}[1]{\left\langle#1\right\rangle}$

It's about as simple as Asaf Karagila's answer above, and it meets the requirement of Problem 112, which is to use a well-ordering on $V$, rather than a sufficiently large ordinal or [other] well-ordered set, as an indexing set for a family of elements or subsets of $V$.

For the latter kind of argument, it makes little difference to its complexity whether you choose to construct a family of elements of $V$ (as in the authors' proof, only briefly sketched above), or on the other hand, a family of subsets of $V$ (as in Karagila's proof); whereas, in the kind of argument that uses a well-ordering of $V$, it was a bad initial choice that I made to construct a function from $V$ into $\powerset(V)$, when I should have constructed [the characteristic function of] a subset of $V$ directly.

I now think that it was not so much my specific discomfort with set theory (real though that is), nor an excessive impulse towards formality - that may have played a part, too, but I felt I needed to take great care over my argument, to control the unexpected intricacy of what I had expected to be a straightforward exercise - but this bad initial choice of proof strategy, accompanied by mental rigidity and tunnel vision (a more general kind of mathematical disability!), that largely accounts for the sense of clumsiness and awkwardness about my proof, which led me to post my question.

I now feel quite sure it was an argument of the following kind that the authors had in mind, and expected the reader to come up with; but it will be interesting to see if there are more suggestions.

As before, well-order $V$ as a sum $S + (V \setminus S)$.

Abusing notation if necessary, let $2 = \{0, 1\}$, with $0$ signifying 'false', and $1$ signifying 'true'. Recursively, construct a function $\chi: V \to 2$, where, for all $x \in V$, $\chi(x)$ is $0$ or $1$ according to whether $x$ is or is not a linear combination of finitely many elements of $V$ strictly less than $x$.

This is simpler than asking if $x$ is a combination of elements $w < x$ satisfying $\chi(w) = 1$, although that would also work.

Actually, it's so simple, it doesn't even need the theorem of transfinite recursion! The definition: $$ \chi(x) = \begin{cases} 0 & \text{if } x \in \sspan{[\bot, x)} \\ 1 & \text{if } x \notin \sspan{[\bot, x)} \end{cases} $$ is already perfectly explicit.

Define: $$ S' \left[ = \chi^{-1}(1) \right] = \{x \in V : x \notin \sspan{[\bot, x)}\}. $$

So, we didn't really need $\chi$ at all - but I'll leave it in, if only as a salutary embarrassing reminder of the persistence of tunnel vision even when you think you've seen the light at the end of the tunnel!

The fact that $S'$ extends $S$ is obvious, because $S$ is linearly independent, and an initial segment of $V$.

The fact that $S'$ itself is linearly independent is proved in the same way as before, viz. by taking the largest element of a finite subset $F \subseteq S'$ w.r.t. the ordering of $V$.

That $S'$ spans $V$ is proved by transfinite induction on $x \in V$, thus: suppose $w \in \sspan{S'}$ for all $w < x$; then $[\bot, x) \subseteq \sspan{S'}$; and either $x \in S' \subseteq \sspan{S'}$, or else $x \in \sspan{[\bot, x)} \subseteq \sspan{S'}$. Q.E.D.

(I think that's all there is to it, but it seems almost too good to be true, so I may have slipped up.)