Is $\mathbb{H}P^\infty_{(p)}$ an H-space?

No. If it were an $H$-space, there would be self maps of $\mathbb{H}P^\infty_{(p)}$ inducing multiplication by $k$ in degree $4$ homology for all integers $k$. But this is not the case by a Theorem of S. Feder and S. Gitler in "Mappings of quaternionic projective spaces", Bol. Soc. Mat. Mex. 34 (1975) 12-18. Using Adams operations in complex $K$-theory they show that such a $k$ must be a $p$-adic square.

Sorry for dredging up this question, but here is another argument (at least for $p$ odd, but maybe you don't need this) that came up while thinking about an unrelated problem. If $\mathbf{H}P^\infty_{(p)}$ was a H-space, then all Whitehead products must vanish, so it suffices to establish that there's a nontrivial Whitehead product. One that does not vanish is the following: take the element $\iota:\mathbf{H}P^1\hookrightarrow \mathbf{H}P^\infty$ in $\pi_4(\mathbf{H}P^\infty)$; then, the $(p+1)/2$-fold Whitehead product $[\iota, \cdots, \iota]\in \pi_{2p+1}(\mathbf{H}P^\infty)$ is nonzero. Under the isomorphism $\pi_{2p+1}(\mathbf{H}P^\infty) \cong \pi_{2p}(S^3)$, it's precisely the unstable representative for $\alpha_1$, aka the first nontrivial attaching map in $\mathbf{C}P^{p}_{(p)}$.