The problem of self-force on point charges
I'm not sure if this problem was ever solved in classical electrodynamics.
However, it is (somewhat) solved in quantum field theory electrodynamics (QED). In QED, self-interaction has noticeable effects on quantities such as the observed mass of a particle. Furthermore, the self-interaction effects create infinities in the theoretical predictions for such quantities (which is why I said "somewhat" above). But, these infinities can be cancelled out for any observable (such as energy or mass, etc...) This process of cancelling out the infinities is known as re-normalization.
To get a sense of how it works, imagine that your theory predicted the energy of a particle to be something like $$E_\textrm{theoretical} = \lim_{\lambda\to \infty} (\log\lambda + E_\textrm{finite})$$ where $\lambda$ represents the part of our calculation that becomes infinite. For example, if an integral diverges we can take set the upper bound of the integral to be a variable (such as $\lambda$) and then at the end take the limit as $\lambda$ goes to infinity. Methods such as these are called "regularization" (i.e. a way of rewriting the equation such that the divergent part of the calculation is contained within a single term).
Now in this limit, the total energy will be infinite. However, in the lab we can only measure changes in energy (that is, we need a reference point). So, let us then choose a reference point such that $E_\mathrm{0,finite}=0$. In that case, we subtract the reference point from the theoretical energy to get $$\Delta E_\textrm{observed} = \lim_{\lambda\to\infty} (\log{\lambda} + E_\textrm{finite} - \log{\lambda} - 0) = E_\textrm{finite}$$ and all is well. This last step is called re-normalization.
I suppose my first major question is simply, has this problem been solved yet? After a bit of research I came across the Abraham-Lorentz force which appears to refer exactly to this "problem of self-force". As the article states the formula is entirely in the domain of classical physics and a quick Google search indicates it was derived by Abraham and Lorentz in 1903-4, why is it that Feynman state the problem was still unsolved in 1963? Has it been solved in the classical case but not in QED?
This is still only a theoretical problem, as a measurement of the expected self-force needs to be very sensitive and was never accomplished. Theoretically, self-force can be said to be described satisfactorily (and even there, only approximately) only for rigid charged spheres. For point particles, the common notion of self-force (Lorentz-Abraham-Dirac) is basically inconsistent (with basic laws of mechanics) and can be regarded as unnecessary - for point particles there exist consistent theories like Frenkel's theory or Feynman-Wheeler theory (with or without the absorber condition) and their variations without self-force (there are other works free of self-force too).
J. Frenkel, Zur Elektrodynamik punktfoermiger Elektronen, Zeits. f. Phys., 32, (1925), p. 518-534. http://dx.doi.org/10.1007/BF01331692
J. A. Wheeler, R. P. Feynman, Classical Electrodynamics in Terms of Direct Interparticle Interaction, Rev. Mod. Phys., 21, 3, (1949), p. 425-433. http://dx.doi.org/10.1103/RevModPhys.21.425
The electromagnetic self-force problem has been solved recently, see here; the gravitational self-force problem has also been solved recently, see this article.