Necessary and sufficient condition for diagonalizable matrix

Let $M(X)\in\Bbb{R}[X]$ be the minimal polynomial of $A$. Let $P(X)\in\Bbb{R}[X]$ be a polynomial without repeated roots that annihilates $B$. Then $P(B)=0$ and so $P(A)=0$ and $AP'(A)=0$, as you note. Then $M(X)$ divides $P(X)$ and $M(X)$ dividex $XP'(X)$. Because $P(X)$ has no repeated roots, it has no common root with its derivative $P'(X)$. It follows that $M(X)$ divides $X$ and so $A=0$.

Here's an alternative approach: let $\lambda$ be any non-zero eigenvalue of $A$, and let $x$ be a corresponding eigenvector. We see that $$ (B - \lambda I) \pmatrix{0\\x} = \pmatrix{Ax\\ 0} \neq 0\\ (B - \lambda I)^2 \pmatrix{0\\x} = 0 $$ So we see that $B$ fails to be diagonalizable.

Thus, $A$ must have $0$ as its only eigenevalue. However, since $B$ is block upper triangular with $A$s on the diagonal, we see that if $A$ has $0$ as its only eigenvalue, then so does $B$. For $B$ to be diagonalizable with $0$ as its only eigenvalue, we must have $B = 0$.

If $B = 0$, then $A = 0$, as required.