Do two matrices close in spectral norm have close traces?

Yes, it does, and you don't need Weyl's inequality. Note that $$ |\operatorname{tr}(A_p-B_p)|\le\sum_i|\lambda_i(A_p-B_p)|\le p\|A_p-B_p\|_2\le1. $$ Therefore $\operatorname{tr}(A_p)= p+\operatorname{tr}(A_p-B_p) \in [p-1,p+1]$ and $$ \frac{-1}{p+1}=\frac{p}{p+1}-1\le\frac{p}{\operatorname{tr}(A_p)}-1\le\frac{p}{p-1}-1=\frac{1}{p-1}. $$ In turn, $\left|\frac{p}{\operatorname{tr}(A_p)}-1\right|=O(\frac1p)$. Since we have assumed that the spectrum of $B_p$ is confined inside a compact interval, we also have $\|B_p\|_2=O(1)$. Thus \begin{align*} \left\|\frac{p}{\operatorname{tr}(A_p)}A_p-B_p\right\|_2 &\le\frac{p}{\operatorname{tr}(A_p)}\|A_p-B_p\|_2 + \left|\frac{p}{\operatorname{tr}(A_p)}-1\right| \|B_p\|_2\\ &=O(1)O(\frac1p) + O(\frac1p)O(1) = O(\frac1p). \end{align*}