Does Rademacher's convergent series for p(n) define an analytic function?
Not a direct answer to the question, but a brief numerical exploration of this function.
First, a trivial observation: we can write either $e^{\pi i x}$ or $\cos(\pi x)$ in the formula for the exponential sum $A_k(n)$ in the Rademacher series. This makes no difference at integer $n$, but gives different generalizations for noninteger $n$. The cosine version has the nice property of being real-valued on the real line and conjugate symmetric.
Here is a plot of the cosine-extended $p(n)$ on the real line:
The exponential version (real and imaginary parts):
Either version of the function seems to have simple zeros at the negative integers $-1, -2, -3, ...$. (This is quite nice if correct, because it matches the obvious combinatorial interpretation $p(-n) = 0$.) The cosine version has additional zeros on the negative real line (the first near $-0.93$).
At half-integers, it appears that all terms in the cosine version of the Rademacher series except the first term vanish, and so one has a trivial closed-form evaluation of $p(k+\tfrac{1}{2})$, $k \in \mathbb{Z}$. This is not the case for the exponential version.
Taking this leading term as a cue for the asymptotics on the real line, the origin is a turning point between exponential growth to the right and $\text{oscillation} \cdot O(n^{-1})$ behavior to the left.
Viewed in the imaginary direction, the exponential version appears to grow exponentially as $n \to +i \infty$ but remains small as $n \to -i \infty$. Plot of the real and imaginary parts of $p(i x)$:
The cosine version looks like the upper-half-plane exponential version:
There are additional zeros in complex plane. Since the Rademacher series converges slowly when the imaginary part of $n$ is large, it's a bit difficult to explore these zeros numerically. Here is a low-resolution plot of the exponential version of $p(z)$ on $z \in [-4,4] + [-2,2] i$:
And the cosine version:
The slow convergence also makes it difficult to search numerically for other potential closed forms (it is expensive to get more than ~6 digits). A faster algorithm for computing $p(n)$ to high precision near the origin would be very exciting.
Unoptimized Python implementation that I used to create these plots: https://gist.github.com/fredrik-johansson/7c2711887811ef9f2d7038b8451a4e63
Edit. We can write the series in the form $$p(n)=\sum A_k(z)\frac{d}{dz}f(z/k^2),$$ where $|A_k(z)|\leq Ck^{1/2}e^{C_1(\Im z)^+},$ where $y^+=\max\{ y,0\},$ and $C_j$ are various positive absolute contants, $f(z)=(\sinh\sqrt{z})/\sqrt{z},\; z=C_2(n-1/24)$. Notice that $f$ is an even entire function of order $1/2$, normal type so $f'(0)=0$. So $$p(n)=\sum c_k(z)f'(z/k^2),$$ where $|c_k(z)|\leq C_3k^{-3/2}e^{C_1(\Im z)^+}$. Since $f'$ is an entire function of order $1/2$, normal type and $f'(0)=0$, we have an estimate of the form $$|f'(z)|\leq C_2|z|e^{A\sqrt{|z|}},$$ and we see that the sum above must satisfy the estimate $$|p(z)|\leq C_4e^{C_1(\Im z)^++A\sqrt{|z|}}.$$
Therefore $p(n)$ extends to an entire function in the plane of exponential, type. An interesting question about such a function is the location of its zeros, and the related question, asymptotics for large $|z|$. One could try to plot $|p(z)|$ on a computer and look what to expect. Even a plot on the imaginary line may tell us something interesting.
Based on this argument, and pictures of Fredrik Johansson, I conjecture that zeros are asymptotic to some parabola $y=C\sqrt{x}$ in the right half-plane, and close to the real axis in the left half-plane (the pictures indicate infinitely many negative zeros). To prove this one has to prove the asymptotics $$p(z)=\exp\left(B(\Im z)^++A\Re\sqrt{z}+o(\sqrt{|z|})\right)$$ outside some small circles around zeros.
Remark. Unfortunately, this crude argument gives $C_1=B=2\pi$, and in such class of entire functions extension from integers to the complex plane is not unique.