Evaluate $\sum_{n=0}^{\infty} \frac{z^{kn+p}}{(kn+p)!}$

Hint:

Integrate both sides of your equation $ p$ times from $0$ to $z'$ over $z$ (after each integration replace $z'$ with $z$).

The result is:

$$\sum_{n=0}^{\infty} \frac{z^{kn+p}}{(kn+p)!} = \frac{1}{k}\sum_{i=0}^{k-1}\frac1{\varepsilon_i^p}\left(e^{\varepsilon_iz}-\sum_{q=0}^{p-1}\frac{(\varepsilon_i z)^q}{q!}\right)\tag1$$

A closer look at the expression in parentheses reveal that it is the exponent of $\varepsilon_i z$ without first $p$ terms of its Maclaurin series.

The rhs of (1) can be further simplified to $$\frac{1}{k}\sum_{i=0}^{k-1}\frac{e^{\varepsilon_iz}}{\varepsilon_i^p}-\sum_{i=1}^{\left\lfloor\frac pk\right\rfloor}\frac{z^{p-ki}}{(p-ki)!}$$ since $\frac1k\sum_{i=0}^{k-1}{\varepsilon_i^{q-p}}=\delta_{q\operatorname{mod}k,p}.$

The expression (1) can be easily proved by induction. Let $$f_p (z)=\sum_{n=0}^{\infty} \frac{z^{kn+p}}{(kn+p)!},\quad F_p (z)=\frac{1}{k}\sum_{i=0}^{k-1}\frac1{\varepsilon_i^p}\left(e^{\varepsilon_iz}-\sum_{q=0}^{p-1}\frac{(\varepsilon_i z)^q}{q!}\right)$$

As you have proved: $$f_0(z)=F_0 (z).$$ Further: $$ f_p(z)=F_p (z)\implies f_{p+1}(z)=F_{p+1}(z).$$ Indeed: $$ f_{p+1}(z)=\int_0^zf_p(z')dz'\stackrel {I.H.}= \int_0^z F_p (z')dz'=F_{p+1}(z).$$

In general, let $f(z)$ be a power series, say $$f(z)=\sum_{n=0}^\infty a_nz^n,$$ where $a_0,a_1,a_2,\ldots$ are complex numbers. Then define $$f_{k,p}(z)=\sum_{n=0}^\infty a_{kn+p}z^{kn+p}$$ for every positive integer $k$ and for any non-negative integer $p$. Note that when $p=0,1,2,\ldots,k-1$, $f_{k,p}(z)$ is given by $$f_{k,p}(z)=\frac{1}{k}\sum_{r=0}^{k-1}e^{-\frac{2rp\pi i}{k}}f\left(e^{\frac{2r\pi i}{k}}z\right).$$ This is due to the following identity: for any integer $t$, $$\frac{1}{k}\sum_{r=0}^{k-1} e^{\frac{2rt\pi i}{k}}=\left\{\begin{array}{ll}1&\text{if }k\mid t,\\0&\text{otherwise}.\end{array}\right.$$ If $p\geq k$, I don't think there is a nice way to simplify $f_{k,p}$ except through truncation: $$f_{k,p}(z)=f_{k,s}(z)-\sum_{n=0}^{q-1}a_{kn+s}z^{kn+s}$$ if $$p=kq+s$$ for some integers $q$ and $s$ such that $q\geq 1$ and $0\leq s<k$.