number of subgroups index $p$ equals number of subgroups order $p$

Consider the homomorphism "multiplication by $p$'' from $A$ to itself; the kernel is $A_p$ and the cokernel (quotient of the codomain by the image) is $A/A^p$. Since $A$ itself is finite, an analogue of the rank nullity theorem in linear algebra shows that $A_p$ and $A/A^p$ have the same order. Since they are abelian groups in which every element is killed by $p$, they can be thought of as vector spaces of the same finite dimension over $\mathbb F_p$.

Now subgroups of order $p$ are contained in $A_p$, and so are precisely the one-dimensional $\mathbb F_p$-vector subspaces of $A_p.$ On the other hand, subgroups of index $p$ have to contain $A^p$, and so are in bijection (under the quotient map) with the codimension one subspaces of $A/A^p$.

So now you are reduced to checking that if $V$ is a finite-dimensional vector space over $\mathbb F_p$, the number of one-dimensional subspaces and the number of codimension one subspaces are the same. This can be checked by using the fact that $V^*$ (the dual space to $V$) and $V$ are (non-canonically) isomorphic, and that the one-dimensional subspaces of $V$ are put in bijection with the codimension one subspaces of $V^*$ by considering annihilators.

[Note: this answer is the same in spirit as Chris Godsil's, but I have replaced duality theory for finite abelian groups by linear algebra over $\mathbb F_p$, which might be more familiar.]

We may assume $A=E_{p^r}$.

Since every nonidentity element in $A$ is of order $p$ and $A$ is abelian, we can generate a subgroup of order $p^{r-1}$ by the following procedure. First, we choose an element $x_1$ in $A\setminus\{1\}$ to get a subgroup $\langle{x_1}\rangle$ of order $p$; Second, choose an element $x_2\in A\setminus \langle{x_1}\rangle$ to get a subgroup $\langle{x_1,x_2}\rangle$ of order $p^2$; \dots; Finally, we can choose an choose an element $x_{r-1}\in A\setminus \langle{x_1,\dots,x_{r-2}}\rangle$ to get a subgroup $\langle{x_1,\dots,x_r}\rangle$ of order $p^{r-1}$. The total choices of this procedure is $(p^r-1)(p^r-p)\cdots(p^r-p^{r-2})$. However, there are $(p^{r-1}-1)(p^{r-1}-p)\cdots(p^{r-1}-p^{r-2})$ choices in this procedure will generate a same subgroup of order $p^{r-1}$. Therefore, the number of subgroups of $A$ of index $p$ is $\frac{(p^r-1)(p^r-p)\cdots(p^r-p^{r-2})}{(p^{r-1}-1)(p^{r-1}-p)\cdots(p^{r-1}-p^{r-2})}=\frac{p^r-1}{p-1}$.

Remark. It seems that by the same method, we can get for $1\leqslant i \leqslant r-1$, the number of subgroups of $E_{p^r}$ of order $p^i$ is $\frac{p^r-1}{p-1}$. They are all same!

Subgroups of $A$ with index $p$ give characters of $A$ with multiplicative order $p$, and so the number of subgroups of index $p$ in $A$ is equal to the number of subgroups of order $p$ in the character group of $A$. But a finite abelian group is isomorphic to its character group.