Is there a set whose power set is countably infinite?
No.
Let's call a cardinal $\kappa$ a strong limit cardinal, if whenever $A$ is a set of cardinality strictly less than $\kappa$, also $\mathcal P(A)$ has cardinality $<\kappa$.
It is easy to see that $\aleph_0$ is a strong limit cardinal, exactly because everything smaller is finite, and the power set of a finite set is finite.
Now we can prove a general theorem:
Suppose that $\kappa$ is a strong limit cardinal. There is no set $A$ such that $|\mathcal P(A)|=\kappa$.
Proof. Recall Cantor's theorem, for all sets $A$, $|A|<|\mathcal P(A)|$. If $|\mathcal P(A)|=\kappa$, then $|A|<\kappa$. But now by virtue of being a strong limit cardinal, $|\mathcal P(A)|<\kappa$ as well. $\square$
In particular, it means there is no set whose power set is countably infinite.
The size of the power set of a set is larger than the size of the set itself. This statement is called Cantor's theorem.
If $n$ is finite, then the size of its power set is $2^n$ which is finite. So, the desired set has to be infinite. But then an infinite set has to have a set of the size of natural numbers (countable) inside it.
By Cantor's theorem again, the size of the power set of $\mathbb{N}$ is therefore greater than the size of $\mathbb{N}$ itself. This means that the size of $\mathcal{P}(\mathbb{N})$ has to be strictly larger than countable, i.e. uncountable.
Since the set you are looking for has a set of the size of natural numbers inside it, its cardinality must be strictly larger than countable. This shows that such a set whose power set is countable cannot exist.