Is it possible for a polynomial to be integer-valued only for prime inputs?
No. Suppose there is such a polynomial $f(x)$ of degree $k$, say. First off, this polynomial has to have rational coefficients (by Lagrange interpolation, say). Let $N$ be the least common denominator of the coefficients. Then $g(x)=Nf(x)$ is a polynomial with integer coefficients and $g(2)$ is divisible by $N$. But $g(2-N)\equiv g(2)\equiv 0\pmod N$, so $g(2-N)$ is divisible by $N$, hence $f(2-N)$ is an integer. But clearly $2-N$ is not a prime, so we get a contradiction.
Edit: for the sake of clarifying, here is how I have interpreted the question: is there a polynomial $f$ such that, for $n\in\mathbb Z$, $f(n)$ is an integer iff $n$ is prime? I show that there is no such polynomial.
Edit 2: (in reply to Yves Daoust) Suppose that $f$ is a polynomial of degree $k$ which takes integer value at every prime. Pick any primes $p_1,\dots,p_{k+1}$. Consider polynomial $L(x)$ defined here for $x_i=p_i,y_i=f(p_i)$. Since every $\ell_i$ is a polynomial with rational coefficients of degree at most $k$, the same holds for $L$.
Now note $f-L$ is a polynomial of degree at most $k$ which is zero at at least $k+1$ points $p_1,\dots,p_{k+1}$, hence it must be a zero polynomial. Thus $f=L$ has rational coefficients.
Yes.
All polynomials of the form
$$Q(x)+\sqrt2\prod_{k=1}^n(x-p_k)$$ where $Q$ is a polynomial of integer coefficients, are integer only at the primes $p_k$.
Addendum:
The interpretation of the question as "the only integer values of the polynomial over $\mathbb R$ occur at primes" wouldn't make sense as for sufficiently large $n$, the slope of any polynomial exceeds $1$ and the integer values occur at arguments closer than $1$.