How does the momentum operator act on state kets?
In my opinion, manipulations involving $\hat p$ and position bras and kets are most easily done by considering the action of $\hat p$ on the position bras, which is simply $$ \boxed{ \vphantom{\begin{array}{}make\\the box\\taller\end{array}} \quad\,\,\, \langle x|\hat p=-i\hbar\frac{\text d}{\text dx}\langle x|. \quad\,\,\,} \tag 1 $$
You can get this easily by seeing that for any state $|\psi\rangle$ with position-representation wavefunction $\psi(x)=\langle x|\psi\rangle$, the action of the momentum operator on the state gives a derivative on the wavefunction. That is, $$\langle x|\hat p|\psi\rangle =-i\hbar\frac{\text d}{\text dx}\langle x|\psi\rangle.$$ Since this equation holds for all states $|\psi\rangle\in\mathcal H$, you can "cancel $|\psi\rangle$ out". (More technically, since the action of the bras $\langle x|\hat p$ and $-i\hbar\frac{\text d}{\text dx}\langle x|$ is the same for all vectors, they must be equal as linear functionals.)
Looking at the question
I think my question boils down to: does $\hat p$ act on the basis kets $|x\rangle $ or on their coefficients?
one can safely identify that you're confused about something but it's harder to figure out what the question really is. So let me repeat some basic things here – I am confident that you must be confused about one of them, despite their basic character.
The symbol $\hat p$ is an operator. It means an object that acts on any ket vector and gives you another (or the same) ket vector. The map must be linear and so on. So $\hat p$ surely acts on vectors, not on "coefficients".
On the other hand, when it acts on a basis vector such as $|x\rangle$, the result may be expressed as a linear combination of the basis vectors in the same basis, $$\hat p |x\rangle = \int dx' f_x(x') |x'\rangle$$ with some coefficients $f_x(x')$. Every ket vector, and $\hat p | x \rangle$ is a ket vector, may be expressed using a basis in a way.
So while you may identify $|x\rangle$ with the "wave function" equal to $\psi(x'') = \delta (x''-x)$ where $x$ is a fixed value of the position, the acted-upon ket vector $\hat p|x\rangle$ is given in terms of the function $f_x(x')$ which encodes the coefficients in front of $|x'\rangle$. This function (storing the coefficients) is fully given by the meaning of the operator $\hat p$ and by the value of $x$ and it replaces the delta-function encoding $|x\rangle$ itself, so in this sense, operators also act on coefficients. One must just know the basic rules how they act on a basis etc. and then he knows everything!
Another thing you may be confused by is even more elementary, what a derivative is. A derivative is not an operator acting on the Hilbert space. A derivative is an operation that takes a function of a real variable and maps it to another function of the real variable $$ \frac{\partial}{\partial x} : f(x)\mapsto f'(x) = \lim_{\varepsilon\to 0}\frac{f(x+\varepsilon)-f(x)}{\epsilon} $$ You must be confused about this definition of a derivative, otherwise you wouldn't write the meaningless derivatives in the last sentence. Something should generally depend on the variable with respect to which we are differentiating, and then we differentiate it as a function using the general definition above.
The kernel (or "matrix elements") of $\hat p$ is $$\langle x | \hat p | x'\rangle = -i\hbar \delta'(x-x')=f_{x'}(x)$$ which is the derivative of the delta-function. It's a delta-function whose argument is the difference of the two values $x,x'$ that specify the bra vector and the ket vector between which $\hat p$ was sandwiched. The delta-function is equal to the inner product of the bra vector $\langle x |$ and the vector $\hat p |x\rangle$ which results from the action of $\hat p$ on $|x\rangle$.
The kernel is enough to calculate anything involving $\hat p$ and bra and ket vectors in the $x$-basis. For example, you may multiply my equation for the kernel above by $|x\rangle$ from the left and integrate over $x$. Then one gets (after noticing that $1$ was constructed on LHS via the completeness relation) $$\hat p |x'\rangle = \int dx (-i\hbar) \delta(x-x') |x\rangle = -i\hbar\frac{\partial}{\partial x}|x\rangle_{x= x'} $$ Sorry if there's a sign error anywhere. It makes sense to differentiate with respect to $x$ because the object actually is a function of $x$. If a general wave function is rewritten as a combination of such $|x'\rangle$ vectors from the LHS of the equation above, via an integral and with the coefficients called $\psi(x')$, the equation above becomes the usual $$\hat p:\psi(x')\to -i\hbar \psi'(x')$$ in terms of the coefficients. This doesn't mean that a linear operator is the same thing as a derivative of functions. It just says that in the $x$-basis, when acting on a general combination of these basis vectors, the coefficients transform in this derivative-like way. But that's a special property of this particular operator. Other operators, like $\hat x$, act differently.
There are many good answers already. This answer is basically an expanded version of Emilio Pisanty's answer.
Let us start by recalling the standard convention to write the position wavefunction $$ \psi(x)~=~\langle x | \psi \rangle \tag{1}$$ as an overlap with a position bra state $\langle x |$.
The CCR $$ [\hat{x},\hat{p}]~=~i\hbar{\bf 1}\tag{2}$$ is the first principle of canonical quantization.
The position Schrödinger representation $$\hat{x}~=~x , \qquad \hat{p}~=~\frac{\hbar}{i}\frac{\partial}{\partial x},\tag{3}$$ is the most common representation of the CCR (2), although it is far from being unique, cf. e.g. this Phys.SE post. However, see also the Stone-von Neumann theorem.
It's important to realize that it's implicitly understood that the operators (3) act on bras (as opposed to kets). (However, see eq. (6) below.) Hence it is more proper to write (3) as $$\langle x |\hat{x}~=~x\langle x | , \qquad \langle x |\hat{p} ~=~\frac{\hbar}{i}\frac{\partial \langle x |}{\partial x} ~=~\lim_{\varepsilon\to 0}\frac{\hbar}{i}\frac{ \langle x+\varepsilon |- \langle x |}{\varepsilon}.\tag{4}$$
Note that the position Schrödinger representation (4) on bras realizes the CCR (2) $$\begin{align}\langle x| & [\hat{x},\hat{p}]\cr ~=~& \lim_{\varepsilon\to 0}\frac{\hbar}{i}\left\{ x\frac{ \langle x+\varepsilon |- \langle x |}{\varepsilon} -\frac{ ( x+\varepsilon)\langle x+\varepsilon |- x\langle x |}{\varepsilon}\right\}\cr ~=~&i\hbar \langle x | ~, \tag{5}\end{align}$$ with the correct sign, as it should.
Note that the position Schrödinger representation (4) on bras and the convention (1) implies the standard formulas
$$\hat{x}\psi(x)~=~x\psi(x) , \qquad \hat{p}\psi(x) ~=~\frac{\hbar}{i}\frac{\partial\psi(x)}{\partial x}. \tag{6} $$
- Note in particular that if one insists to act on kets (as opposed to bras), then the position Schrödinger representation comes with the opposite sign: $$\hat{x}|x\rangle ~=~|x\rangle x , \qquad \hat{p}|x\rangle ~=~i\hbar\frac{\partial |x\rangle}{\partial x} ~=~\lim_{\varepsilon\to 0}i\hbar \frac{ |x+\varepsilon \rangle- | x\rangle }{\varepsilon}.\tag{7}$$