Is the force of gravity always directed towards the center of mass?

No. For example, the gravity of a cubical planet of uniform density, which can be computed analytically, is not directed towards its center (or any other single point).

You can also imagine a dumbbell-shaped mass distribution where the two heavy ends are very far apart. If you drop an apple near one end it is going to fall toward that end, not toward the middle of the “neck”.

No. It is not correct.

Consider this ridiculously contrived counter-example... Three spherically symmetric bodies (or point masses if you can tolerate this) are at the three vertices of a 45°, 90°, 45° triangle, ABC. The masses of the bodies are: $m_\text{A}=m,\ \ m_\text{B}=M,\ \ m_\text{C}=2M$. Regard the bodies at B and C as a single body, BC; join them, if you like, by a light rod.

The centre of mass of body BC is at point P, $\tfrac23$ of the way between B and C.

But the pull due to BC experienced by $m$, at A, is not directed towards P, as one can easily show by vector addition of the forces due to B and C. [In this case the forces are of equal magnitude, so the resultant bisects angle BAC and clearly doesn't pass through P!] The reason for the discrepancy is the inverse square law of gravitation.