Finite additivity, atomlessness and countable additivity
Here’s an example.
Define an equivalence relation $\sim$ on $\wp(\Bbb N)$ by $A\sim B$ iff $A\,\triangle\, B$ is finite, where $\triangle$ is symmetric difference, and let $\mathscr{B}=\wp(\Bbb N)/\sim$. For $A\subseteq\Bbb N$ denote by $[A]$ the $\sim$-equivalence class of $A$. Let $\mathscr{U}$ be a free ultrafilter on $\Bbb N$. Note that for any $A\subseteq\Bbb N$, $A\in\mathscr{U}$ iff $[A]\subseteq\mathscr{U}$. Now define a $\{0,1\}$-valued measure $\mu$ on $\mathscr{B}$ by $\mu\big([A]\big)=1$ iff $A\in\mathscr{U}$. Then $\mathscr{B}$ is atomless, and $\mu$ is finitely additive. However, $\mu$ is not countably additive, since it is possible to partition $\Bbb N$ into countably infinitely many infinite sets, none of which is in $\mathscr{U}$.
Added: Michael Greinecker has pointed out that I’m using a notion of atomless that may be different from the one intended by Seamus. Here’s another example that may be preferable.
Let $d:\wp(\Bbb N)\to[0,1]$ be asymptotic density, and let $\mathscr{U}$ be a free ultrafilter on $\Bbb N$. For $A\subseteq\Bbb N$ let $$\mu(A)=\mathscr{U}\text{-}\lim_n\frac{|A\cap\{1,\dots,n\}|}n\;.$$ (For basic information on $\mathscr{U}$-limits see this answer by Martin Sleziak.) Then $\mu$ is a finitely additive non-atomic probability measure on $\wp(\Bbb N)$ such that $\mu(A)=d(A)$ whenever $A$ has an asymptotic density.