Is it theoretically possible to be killed with coin cell batteries?
A CR2032 holds about 2400 useful Joules of energy. Operating entirely within specification, it can supply \$200\:\mu\textrm{A}\$.
If applied subcutaneously, \$10\:\mu\textrm{A}\$ can be sufficient to cause fibrillation. So if you could apply the battery directly near the heart and underneath the skin, I suppose it could directly kill someone without any additional circuitry.
From an editorial in Anesthesia & Analgesia, June 2010, Volume 110, Number 6, International Anesthesia Research Society, pp 1517-1518, "Electrical Safety in the Operating Room: Dry Versus Wet," by Steven J. Barker, PhD, MD, and D. John Doyle, MD, PhD, FRCPC, the following quote is found:
Microshock refers to very small currents (as little as \$10–50\:\mu\textrm{A}\$) and applies only to the electrically susceptible patient, such as an individual who has an internal conduit that is in direct contact with the heart. This conduit can be a pacing wire or a saline-filled central venous or pulmonary artery catheter. In the electrically susceptible patient, even minute amounts of current (\$10\:\mu\textrm{A}\$) may cause ventricular fibrillation.
(The above information was pointed out in a comment here by Russell McMahon and is a substantial improvement over citing a Wiki page.)
I've read that death can be caused with as little as 50 Joules. But I think 100 Joules is a more certain estimate. Lethality (with AC, anyway) is pretty common even at \$200\:\textrm{V}\$. So if I had to make an educated guess, I'd probably guess that using a Sanyo OS-CON Aluminum-Polymer (way too expensive and you'd need lots of them) or aluminum capacitor (such as Vishay BCcomponents' Aluminum Electrolytics), with \$200\:\textrm{V}\$ and 100 Joules would be sufficient. This suggests a value of \$5\:\textrm{mF}\$.
However, it would take a while to achieve. Assume you can design a circuit that is, overall, 50% efficient in charging this capacitor from a single CR2032 while staying fully within specs and drawing just \$200\:\mu\textrm{A}\$ from it. Then on first blush it would take 10000 seconds or about \$2\:\frac{3}{4}\$ hours to charge it up for one such use if you could sustain \$200\:\mu\textrm{A}\$ throughout the process. But the CR2032 is only capable of sustaining about \$600\: \frac{\mu\textrm{J}}{\textrm{s}}\$ of power. So really, I think this would take closer to four days to achieve. (And that doesn't account for capacitor leakage. With the Vishay capacitor mentioned above, leakage power may be below charging power near the end, but it probably will add a fair bit more time to the process.)
So the answer is probably "technically, yes" but rather unlikely as it would be quite odd to open someone up in order to stick a button battery across some tissues inside their body near the heart (read: very low probability) and it is similarly unusual to find a circuit designed to charge up a large, low ESR capacitor from a battery supplying just \$600\:\mu\textrm{W}\$ continuous and requiring almost a week to charge up (read: low probability.)
Of course, now that someone is thinking this way, I am sure such a circuit will be promptly designed and then sold as pet rocks to millions of happy consumers, making this a significant problem in the wild. ;)