Chemistry - Computing two-electron integrals with an STO-3G basis set
Solution 1:
Actually there is a mistake in the analtical expression in Cook's Book. On his web page he has a pdf with the corrected verison
http://spider.shef.ac.uk/
Maybe this solves your problem, but I would also recommend to implement the Obara-Saika Scheme or rys-Quadrature since they are really much more efficient. If your are programming in Python, you might have a look at the PyQuante project, which implements all this stuff. Concerning Obara-Saika you might also read about the Head-Gordon Pople Scheme. It is in principial an adapted version of Obara-Saika which reduces the FLOP count.
Solution 2:
My advice is to implement the Obara-Saika recurrence formulae that are outlined in "Molecular Electronic-Structure Theory" by Helgakar, et al. I would stick with Cartesian functions, since a) they are easier and b) spherical harmonics don't matter for molecules anyway.
I did this years ago (in Mathematica) when I was in a similar place -- having completed Szabo and Ostlund and wanting an arbitrary "angular momentum" code.
The book will cost you a small fortune, but if you can borrow it through Interlibrary Loan, it is a phenomenal text.
Solution 3:
My suggestion would be to use another existing code and run the calculation.
For example, if I do an HF/STO-3G calculation on $\ce{H2O}$, I get:
$$E_\mathrm{H_2O}=-74.9659011\:\mathrm{a.u.}$$
I don't have Cook's book on hand, so I can't look for the error, but I'd suspect some error as you do.