Drupal - What are costs/benefits of integrating Phing, Jenkins together with Drupal?

Ok, as for being a fan myself of continuous integration, I can assure you virtualisation is very handy to control the testing environment. I have built myself a CI setup, forking off from a wonderful package called quickstart. This package depends on Virtual-Box or other VM software, it is basically a Ubuntu made for Drupal with a pre-set up lamp architecture, pre-configured debugging and editors, profilers, etc.

I have installed Jenkins and phing on top of quickstart, and can assure you it is quite low cost($0), quite low hassle. Any one can co-opt in with me on most any project quite easily, the whole dev environnement is reproduced and it comes with quite a few tools by default, and quite a few I have myself upgraded.

Now, the way we use jenkins is most certainly not pure CI, it just run the tests when we commit, then we shut down the thing and forget about it. It could be used for notifications and documentation generation, and on some projects I do generate some code metrics but the main thing I also cared about in my setup was testing. One clear advantage when using a vm software for testing is it can easily be reproduced used to run the tests elsewhere or even on different systems and configurations, you always have control over the environnement (memory, processor usage, peripheral ressources, etc..)

In most projects I contributed on, testing is done before staging I guess this may be different depending on the project managers.

PS: Another thing to consider, in a true testing environnment you would tend to try and reproduce your server setup and load as much as possible. Our projects are hosted on Cent-OS and obviously we don`t quite follow up to the rules on this aspect either, quickastart is Ubuntu based. So far have not had any problems, and as a bonus has many other features pre-configured for rapid Drupal application prototyping and developpment.

Happy CI'ing!