What are the pros and cons of doing PhD study at a big vs. small department?

The main disadvantage of being a student in a big department is that the professor can not actually provide in-depth mentorship to that many people: your "real" advisor will most likely be some post-doc instead. The quality of post-doc advice is more variable since they haven't had as long as a professor to prove and improve themselves. (This one bit me personally, and I switched from a largeish department to a small one where I like it much better.)

On the plus side, a big team will likely have several different funding sources, so if your main grant runs out but you don't have enough work to graduate yet, it'll be easier to find you another contract to tide you over.

Big department

Pros:

1. More funds raised.
2. More faculty members and students to interact, learn and collaborate with.
3. Possibly more equipment and space for your work.
4. More publications which benefit the department as a whole in terms of rankings and securing additional funds.

Cons:

1. Lesser chances of valuable one on one interaction with your supervisor since he/she will have to take care of a lot of students.
2. Relations are usually more impersonal as there isn't enough time or opportunities to interact with everyone properly.
3. May be restricted by goals of senior members of the group since some may require you to play a part in their work as well.

Small department

(Here I am assuming that the department is small out of its own limitations and not by choice like it seems to be with Caltech.)

Pros:

1. More personal relations with the faculty members and fellow students.
2. Motivation to grow and expand to compete with other bigger departments.
3. You may be asked to participate as a co-author on grant proposals etc.

Cons:

1. Funds may be limited.
2. Not enough in-house expertise in the subjects you are interested in.
3. Limited peers to interact with.
4. You may have to build your own contacts from researchgate-like forums. You may also have to out conferences and seminars on your own.

These are some I could think of from looking at my current and past department. I have also read some blogs regarding this. If you are interested in reading experiences of established researchers, search for 'researchwhisperer', 'thesiswhisperer' and blogs of that genre on the interweb.