What's the purpose of capacitors in parallel

A capacitor can contain a certain amount of charge for a given voltage:
$$Q = CV$$ When you have more than one capacitor in parallel, they have the same voltage (because they are in parallel), and each stores a certain charge. The total charge (at a given voltage) will be the sum of the charges on all the capacitors.

Now if you have a certain load (for example a resistor in parallel with the capacitors), that load will draw a particular current (charge per unit time). If more charge is stored (because the capacitance is greater), then the voltage will drop less per unit time. This means that if you have a bridge rectifier, like in your diagram, and you have a certain load (not shown in your diagram), then the "ripple" on the power supply will be less if the capacitance is greater.

The basic effect is shown in this diagram:

You can see the AC signal, the (bridge) rectified signal, and the signal after the capacitor (with a certain current being drawn). As the capacitor gets larger, the amount of voltage droop will be smaller (the slope of the green curve will be less if the capacitance is greater as the capacitor can provide more charge / current without the voltage decreasing).

Incidentally, sometimes people will put capacitors of different types in parallel. For example, a large electrolytic capacitor (1000 µF) and a small ceramic capacitor (100 nF). This is done because "real" capacitors have a series inductance - and in the parallel case, the small capacitor (which has a smaller inductance) will be able to respond quickly to rapid changes in current, while the larger capacitance will take care of "longer term" current demands. This is sometimes called "supply decoupling". It's probably outside of the scope of your current question, but a very important principle in electrical engineering.