Why do we use polarized capacitors?
The physical size of a capacitor is a function of the thickness of the dielectric (among other things).
Early on, it was discovered that the oxides of certain metals (aluminum and tantalum in particular) made good dielectrics, and could be made very thin through a chemical process — orders of magnitude thinner than other dielectrics such as waxed/oiled paper and plastic film. Therefore, the electrolytic capacitor was invented to provide high capacitance in a reasonable volume.
Unfortunately, the chemical process requires that the voltage across the capacitor must have only a single polarity, so these capacitors are "polarized". Reversing the polarity degrades and eventually destroys the oxide layer. It's something we just have to live with in order to take advantage of this technology.
The ability to produce high-value capacitors in nonpolarized technologies such as multilayer ceramic means that it is now possible to use them where only a polarized capacitor would have been previously available. There is generally no problem with making this substitution, although you may need to consider some of the quirks of the technology you're switching to.
For example, some high-K (high dielectric constant) ceramics exhibit significant capacitance changes with voltage. This might be acceptable in a coupling or bypass application, but completely unacceptable in a filter design.
Can I use a non-polarized capacitor with the same voltage and capacitance instead of these polarizing capacitors?
Electrically speaking, non-polarized capacitor is always better than a polarized one. Yes, you can always replace with a non-polarized capacitor with exactly same rating.
But there is an assumption hidden here:
Provided you can find one that's physically small enough to fit on your board and cheap enough to fit in your budget. And the fact that you can't is the only reason we use polarized caps.
I assume that, if we ever learn to make non-polarized caps that are as cheap and dense (capacity-per-volume) as electrolytic ones, the polarized capacitors will vanish.
Side note - voltage and capacitance are not the only electrical parameters of a capacitor. They would suffice for an ideal capacitor, but real world brings other, ugly metrics. Like ESR, capacity coefficient with temperature or voltage, frequency response, etc. Circuits designed around quirks of particular tech can fail if the substitute differs there. Even being too good can cause trouble, eg. high-ESR caps naturally keep peak current in check so substituting with a theoretically superior low-ESR part can cause the whole thing to blow up. Adding ESR is trivial - but that's no longer a drop-in replacement, but rather a circuit redesign. So we don't replace electrolytics with something else not because polarization is important, it's just a nuisance. We keep them because of many other parameters, less obvious than C, V and polarization.
Since you mention protection I'll add that polarized caps should not be used for reverse polarity protection. They will react on a reverse voltage very slowly (seconds or minutes), while typical sensitive components which are worth protecting will be dead within milliseconds. And once a polarized cap starts to absorb the reverse voltage, it may vent, explode or catch fire which (apart from the obvious problem with smoke and fire) can make it non-conductive again, exposing your circuit to the reverse voltage once more.