Why does the action have to be hermitian?

The classical action (of particles or fields) has to be real, because this means a real classical Lagrangian. This is needed because (canonical) momenta are obtained (for instance for a particle) from the Lagrangian by $p_i = \frac{\partial \mathcal {L}}{\partial \dot x^i}$ , and momenta are real.

In QM or QFT, the action has to be understood as a phase, more precisely the probability amplitude to an history $H$ to happen is (in $\hbar = 1$ units) :

$\psi_H = e^{iS_H}$

where $S_H$ is the classical action of the history $H$.

To have the total quantum probability amplitude, you have to sum over all possible histories :

$\psi = \int [DH]\psi_H = \int [Dh]e^{iS_H}$

So, the action does not correspond to a measurable quantity, it is a unobservable phase, it is not an observable.

If the action was an observable, you should be able to select a particular history among all histories, in some sense, extract a specific classical behaviour from a quantum behaviour, which is not possible.

In quantum theory, the S-matrix is unitary in order to preserve the concept of probabilities adding up to 100%. Unitarity is implemented via an appropriate reality constraint/condition. E.g. in the path integral formulation $$Z~=~ \int {\cal D}\phi~e^{\frac{i}{\hbar}S},$$ the action $S$ should be real-valued.