Why heat equation is not time-reversible? (Time arrow in mathematics)

The heat equation $u_t-u_{xx}=0$ is not time-reversible because it involves an odd-order derivative of $t$. Under time reversal $t\mapsto-t$, we get $u_t\mapsto-u_t$. So if $u(x,t)$ is a solution to the heat equation, then $u(x,-t)$ is a solution to a different equation, namely $-u_t-u_{xx}=0$. The only way for $u$ to solve both equations is if $u_{xx}=0$ everywhere, which is not the case for most initial conditions of interest, such as your $\delta(x)$.

By contrast, the wave equation $u_{tt}-u_{xx}$ is time-reversible because it involves only even-order derivatives of $t$. Under time reversal $t\mapsto-t$, we get $u_{tt}\mapsto u_{tt}$. So if $u(x,t)$ is a solution to the wave equation, then $u(x,-t)$ is also a solution to the wave equation.

If you start with a spatial heat distribution $u(x)$ at $t=0$, then the solution at $t > 0$ is infinitely-differentiable because it is averaged by a Gaussian, and the Gaussian couldn't be more smooth. You shouldn't expect to be able to undo such a process. This is a diffusion equation, and the speed of propagation is, essentially, infinite, which makes it physically unrealistic. Even with added viscosity, the process is irreversible.

If you solve the heat equation numerically, then you can ask how the solution at a point on the grid depends on time. As you begin to answer this question, you find yourself looking at the paths back to the initial data, through the numerical algorithm, which can be thought of as the density of paths that refer to the initial data. In the limit of large number of grid points, this begins to look like a central limit problem and, hence, the Gaussian weighting of the initial data. This is definitely not a process you'd expect to reverse. Brownian motion can be used to look at this process of solution.

Heat flow is a diffusion process, and the averaging should not be expeced to be reversible. Entropy is increasing.