Why doesn't a neon sign seem that hot?

There is a difference between temperature and energy.

Plasma is, as you said, very hot - but there isn't very much of it. The density of plasma in the tube is very low. So when it does hit the walls of the tube it transfers very little energy. So the mass of the glass tube increases in temperature only very slightly.

It's like a firework sparkler, the sparks are at 2000degC but they are very small, have very little mass and contain very little energy - so when one lands on you it transfer much less energy than a hot cup of coffee at 80deg C.

As you suggest in your question the contents of the tube are not in thermodynamic equilibrium. At any moment in time there will be a small number of fairly energetic electrons and a large number of low energy neutral gas atoms. The neutral atoms have a temperature of around room temperature.

I'm not sure how much sense it makes to assign a temperature to the electrons because their motion is along electric field lines and isn't random, but I suppose you could see it as a small number of hot electrons heating a large number of neutral atoms. Viewed in this way the average temperature would be around room temperature not the millions of degrees you normally associate with a plasma. An energetic electron colliding with the wall of the tube would heat it, but the low energy neutral atoms would immediately cool it again.

Technically the contents of the tube are a plasma because free electrons and ions are present, but the percentage of atoms that are ionised is miniscule. It isn't a plasma in the sense that the gas in the Sun is a plasma.