Why have a small neon lamp in power supply input circuitry?

It is used as discharger for overvoltage conditions - in case of overvoltage a discharge starts through the lamp and that protects the main circuit from overcurrent.

A neon lamp is used because it is relatively cheap, very reliable and there's zero current through the lamp until the discharge actually starts.

Another perspective would be for safety during troubleshooting. From the venerable Bob Pease:

So, whenever I start work on a high-voltage circuit, I tack in a neon lamp in series with a 100k resistor across the high-voltage busses. Then when I see the neon's glow, I'm graphically reminded that this really is a high-voltage circuit, and that the power is still ON (I don't care what the power switch says) and I should revert to the mode of High-Voltage Cautions. If I grab onto a really hot wire, the shock might not injure me, but I might convulse and jerk backwards. That's not a good idea if I'm standing on top of a ladder, for instance. So, looking for the glow of a neon lamp is a way to remind me to be serious, and I recommend it for you, too.

From http://electronicdesign.com/electromechanical/whats-all-reflex-response-stuff-anyhow

On some old electronics (especially during vacuum tube era) the neon bulb was put in parallel with a fuse and used as a blown fuse indicator. Other times it was merely a power applied indicator (pilot lamp).