Can the light at the bottom of an optical mouse damage your eyes?

Unless the mouse is labelled as a class 2 laser device, then it will not cause damage to your eyes. Most (possibly all) mice are either not lasers or class 1 laser devices, which are safe under all conditions, and do not require labelling.

If you want to know a lot more detail, there is a company called Broadcom Inc. (formerly Avago) which owns lots of patents in the this area. They have produced a detailed document covering aspects of safety with regard to optical and laser mice [PDF].

Well, I did spend some time experimenting with this few years ago.

I noticed that after spending several minutes staring directly at mouse LED at close range (as in 1-2 cm), the way I perceive colors changes. It resets back to normal after few minutes of not staring into the mouse, and as far as I can see, there was no permanent damage.

It will deplete the pigments in the cells of your retina where the light falls, but it doesn't contain enough energy to actually cause any lasting damage.