How to automatic adjust color and contrast

A simple linear way of performing "auto-contrast" is to linearly stretch and offset the image intensities. The idea is to find the stretch (contrast) and offset (intensity) correction parameters such that in the corrected image the 5th percentile will be mapped to 0, and the 95th percentile will be mapped to 255.

Check out my answer here for more details.

Just as an alternative to the Adi Shavit's answer there is also a nonlinear algorithm for the image contrast enhancement described in a paper http://www.asp.eurasipjournals.com/content/pdf/1687-6180-2014-70.pdf. Someone may be interested in it.