How do I get the value of the pixel under the cursor?

In QGIS you can use the "identify tool" that give you the value where you click (very left on the image, which comes from QGIS 1.6. It looks a bit different with other version). This work both for raster and feature layers. You can also use the shortcut "Ctrl+Shift+I". In QGIS 2.0, you can save the results using "copy attributes" at the bottom of the identify pop up.

You can configure the identify tool and select the layers that you want to identify. Note that in QGIS 3, you can also select the "table" or "graphic" views, which shows all layers in a similar way than the "Value Tool". The main difference with the "value tool" is that the "Value tool" continuously update while the identify tool needs a click (then remains unchanged until you click again).

The Value Tool should do what you want. Install it from the Plugins menu, then enable it: View -> Panels -> Value Tool.

N.

you can use point sampling tool in Qgis , it will extract raster values to point shapefile. you have to provide the point shapefile for sampling location.