What's the difference between INPUT and INPUT_PULLUP?

The default is INPUT which sets up the pin as an INPUT. If the pin is totally disconnected, it will randomly read HIGH and LOW. If you tie it to +5V or 0V, it will obviously read HIGH or LOW.

Internal to the Arduino, the Atmega chip has internal pullup resistors with a value around 20k ohm. (See the DigitalPins documentation for more details) These resistors can be optionally connected internally using INPUT_PULLUP. This is functionally (and electrically) equivalent to connecting a ~20k ohm resistor between the pin and +5V, the only difference is that it requires no external components and you can turn it on and off in software during the execution of your program.

So why pull-ups and not pull-downs? There are likely several reasons for it, but when wiring buttons or switches or anything "normally open", you only have to tie them to ground, you don't need to run +5V out to them. Since most boards are going to be designed with large ground pours for shielding reasons anyway, tying to ground is practically reasons.

Some more featured ICs like ARM chips have both pull ups and pull downs, but the 8-bit AVR line only comes with pull-ups. You just have to remember that HIGH is "open" and LOW is "closed".

Note that previous to 1.0.1, you could turn on pullups by using digitalWrite(). And you still can.

The code:

pinMode(13, INPUT);
digitalWrite(13, HIGH); // Turns internal pull-up on
digitalWrite(13, LOW);  // Turns internal pull-up off

This is an important distinction because INPUT_PULLUP obviously turns on the pull-up resistor. However, less obvious, is that starting with 1.0.1, simply calling INPUT forces the pull-up to be turned off. (Previously, the state of the pull-up stayed the same).

To see how a pin operates with and without pull-ups, watch the following videos.

Floating pin on an oscilloscope: http://www.youtube.com/watch?v=dBIBFLYCjMM

With Pull-Up enabled: http://www.youtube.com/watch?v=SAIw7LLVl-U

Full tutorial video on Pull-Ups (note that I made this before INPUT_PULLUP was added to the Arduino library): http://www.youtube.com/watch?&v=jJnD6LdGmUo