What are the differences between shift register ICs?

The easiest way to answer a question like this is to look at the datasheets for the components:

• CD4015 is part of the older 4000-series range of chips. Back when they were introduced, they were CMOS while 7400 chips were TTL, although nowadays 74HC-type chips are also CMOS. They still see some use because they work with a wider voltage range than 74HC chips (up to 15V, versus 7V maximum for 74HC or 5.5V for 74LS). They're also somewhat slower (maximum 3MHz at 5V, versus 25MHz for the 74HC595).

• CD4014 has similar specifications to the 4015, but rather than having pins that let you take all of the values that have been shifted in out at once, it lets you put multiple values in at once and then shift them out one at a time. So the CD4015 is like a serial to parallel converters, but this one is a parallel to serial converter.

• 74HC166 is parallel-in serial-out like the CD4014, but is in the 74HC range so has the smaller voltage range and faster reponse of that range.

• 74HC165 allows both parallel and serial in, and is serial out. It also provides an both inverted and non-inverted output.

• 74HC164 is serial in and parallel out, like the CD4015, but is 74HC series so faster and lower voltage.

• 74HC595 (or more exactly, SN74HC595J) and 74HC595-SMD (which could be a number of different minor variations) are the same component in different packages. The first is a traditional "DIP" package, which is likely what you want if you're working on breadboard, stripboard, or perforated prototype boards. The later is a surface mount package (probably SOIC) which is smaller and easier to solder to a PCB, but can be a bit of a pain for prototyping. These are serial-in parallel-out, but they also have a separate set of registers that the data being input can be copied to. This means that your parallel outputs can be made to change simultaneously, rather than having invalid data in them while the new data is being shifted in.

Some other chips that you may want to have a look at:

• As mentioned by @supercat in the comments, the CD4094 is useful when you need to control more than 8 output lines because it makes cascading the output from one chip to the next easier. The 74HC4094 is a chip with the same behaviour and pin layout but using the 74HC voltages and faster speeds.
• TLC6C5912 is a 12-channel serial in parallel out chip that's specifically designed for driving LEDs, and can handle LEDs with voltages and currents much larger than any of the above.
• TLC5911 is a monster of a chip, but it controls 16 LEDs, and has a constant current driver for each one that can be individually controlled to one of 128 levels, i.e. you can use it individually dim each LED, by shifting in 7 bits of brightness information for each one rather than just 1 single on/off bit. Useful for signs that display images/videos.

For a newcomer, the main distinction in shift registers is probably parallel in/serial out (PISO) and serial in/parallel out (SIPO).

As the names suggest, a PISO takes in, say, an 8-bit-wide signal, and lets you individually shift those bits out, one at a time, (in series) with single clock pulses.

A SIPO lets you shift each of the bits in sequentially, then have all, say, 8 of those bits be present on 8 output pins simultaneously, i.e., in parallel.

To add to the answer of schadjo:

The two most commonly used for Arduino (but not only) are 74HC165 and 74HC595.

The 74HC165 can be used to connect upto 8 inputs (e.g. switches) to just a few GPIO's.

The 74HC595 can be used to connect upto 8 outputs (e.g. LEDs) to just a few GPIO's.