How to choose an IC to use with your project?

I'm going to answer this question from a very specific point of view: an electrical engineer designing something for volume production. And I define "volume production" as 1,000 to 10,000 units per year, and in production for up to 10 years. I know that this doesn't apply to many of the people here, but I think it should be interesting and informative none the less.

There are two parts to this: finding a selection of parts that would work and picking the correct part from that group.

For finding the right part I uses a variety of sources (in order of decreasing importance): My own knowledge of manufacturers and parts, information from manufacturer reps/Distributors/FAE's/Etc, searches of manufacturer web sites, and random things read from trade magazines.

For picking the right part I have to weigh many factors, including (in no particular order): cost, availability, second sources, what we use for other projects, how long we think that chip will be manufactured, how useful the rep/distributor/FAE is, and of course the performance/features of the chip itself.

Let's say that I have to design an MP3 player. For that I need a CPU, Flash, USB interface, and an audio output (I'm ignoring power).

The CPU should be a 32-bit CPU with interfaces for NAND Flash, an audio DAC, and USB interface. I would first narrow it down to a CPU architecture and a couple of manufacturers. For ARM, I would mainly look at TI. Freescale makes ARM stuff, but I hate their development tools so they are out. Analog Devices has a DSP with integrated audio DAC that could also be worth looking at. But let's go with TI. TI's web site has a nice CPU selection guide that would narrow it down to maybe 4 or 6 chips. Using that, and by talking to TI's FAE and distributor I would select one from the 4 or 6.

Finding NAND flash is somewhat easy. There are only about 4 manufacturers worth talking to, and 2 of the 4 don't work well with these "low production volumes". I'd just email the local rep/FAE/Disty with my specs and have then recommend parts. From that I would select the part to use.

Almost all of the USB interface circuitry should be in the CPU itself. Any external circuitry will be documented in the CPU datasheet and application notes. I'd just copy that. Any critical components would have been selected for me.

Reasonable audio DACs are only made by 4 companies: Cirrus Logic, TI, AKM, and Wolfson. Since this application isn't critical, any manufacturer is just as good as any other. Since I already use a Cirrus Logic DAC in another project, I would use that same part for this one. But if I had to choose, I'd search the mfg web sites to narrow it down. Then, getting quotes and opinions from the various distributors would give me the final info.

The audio output also needs some analog parts (mostly op-amps). Experience tells me which ones to consider, but there are lots of stuff on the various manufacturer web sites to help with this. Also the FAEs could be a huge help here. Honestly, choosing an op-amp might take longer than selecting the CPU! It might make sense to choose several op-amps from different mfgs that have the same package and pinouts-- so we can try different chips in the prototypes and select the best one.

From start to end, the parts selection process for this MP3 might take 2 calendar weeks. 3 weeks if you add the power supplies and battery charger. Most of this time is playing email tag with the various people, so other valuable work happens during this time. This sounds like a long time, but a mistake at this point could cost tens of thousands of dollars. It's worth taking the time.

I think everyone probably has these thoughts at some point.

There are books/sites which recommend a bunch of "useful components" to have available. The only trouble is these things go out of date very quickly. For example the 741 and PIC16F84 are still being recommended in places even though they have both long (long long) since been surpassed.

If you know what is needed spec wise for your project (as you should do) then you can pick the components based on the specs. For example if you need 10MHz analogue bandwidth and you are using a 5V supply then you can filter opamp results accordingly. What speed does your uC need to run at 10MIPS? 40MIPS? what peripherals does it need? USB? SPI? CAN? Filter accordingly. Parametric searches are your friend here (all the decent sites like Mouser, Farnell, Digikey, etc have them)
This applies to just about everything, so there's no quick way around reading through datasheets and making an informed decision (unless you already used the part of course)

So I would say the answer is probably to jump in head first and start learning how to use the search tools properly (look at any help available) picking your way through the components, read the datasheets, learn about what the specs/graphs/etc mean.

I certainly remember being overwhelmed by all the different options and long winded datasheets, but after a while your brain gets pretty good at filtering out the info you need quickly. Still takes time though, and is not the most enjoyable task.
Of course, if you're sure of the specs and you can get someone else to do it for you that's always a good option ;-)
Keeping up to date with new chips is useful too, I sign up for all the new product notifications for various places.

As others said, for one-off designs, it is not really worth spending a lot of time to absolutely minimize the cost of your design. And if you are, for example, already familiar with a particular uC family, then it often is perfectly valid to just stick with that family if it will do the job.

But if you are truly confronted with dozens or more of parts that meet your needs, then sorting by price and taking the lowest-cost one is a reasonable approach. At the very least you will gain some familiarity with a relatively low-cost part that you might be able to use on a future project where cost does matter.