Develop games with pure actionscript or adobe flash cs4 professional

Man are you on the verge of a brave new world, ha.

Alright, so basically everything that you're mentioning is a viable option. A lot of it is going to come down to personal preference, honestly.

If you are a very visual person and it helps you to see things on the stage as you control them, go with Flash. It's not quite as powerful as Flex when it comes to dealing with data, but allowing you to link up your classes to sprites and movie clips in your library is huge and honestly underrated by a lot of "serious" developers.

If you're going for multiplayer, you might consider looking into Adobe LiveCycle Collaboration Services. It's a hosted solution from Adobe that takes a lot of the work out of setting up a multiuser environment, though it's not free.

Finally, if you can wait a few months, Flash CS5 will allow much tighter integration between FlashBuilder (the upgrade to Flex Builder) and Flash. Basically you'll be able to use both together very easily.

If you want to get started learning this stuff, I strongly recommend Lee Brimelow's excellent site gotoandlearn. The first few on there are AS2 so feel free to skip them, but just watch all of his AS3 examples in order and when you're done you'll be a flash developer.

You can also check out the Adobe Flash/Flex Cookbooks to look up specific questions, or google "Flex in a Week" to learn flex.

Good luck!