How to create multiboot flash drive

You do need to add the respective entries to the menu.lst file. That menu.lst already has the relevant command for Ubuntu 10.04, you just need to place the ubuntu-10.04-desktop-i386.iso file in the root folder of the USB drive.

I doubt that Windows would be able to boot from an ISO directly. And even after extracting it, I have no idea about how to set up grub to start the Windows installer.

PenDrive Linux takes care of windows and ubuntu for you natively. You just have to boot the the flash drive and select the .iso that you wish to boot to. If you are wanting to manually install an .iso that is not supported by pendrive's multiboot, then you'll need to add an entry manually. You edit this file similarly to editing a grub menu file.

There are many articles on www.rmprepusb.com which show you how to add Windows Installer ISOs to a grub4dos USB drive - e.g. #43. However, I would recommend Easy2Boot as a multiboot USB solution.