A computer that fails to boot if certain USB devices are plugged in
Here's some insight.
Some BIOSes are buggy. A good BIOS should skip over invalid or unknown devices but not all of them do for whatever reason. Try updating your BIOS. I will add that to me, it seems when PCs started gaining the ability to boot off of things other than a floppy or locally attached hard drive, weird bugs have been present on particular BIOSes and systems.
Some USB drives I suspect don't implement the USB mass storage standard 100% correctly. This would usually be a problem with MP3 players that look like hard drives, devices that actually present two storage devices to the system (I had an MP3 player that presented its internal and external storage as two USB mass storage "subdevices" and I always had to make sure it was unplugged during boot).
Some devices may be flaky and be causing the bus to hang or other weird behavior. USB drives are cheap and it's likely no one cares about QC'ing them against every possible hardware combination. It's also possible the motherboard chipset may have idiosyncracies that only shows up with specific devices.
So much so that when I boot, I try to make sure it doesn't have these devices plugged in.
This is what I do with pretty much any system, IMHO it's a best practice.
You could try checking the boot order of the BIOS in question. If a USB device is listed before the HDD or CD drive, then the computer will attempt to use it to boot up. If there is no bootable partition on the USB device, then you may get a blank screen or some kind of "Boot record not found" type message.
Apart from the boot order, there may be some compatibility issues between the motherboard and USB device. If the motherboard is unable to recognise the device, or only looks at a certain port, then this would explain the apparent inconsistent behaviour.