Why can I get a virus or trojan from visiting a website?

To answer how it's possible:

The site has some script that gets executed as you load the page. This is what infects your Windows PC - I'm not 100% sure of the details though, whether it downloads the code or just runs it. This page has an example of how it was done in one case. A vulnerable browser is also required, and virtually any popular browser is a vulnerable one as something running on a lot of computers is worth targeting.

It will be Windows PC's for the most part that get infected as people run as admin's rather than restricted users. The reasons for this are many and varied. As Roger points out in his comment its popularity rather than any intrinsic weakness that's the main factor here:

Windows is targeted more because it is more popular. Some say that Windows is less secure than alternatives too, but I have to say that in the way you highlight, it's not. I run Linux at home and if a trojan could run under my user account it could still do quite a lot of damage to files that I care about quite a bit, it just couldn't take over the system.

Though by running with a restricted rights user you can limit the damage, but not necessarily eliminate it.

With Vista and now 7 having tighter control over what gets run as admin you might start to see a drop in these sorts of sites - though it will only be when the majority are running the newer OS's.

Well, when you open a website, the website can direct the browser to do all kinds of things on its behalf. In the simplest case, it will just send text and pictures to be displayed, but it can also send small programs that the browser will run (in JavaScript, which browsers have built-in, or using a browser plugin such as Java or Flash).

In principle, getting infected just from visiting a website should be impossible:

While browsers will execute programs on behalf of websites you visit, these programs are carefully restricted so they can only do "harmless" things. From JavaScript, you can for example alter the page that the JavaScript belongs to (since both come from the same website, there can be no harm), but JavaScript may not change a page from a different website (so a sleazy site cannot alter your home banking display), and it may not directly access files on your computer.

Similar restrictions exist for most plugins (at least for Java and Flash). This is commonly referred to as sandboxing, as the code is essentially in a box of its own, isolated from the computer it runs on. In particular, it cannot read files on your hard disk, or start programs for you, like "regular" programs running on your computer can.

Now, the thing is: while in principle you're safe, in practice you may not be. The reason is that the sandbox system, like all programs, has bugs. Sometimes these bugs allow a program to "break" the sandbox and trick the browser or plugins to do things it should not be allowed to do. These tricks can be quite elaborate.

Examples:

• Like regular programs, the implementation of the browser or plugin can have buffer overflows, which can allow a website to run specially crafted code by sending it to the browser as input.
• There was a vulnerability in old versions of Sun's Java plugin with respect to the sandbox. The sandbox disallowed (and still disallows) access to all Java functions which could allow a program to do damage, such as reading or deleting local files. However, while the sandbox did correctly block access to these functions from a Java applet, browsers also allowed indirect access to these functions from JavaScript (via a technique called "reflection"). This "backdoor" had not been sufficiently considered by developers, and allowed to bypass the sandbox restrictions, breaking the sandbox. See https://klikki.fi/adv/javaplugin.html for details.

Unfortunately, there have been several vulnerabilities in the sandboxes of JavaScript, of Java and of Flash, just to name a few. It's still a kind of race between malicious hackers who detect these vulnerabilities to exploit them, and good hackers and developers who detect and fix them. Usually, they are fixed quickly, but there is sometimes a window of vulnerability.

BTW: The sandbox is the reason some Java applets pop up a "Do you trust this applet" warning on launch: These applets ask you to let them out of their sandbox and to give them access to your computer. This is sometimes necessary, but should only be granted with good reasons.

P.S.: The reason that ActiveX (at least early versions) were so horribly insecure is that ActiveX did not use a sandbox. Any ActiveX code from a web page had full access to your system. To be fair, this was (partially) rectified in later versions.

I mention this to answer your last question about preemptive actions. One not-so-common option is to use a virtual machine (well, it is common among security circles). There are a few free ones available. Install your OS, browser, and add-ins in the virtual machine and save the state. You can then browse to any site. When finished, you revert to that saved state and anything that happened in the virtual machine after that point is discarded. It's very simple once you get into it, but may pose a slight learning curve.

Note: Reverting state will literally discard any changes to the virtual machine; including browser history, cookies, updates, etc. In this case, you could revert to that state, apply updates, and save a new state. The same can be done for anything else you wish to keep. None of this affects your actual computer, only the virtual machine.