Is it fair for me to drink caffeine during tests?

That would be totally acceptable. Remember, you are also in a totally different state if you went to the toilet right before the exam. If you drank a litre of water in the morning. If you ate something for breakfast that happens to contain lots of short-chained carbohydrates, providing lots of energy for your brain.

You have to draw the line somewere and coffee is definitely acceptable, if not expected.

Edit: As several comments suggest, your other question about rudeness because of smelling beverages: that's different from course to course, and you will probably have to employ some degree of common sense there. People are usually not bothered by coffee smell as opposed to french fries or the sounds of eating potato crisps. If in doubt, just ask your TA or fellow students before going to the exam.

You might find that your university policy prohibits any beverages except water during exams. Coffee has a strong aroma that can be distracting, especially to the less-awake students who might then crave coffee.

Even if coffee had no aroma, it would still be distracting. A cup of tea or coffee is usually pretty identifiable by sight. If someone walks into my office with a mug of green tea, I usually want a mug of tea myself long before I can smell it. "That guy 20m away is drinking some unidentified hot beverage. Now I want coffee!"

Sure, the consumption of coffee beforehand should be fair and allowed in any country where it's a legal and commonly consumed drug - chances are many classmates drink it too. You still have to have learned the material in order for the coffee to help you remember it. It's just not fair to be distracting to others during a test.

Working in computer science and looking at the consume of caffeinated beverages in the department I'm wondering: Is it possible to work without? Seriously, you are legally allowed to buy this stuff and as long as it is not forbidden by any rule of your university it is totally fine to do so (edit: As long as you fit 'social norms'). Some people will eat dextrose/glucose or do other things they think will help them. Everybody has his/her own ritual I guess.

Taking other drugs to be able to learn better is a whole different story but here I don't see any problem as long as your university allows it. So it is neither rude nor any form of cheating.