What is the simplest way to interact with an I2C peripheral?

In my experience the easiest way is the Bus Pirate, which is also a cheap alternative: http://www.seeedstudio.com/depot/bus-pirate-v3-assembled-p-609.html?cPath=61_68

You can find a good tutorial for it: http://dangerousprototypes.com/bus-pirate-manual/i2c-guide/

And people in forums such as this is are familiar with it. For hobbyists, bus pirate is the way to go.

The are plenty of USB to I2C converters around, this would maybe be the "least painful" way to connect to your peripheral. Here is an example:

This one is a bus master, and seems to have half decent documentation and some sample C# code.