Simple and safe solution to control a power plug with Arduino or PC

I would suggest two options.

First - and this involves a small amount of wiring and soldering - using a PowerSwitchTail II. This is a very safe way of switching mains voltage. There is adequate isolation, it is safely cased and tested.

If you really want to do no wiring or soldering at all, then I would suggest finding one of the common 434/868/915MHz (depending on region) remote controlled sockets using a basic OOK protocol, and buy a simple 434MHz transmitter. A number of the protocols are documented in various places. The only issue with these is that you have no sure way of knowing if the switch is on or off.

This is how you control a relay. A relay gives you proper isolation from the mains, and with the appropriate relay you'll be able to switch more than a kW at 230V.

The relay will need more current than the Arduino can supply. That's where transistor Q1 comes in. The transistor multiplies the input current from the Arduino (the "on/off" at the left) by about a factor 100 and uses that higher current to activate the relay.

edit
An SSR (Solid State Relay) is even simpler, but usually a bit more expensive. (This one seems to be very low priced, at USD 3.50 at Digikey.)

This SSR connects via a resistor to the Arduino, that's the two left pins, and the other pins are the switch. This SSR can switch up to 1A.

How about this:

You connect a 5V power supply, and the 4 inputs to 4 of your Arduino's outputs, and you're done. USD 16.50.