How can I test a TCP connection to a server with C# given the server's IP address and port?

You could use the Ping class (.NET 2.0 and above)

    Ping x = new Ping();
    PingReply reply = x.Send(IPAddress.Parse("127.0.0.1"));

    if(reply.Status == IPStatus.Success)
        Console.WriteLine("Address is accessible");

You might want to use the asynchronous methods in a production system to allow cancelling, etc.

Assuming you mean through a TCP socket:

IPAddress IP;
if(IPAddress.TryParse("127.0.0.1",out IP)){
    Socket s = new Socket(AddressFamily.InterNetwork,
    SocketType.Stream,
    ProtocolType.Tcp);

    try{   
        s.Connect(IPs[0], port);
    }
    catch(Exception ex){
        // something went wrong
    }
}

For more information: http://msdn.microsoft.com/en-us/library/4xzx2d41.aspx?ppud=4

Declare string address and int port and you are ready to connect through the TcpClient class.

System.Net.Sockets.TcpClient client = new TcpClient();
try
{
    client.Connect(address, port);
    Console.WriteLine("Connection open, host active");
} catch (SocketException ex)
{
    Console.WriteLine("Connection could not be established due to: \n" + ex.Message);
}
finally
{
    client.Close();
}