How to use the private and public keys?
You might find The First Few Milliseconds of an HTTPS Connection by Jeff Moser useful reading. It provides a detailed explanation of what happens on the wire when a browser sets up an HTTPS connection with a server.
This interesting video illustrates public key cryptography by mixing paint colours!
You should also look at the OpenSSL command line tool. You can use it to: generate key pairs, generate certificates, setup a mini CA, run a server that uses one of the certs you generated, connect to any SSL server and get detailed information about its SSL configuration and certificates used. See the OpenSSL Command-Line HOWTO for lots of examples of what you can do with OpenSSL.
Also you can use online tools like these to get detailed information about an internet facing website's certificate and its SSL configuration:
SSL Checker - SSL certificate checks and detailed certificate information
Website's SSL Configuration - looks deeply at the SSL setup of a website
As far as setting up SSL in your environment, it will depend on what web server you are using and/or your hosting provider. For example, you could try googling for "setting up SSL on" + the name of you web server or host - e.g., "Setting up SSL on Apache". Also, most certification authorities (e.g., Verisign) typically provide good information on setting up various web servers and applications to use SSL.
The certificates used for HTTPS are X.509 certificates (they're the only ones mentioned in the HTTP over TLS, i.e. HTTPS, specification).
A (server) certificate is a piece of information used to prove the server's identity to the client. It's the combination between a public key and various pieces of information (mostly regarding the identity of the owner of the key pair, whoever controls the private key), this combination being signed using the private key of the issuer of the certificate.
Trusting a certificate means that you trust its content, in particular, that you trust that the public key belongs to the identity represented in the certificate. From a server point of view, the identity will be the server name (see RFC 2818 Section 3.1 or RFC 6125). Certificates are in principle public pieces of information.
Where the server's private key (not necessarily the issuer's private key) is used is for the authenticated SSL/TLS key-exchange to take place. During this key exchange, depending on whether you're using RSA or Diffie-Helman key exchange (depending on the cipher suite, DH is quite common nowadays), the private key will either be used to decrypt the a piece of information that the client has encrypted with the certificate's public key (RSA) or be used to sign a piece of information generated by the server (DH). Either way, it's what proves to the client that the remote party has the private key for that certificate. At the end of the key exchange mechanisms, both the client and the server use a shared key. (This question might be of interest regarding the key exchange, although it's probably better to read the TLS specification itself, possibly a number of times...)
Once you have both, i.e. trusting the binding between public key and identity (by verifying the certificate) and knowing that the remote party has the private key for the public key in that certificate, you know that the remote party is indeed the server whose identity is described in the certificate (in the Subject Alternative Name or, if absent, in the Common Name of the Subject DN).
You may also be interested in this answer to "What makes SSL secure?".
If you want to try this yourself for a website, you'll need more than a public key, you'll need to make a certificate, possibly a self-signed certificate or via your own CA. Make sure the host name you use is at least in the Common Name of your Subject DN or better, in a Subject Alternative Name DNS entry. There are a number of tutorials to create self-signed certificates with OpenSSL, for example. OpenSSL's CA.pl (see man-page) may be useful too if you want to create your mini-CA.