Whose key is used to encrypt a HTTPS response?

Public keys are not directly used to encrypt any of the underlying HTTP traffic on an HTTPS connection; neither the HTTP request nor the HTTP response are encrypted this way. Rather, during the initial SSL handshake, a session specific symmetric key is negotiated between the client and the server, and it's the symmetric key that is then used to encrypt all traffic on the HTTP connection in both directions.

The specific mechanism by which the symmetric key is negotiated depends on the specific cipher suite that is negotiated between client and server. This negotiation always involves the server's public key and a value sent by the client; it may also involve items like a client public key or separate connection specific public keys from the server and client.

Additional detail can be found in RFC 5246 starting here:

https://www.rfc-editor.org/rfc/rfc5246#section-7.3

My server maintains its private key and publish a public key

Yes.

that any clients can use to encrypt their request.

No.

Since my server is the only one who has the private key to decrypt any message encryped using server's public key, any request sent this way can be considered secure.

That's not how it works.

However my question is at the response part. When the server sends the response back to the client, whose public key will the server use to encrypt the response message?

Neither.

I assume the server will use client's public key to encrypt the response (by default? or upon configuration?).

No. See below. In most SSL connections the server doesn't even know the client's public key, if indeed the client even has one. This only occurs in so-called 'mutual' SSL, where both peers authenticate to each other.

If so, does the server knows the client's public key from the request it sends to the server, or somehow else?

It only knows the public key if the client sends its certificate, which only happens if the server is configured to request it, which it usually isn't.

HTTPS runs over SSL, which uses a symmetric session key negotiated independently by both peers. It is never transmitted. The server's keypair is only used to provide a digital signature over its certificate, which the client can verify, which proves that that server owns that certificate. From there on it is all symmetric negotiation and encryption.

Authority: RFC 2246 and successors.

None.

Public and private key are used to negotiate a symmetric encryption key, that will be used during that session. Asymmetric encryption uses two keys (private and public), symmetric encryption uses only one.

Your server sends its public key to a client, the client validates that key signature (check the CA and all that) then use it to encrypt a randomly selected key that will be used as symmetric encryption key, and send it to the server. Because only the private key can decrypt that message, the message is secure, only the server can decrypt it. Then the server accepts that key selected by the client, and they start to transmit data using symmetric encryption.

Why all this? asymmetric encryption is quite computionally expensive, so it is just used to ensure that client and server can negotiate a secure symmetric key without sending it in plain text. Symmetric encryption is cheap.

Symmetric encryption is also safe, the problem is that both parts must know the key before start, and that is a big issue. By using asymmetric encryption for negotiating the key, this problem is solved.

UPDATE

Well, it seems that @EJP does not agree with my answer, so I tried to find some more documentation that explain the thing in an easy way.

http://www.techradar.com/news/software/how-ssl-and-tls-works-1047412

SSL explained

When you visit a bank's website, the bank's server will automatically redirect you to its secure site using the HTTPS protocol before you can log in. This results in your browser and the bank's site negotiating a secure channel using SSL.

This negotiation goes a little like this (note that I've simplified it greatly). The browser sends a message stating what the latest version of SSL it can support and a list of symmetric algorithms it can use. The web server sends back a message with the version of SSL and the algorithm that will be used.

It sends its certificate as well. The client verifies the certificate using the known certificates that came with the browser; in other words, it checks that it has been signed by a trusted CA and that it hasn't expired.

If the certificate is valid, the browser generates a one-time key for the session, encrypts it with the server's public key (it's part of the certificate), and sends it to the server. The server decrypts the key, then uses that key together with the agreed symmetric algorithm for the rest of the session.

I may be confused.