Trust Store vs Key Store - creating with keytool
The terminology is a bit confusing indeed, but both javax.net.ssl.keyStore and javax.net.ssl.trustStore are used to specify which keystores to use, for two different purposes. Keystores come in various formats and are not even necessarily files (see this question), and keytool is just a tool to perform various operations on them (import/export/list/...).
The javax.net.ssl.keyStore and javax.net.ssl.trustStore parameters are the default parameters used to build KeyManagers and TrustManagers (respectively), then used to build an SSLContext which essentially contains the SSL/TLS settings to use when making an SSL/TLS connection via an SSLSocketFactory or an SSLEngine. These system properties are just where the default values come from, which is then used by SSLContext.getDefault(), itself used by SSLSocketFactory.getDefault() for example. (All of this can be customized via the API in a number of places, if you don't want to use the default values and that specific SSLContexts for a given purpose.)
The difference between the KeyManager and TrustManager (and thus between javax.net.ssl.keyStore and javax.net.ssl.trustStore) is as follows (quoted from the JSSE ref guide):
TrustManager: Determines whether the remote authentication credentials (and thus the connection) should be trusted.
KeyManager: Determines which authentication credentials to send to the remote host.
(Other parameters are available and their default values are described in the JSSE ref guide. Note that while there is a default value for the trust store, there isn't one for the key store.)
Essentially, the keystore in javax.net.ssl.keyStore is meant to contain your private keys and certificates, whereas the javax.net.ssl.trustStore is meant to contain the CA certificates you're willing to trust when a remote party presents its certificate. In some cases, they can be one and the same store, although it's often better practice to use distinct stores (especially when they're file-based).
There is no difference between keystore and truststore files. Both are files in the proprietary JKS file format. The distinction is in the use: To the best of my knowledge, Java will only use the store that is referenced by the -Djavax.net.ssl.trustStore system property to look for certificates to trust when creating SSL connections. Same for keys and -Djavax.net.ssl.keyStore. But in theory it's fine to use one and the same file for trust- and keystores.
To explain in common usecase/purpose or layman way:
TrustStore : Used to store the certificates of trusted entities. A process can maintain a store of certificates of all the trusted parties which it trusts.
keyStore : Used to store the server keys (both public and private) along with the signed certificate.
During the SSL handshake,
A client tries to access https://
And thus, Server responds by providing a SSL certificate (which is stored in its keyStore)
Now, the client receives the SSL certificate and verifies it via trustStore (i.e the client's trustStore already has pre-defined set of certificates which it trusts.). Its like : Can I trust this server ? Is this the same server whom I am trying to talk to ? No middle man attacks ?
Once, the client verifies that it is talking to server which it trusts, then SSL communication can happen over a shared secret key.
Note : I am not talking here anything about client authentication on server side. If a server wants to do a client authentication too, then the server also maintains a trustStore to verify client. Then it becomes mutual TLS.