JDK 11 SSL Error on valid certificate (working in previous versions)
The issue is currently resolved in JDK 12 https://bugs.openjdk.java.net/browse/JDK-8209965, and was included in ea-9.
The backport to JDK 11 has also been resolved https://bugs.openjdk.java.net/browse/JDK-8210005 and is included in
- 11.0.3 (Oracle JDK)
- 11.0.2 (OpenJDK)
Some background to this can be found in the comments here https://github.com/openssl/openssl/pull/4463/files
TLS 1.3 adds a scheme for the server to indicate to the client its list of supported groups in the EncryptedExtensions message, but none of the relevant specifications permit sending supported_groups in the ServerHello.
Nonetheless (possibly due to the close proximity to the "ec_point_formats" extension, which is allowed in the ServerHello), there are several servers that send this extension in the ServerHello anyway.
Up to and including the 1.1.0 release, we did not check for the presence of nonpermitted extensions, so to avoid a regression, we must permit this extension in the TLS 1.2 ServerHello as well.