getting javax.net.ssl.SSLException: Received fatal alert: protocol_version while scraping data using Jsoup
You want to use Java 8 here since it supports TLSv1.2 by default with additional required cipher suites.
Why not Java 7?
I tested on my box with Java 7 (1.7.0_45) and got the same error.
I activated the debugging messages and forced TLSv1.2.
System.setProperty("javax.net.debug", "all");
System.setProperty("https.protocols", "TLSv1.2");
Then I hit this new error:
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
Finally, I went to Comodoca's SSL analyzer and see something interesting. According to SSL analyzer, the site you're targeting has only enabled the following cipher suites:
Cipher Suites Enabled Name (ID) Key Size (in bits) TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (0xC02F) 128 ECDH 256-bit (P-256) TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xC030) 256 ECDH 256-bit (P-256) TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 (0x9E) 128 DH 2048-bit TLS_DHE_RSA_WITH_AES_256_GCM_SHA384 (0x9F) 256 DH 2048-bit
(see Full details)
On my side, I don't have any of the above suites. Check if you have them:
SSLContext sc = SSLContext.getInstance("TLS");
sc.init(null, null, new java.security.SecureRandom());
String[] scs = sc.getSocketFactory().getSupportedCipherSuites();
Arrays.sort(scs);
for(String s : scs) {
System.out.println(s);
}
See the SSLSocketFactoryEx for enabling the needed cipher suites.
Why Java 8?
On the other hand, I succeed in runnnig the code by moving from Java 7 to Java 8 (1.8.0_20) which support TLS v1.2 by default and provides the needed cipher suites.
Here is a trimmed list of supported cipher suites (71 suites in total) for Java 8 (1.8.0_20) on Windows 7.
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
...
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Snippet
try {
// Create a trust manager that does not validate certificate chains
TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
public java.security.cert.X509Certificate[] getAcceptedIssuers() {
return null;
}
public void checkClientTrusted(java.security.cert.X509Certificate[] certs, String authType) {
}
public void checkServerTrusted(java.security.cert.X509Certificate[] certs, String authType) {
}
} };
// Install the all-trusting trust manager
SSLContext sc = SSLContext.getInstance("SSL");
sc.init(null, trustAllCerts, new java.security.SecureRandom());
HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
// Fetch url
String url = "https://www.sos.nh.gov/corporate/soskb/SearchResults.asp?FormName=CorpNameSearch&Words=All&SearchStr=facebook&SearchType=Search";
Connection.Response response = Jsoup //
.connect(url) //
.timeout(60000) //
.method(Connection.Method.GET) //
.userAgent("Mozilla/5.0 (Windows NT 10.0; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0") //
.execute();
Document document = response.parse();
System.out.println(document);
} catch (Exception e) {
e.printStackTrace();
}
Final thought:
When it comes to security, ALWAYS use the latest updated version.
(From comment for closure, expanded a little for future finders)
By experiment, that site requires protocol version TLSv1.2 and although Java7 JSSE implements this, client side by default disables 1.2 and 1.1. Java8 does enable them by default; or in Java7 since Jsoup uses HttpsURLConnection
you can change the enabled versions with system property https.protocols
. You need to include at least TLSv1.2
and for greatest flexibility should use all currently acceptable protocols https.protocols=TLSv1,TLSv1.1,TLSv1.2
.
Also, using that all-trusting TrustManager
means that pretty much any baddie with access to your network can fake this site and expose any sensitive data you send it. It's better to set your local truststore so it accepts the certificate(s) and thus server(s) you need, but not bogus ones.