Why does SSL handshake give 'Could not generate DH keypair' exception?

The problem is the prime size. The maximum-acceptable size that Java accepts is 1024 bits. This is a known issue (see JDK-6521495).

The bug report that I linked to mentions a workaround using BouncyCastle's JCE implementation. Hopefully that should work for you.

UPDATE

This was reported as bug JDK-7044060 and fixed recently.

Note, however, that the limit was only raised to 2048 bit. For sizes > 2048 bit, there is JDK-8072452 - Remove the maximum prime size of DH Keys; the fix appears to be for 9.


Here is my solution (java 1.6), also would be interested why I had to do this:

I noticed from the javax.security.debug=ssl, that sometimes the used cipher suite is TLS_DHE_... and sometime it is TLS_ECDHE_.... The later would happen if I added BouncyCastle. If TLS_ECDHE_ was selected, MOST OF the time it worked, but not ALWAYS, so adding even BouncyCastle provider was unreliable (failed with same error, every other time or so). I guess somewhere in the Sun SSL implementation sometimes it choose DHE, sometimes it choose ECDHE.

So the solution posted here relies on removing TLS_DHE_ ciphers completely. NOTE: BouncyCastle is NOT required for the solution.

So create the server certification file by:

echo |openssl s_client -connect example.org:443 2>&1 |sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p'

Save this as it will be referenced later, than here is the solution for an SSL http get, excluding the TLS_DHE_ cipher suites.

package org.example.security;

import java.io.BufferedInputStream;
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.InetAddress;
import java.net.Socket;
import java.net.URL;
import java.net.UnknownHostException;
import java.security.KeyStore;
import java.security.cert.Certificate;
import java.security.cert.CertificateFactory;
import java.security.cert.X509Certificate;
import java.util.ArrayList;
import java.util.List;

import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLParameters;
import javax.net.ssl.SSLSocket;
import javax.net.ssl.SSLSocketFactory;
import javax.net.ssl.TrustManagerFactory;

import org.apache.log4j.Logger;

public class SSLExcludeCipherConnectionHelper {

    private Logger logger = Logger.getLogger(SSLExcludeCipherConnectionHelper.class);

    private String[] exludedCipherSuites = {"_DHE_","_DH_"};

    private String trustCert = null;

    private TrustManagerFactory tmf;

    public void setExludedCipherSuites(String[] exludedCipherSuites) {
        this.exludedCipherSuites = exludedCipherSuites;
    }

    public SSLExcludeCipherConnectionHelper(String trustCert) {
        super();
        this.trustCert = trustCert;
        //Security.addProvider(new BouncyCastleProvider());
        try {
            this.initTrustManager();
        } catch (Exception ex) {
            ex.printStackTrace();
        }
    }

    private void initTrustManager() throws Exception {
        CertificateFactory cf = CertificateFactory.getInstance("X.509");
        InputStream caInput = new BufferedInputStream(new FileInputStream(trustCert));
        Certificate ca = null;
        try {
            ca = cf.generateCertificate(caInput);
            logger.debug("ca=" + ((X509Certificate) ca).getSubjectDN());
        } finally {
            caInput.close();
        }

        // Create a KeyStore containing our trusted CAs
        KeyStore keyStore = KeyStore.getInstance("jks");
        keyStore.load(null, null);
        keyStore.setCertificateEntry("ca", ca);

        // Create a TrustManager that trusts the CAs in our KeyStore
        String tmfAlgorithm = TrustManagerFactory.getDefaultAlgorithm();
        tmf = TrustManagerFactory.getInstance(tmfAlgorithm);
        tmf.init(keyStore);
    }

    public String get(URL url) throws Exception {
        // Create an SSLContext that uses our TrustManager
        SSLContext context = SSLContext.getInstance("TLS");
        context.init(null, tmf.getTrustManagers(), null);
        SSLParameters params = context.getSupportedSSLParameters();
        List<String> enabledCiphers = new ArrayList<String>();
        for (String cipher : params.getCipherSuites()) {
            boolean exclude = false;
            if (exludedCipherSuites != null) {
                for (int i=0; i<exludedCipherSuites.length && !exclude; i++) {
                    exclude = cipher.indexOf(exludedCipherSuites[i]) >= 0;
                }
            }
            if (!exclude) {
                enabledCiphers.add(cipher);
            }
        }
        String[] cArray = new String[enabledCiphers.size()];
        enabledCiphers.toArray(cArray);

        // Tell the URLConnection to use a SocketFactory from our SSLContext
        HttpsURLConnection urlConnection =
            (HttpsURLConnection)url.openConnection();
        SSLSocketFactory sf = context.getSocketFactory();
        sf = new DOSSLSocketFactory(sf, cArray);
        urlConnection.setSSLSocketFactory(sf);
        BufferedReader in = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
        String inputLine;
        StringBuffer buffer = new StringBuffer();
        while ((inputLine = in.readLine()) != null) 
            buffer.append(inputLine);
        in.close();

        return buffer.toString();
    }

    private class DOSSLSocketFactory extends javax.net.ssl.SSLSocketFactory {

        private SSLSocketFactory sf = null;
        private String[] enabledCiphers = null;

        private DOSSLSocketFactory(SSLSocketFactory sf, String[] enabledCiphers) {
            super();
            this.sf = sf;
            this.enabledCiphers = enabledCiphers;
        }

        private Socket getSocketWithEnabledCiphers(Socket socket) {
            if (enabledCiphers != null && socket != null && socket instanceof SSLSocket)
                ((SSLSocket)socket).setEnabledCipherSuites(enabledCiphers);

            return socket;
        }

        @Override
        public Socket createSocket(Socket s, String host, int port,
                boolean autoClose) throws IOException {
            return getSocketWithEnabledCiphers(sf.createSocket(s, host, port, autoClose));
        }

        @Override
        public String[] getDefaultCipherSuites() {
            return sf.getDefaultCipherSuites();
        }

        @Override
        public String[] getSupportedCipherSuites() {
            if (enabledCiphers == null)
                return sf.getSupportedCipherSuites();
            else
                return enabledCiphers;
        }

        @Override
        public Socket createSocket(String host, int port) throws IOException,
                UnknownHostException {
            return getSocketWithEnabledCiphers(sf.createSocket(host, port));
        }

        @Override
        public Socket createSocket(InetAddress address, int port)
                throws IOException {
            return getSocketWithEnabledCiphers(sf.createSocket(address, port));
        }

        @Override
        public Socket createSocket(String host, int port, InetAddress localAddress,
                int localPort) throws IOException, UnknownHostException {
            return getSocketWithEnabledCiphers(sf.createSocket(host, port, localAddress, localPort));
        }

        @Override
        public Socket createSocket(InetAddress address, int port,
                InetAddress localaddress, int localport) throws IOException {
            return getSocketWithEnabledCiphers(sf.createSocket(address, port, localaddress, localport));
        }

    }
}

Finally here is how it is used (certFilePath if the path of the certificate saved from openssl):

try {
            URL url = new URL("https://www.example.org?q=somedata");            
            SSLExcludeCipherConnectionHelper sslExclHelper = new SSLExcludeCipherConnectionHelper(certFilePath);
            logger.debug(
                    sslExclHelper.get(url)
            );
        } catch (Exception ex) {
            ex.printStackTrace();
        }

The "Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files" answer did not work for me but The BouncyCastle's JCE provider suggestion did.

Here are the steps I took using Java 1.6.0_65-b14-462 on Mac OSC 10.7.5

1) Download these jars:

  • bcprov-jdk15on-154.jar

  • bcprov-ext-jdk15on-154.jar

2) move these jars to $JAVA_HOME/lib/ext

3) edit $JAVA_HOME/lib/security/java.security as follows: security.provider.1=org.bouncycastle.jce.provider.BouncyCastleProvider

restart app using JRE and give it a try