How to connect to FTPS server with data connection using same TLS session?

Indeed some FTP(S) servers do require that the TLS/SSL session is reused for the data connection. This is a security measure by which the server can verify that the data connection is used by the same client as the control connection.

Some references for common FTP servers:

  • vsftpd: https://scarybeastsecurity.blogspot.com/2009/02/vsftpd-210-released.html
  • FileZilla server: https://svn.filezilla-project.org/filezilla?view=revision&revision=6661
  • ProFTPD: http://www.proftpd.org/docs/contrib/mod_tls.html#TLSOptions (NoSessionReuseRequired directive)

What may help you with the implementation is that Cyberduck FTP(S) client does support TLS/SSL session reuse and it uses Apache Commons Net library:

  • https://github.com/iterate-ch/cyberduck/issues/5087 – Reuse Session key on data connection

  • See its FTPClient.java code (extends Commons Net FTPSClient), particularly its override of _prepareDataSocket_ method:

     @Override
 protected void _prepareDataSocket_(final Socket socket) {
     if(preferences.getBoolean("ftp.tls.session.requirereuse")) {
         if(socket instanceof SSLSocket) {
             // Control socket is SSL
             final SSLSession session = ((SSLSocket) _socket_).getSession();
             if(session.isValid()) {
                 final SSLSessionContext context = session.getSessionContext();
                 context.setSessionCacheSize(preferences.getInteger("ftp.ssl.session.cache.size"));
                 try {
                     final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
                     sessionHostPortCache.setAccessible(true);
                     final Object cache = sessionHostPortCache.get(context);
                     final Method putMethod = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
                     putMethod.setAccessible(true);
                     Method getHostMethod;
                     try {
                         getHostMethod = socket.getClass().getMethod("getPeerHost");
                     }
                     catch(NoSuchMethodException e) {
                         // Running in IKVM
                         getHostMethod = socket.getClass().getDeclaredMethod("getHost");
                     }
                     getHostMethod.setAccessible(true);
                     Object peerHost = getHostMethod.invoke(socket);
                     putMethod.invoke(cache, String.format("%s:%s", peerHost, socket.getPort()).toLowerCase(Locale.ROOT), session);
                 }
                 catch(NoSuchFieldException e) {
                     // Not running in expected JRE
                     log.warn("No field sessionHostPortCache in SSLSessionContext", e);
                 }
                 catch(Exception e) {
                     // Not running in expected JRE
                     log.warn(e.getMessage());
                 }
             }
             else {
                 log.warn(String.format("SSL session %s for socket %s is not rejoinable", session, socket));
             }
         }
     }
 }

  • It seems that the _prepareDataSocket_ method was added to Commons Net FTPSClient specifically to allow the TLS/SSL session reuse implementation:
    https://issues.apache.org/jira/browse/NET-426

    A native support for the reuse is still pending:
    https://issues.apache.org/jira/browse/NET-408

  • You will obviously need to override the Spring Integration DefaultFtpsSessionFactory.createClientInstance() to return your custom FTPSClient implementation with the session reuse support.

The above solution does not work on its own anymore since JDK 8u161.

According to JDK 8u161 Update Release Notes (and the answer by @Laurent):

Added TLS session hash and extended master secret extension support
...
In case of compatibility issues, an application may disable negotiation of this extension by setting the System Property jdk.tls.useExtendedMasterSecret to false in the JDK

I.e., you can call this to fix the problem (you still need to override the _prepareDataSocket_):

System.setProperty("jdk.tls.useExtendedMasterSecret", "false");

Though this should be a considered a workaround only. I do not know a proper solution.

An alternative implementation is here:
https://issues.apache.org/jira/browse/NET-408

There's a separate question about problems in 1.8.0_161:
SSL Session reuse in Apache FTPS client in JDK 8u161

I actually had the same problem in the past (just in C++/OpenSSL, I do not do Java), so I knew what to google for.


To make Martin Prikryl's suggestion work for me I had to store the key not only under socket.getInetAddress().getHostName() but also under socket.getInetAddress().getHostAddress(). (Solution stolen from here.)


You can use this SSLSessionReuseFTPSClient class :

import java.io.IOException;
import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.net.Socket;
import java.util.Locale;

import javax.net.ssl.SSLSession;
import javax.net.ssl.SSLSessionContext;
import javax.net.ssl.SSLSocket;

import org.apache.commons.net.ftp.FTPSClient;

public class SSLSessionReuseFTPSClient extends FTPSClient {

    // adapted from:
    // https://trac.cyberduck.io/browser/trunk/ftp/src/main/java/ch/cyberduck/core/ftp/FTPClient.java
    @Override
    protected void _prepareDataSocket_(final Socket socket) throws IOException {
        if (socket instanceof SSLSocket) {
            // Control socket is SSL
            final SSLSession session = ((SSLSocket) _socket_).getSession();
            if (session.isValid()) {
                final SSLSessionContext context = session.getSessionContext();
                try {
                    final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache");
                    sessionHostPortCache.setAccessible(true);
                    final Object cache = sessionHostPortCache.get(context);
                    final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class);
                    method.setAccessible(true);
                    method.invoke(cache, String
                            .format("%s:%s", socket.getInetAddress().getHostName(), String.valueOf(socket.getPort()))
                            .toLowerCase(Locale.ROOT), session);
                    method.invoke(cache, String
                            .format("%s:%s", socket.getInetAddress().getHostAddress(), String.valueOf(socket.getPort()))
                            .toLowerCase(Locale.ROOT), session);
                } catch (NoSuchFieldException e) {
                    throw new IOException(e);
                } catch (Exception e) {
                    throw new IOException(e);
                }
            } else {
                throw new IOException("Invalid SSL Session");
            }
        }
    }
}

And With openJDK 1.8.0_161 :

We must set : 

System.setProperty("jdk.tls.useExtendedMasterSecret", "false");

according to http://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html

Added TLS session hash and extended master secret extension support

In case of compatibility issues, an application may disable negotiation of this extension by setting the System Property jdk.tls.useExtendedMasterSecret to false in the JDK

Tags:

Java

Ftp

Ftps

Apache Commons Net

Session Reuse

Recent Posts