Copy data through SSH tunnel over multiple hops

Solution 1:

By far, the easiest way is to just copy it via scp. Plus, this syntax actually works unlike some of the other suggestions.

You can't beat this syntax for ease. It allows you to recursively copy, rsync or what ever you'd like without the hassle of considering potentially complex pipes. This syntax is intuitively clear, will be more readily supportable by Sys Admins that follow you and does not make useless use of cat.

scp -3 devappserver:/path/to/copy/from qaappserver:/path/to/copy/to

From the scp man page: -3 Copies between two remote hosts are transferred through the local host. Without this option the data is copied directly between the two remote hosts. Note that this option disables the progress meter.

In the example below

• Your workstation is named MacBook-Pro.
• Dev Jump Box is named devjumpserver
• Dev Application Server is named devapplicationserver
  • Is on LAN DNS zone named .local
• QA Jump Box is named qajumpserver
• QA Application Server is named qaapplicationserver
  • Is on LAN DNZ zone named .local
• We'll perform a test copy of a 670GB /etc/hosts file ;-)
• Assumption is made that you have SSH public key authentication configured.

Here is an ~/.ssh/config file that sets up the direct access from your workstation to the application servers via the appropriate jump (aka bastion server).

MacBook-Pro:~ barrychapman$ cat ~/.ssh/config
Host *
  ServerAliveInterval 60
Host devapplicationsever
  HostName devapplicationserver.local
  ProxyCommand ssh -i ~/.ssh/id_rsa [email protected] -W %h:%p
  User barrychapman
Host qaapplicationserver
  HostName qaapplicationserver.local
  ProxyCommand ssh -i ~/.ssh/id_rsa [email protected] -W %h:%p
  User barrychapman

MacBook-Pro:~ barrychapman$

Testing for presence of file on target server, it won't be there.

MacBook-Pro:~ barrychapman$ ssh qaapplicationserver ls /tmp/hosts
ls: cannot access /tmp/hosts: No such file or directory
Killed by signal 1.
MacBook-Pro:~ barrychapman$

Now let's copy a file from Dev Application server to QA Application via your workstation.

MacBook-Pro:~ barrychapman$ scp -3 devapplicationserver:/etc/hosts qaapplicationserver:/tmp/
Killed by signal 1.
Killed by signal 1.
MacBook-Pro:~ barrychapman$

Now let's check for the presence of the copied file on the QA Application Server. It will be there this time.

MacBook-Pro:~ barrychapman$ ssh qaapplicationserver ls /tmp/hosts
/tmp/hosts
Killed by signal 1.
MacBook-Pro:~ barrychapman$ 

Note

When closing a ProxyCommand connection, you will see the warning message "Killed by signal 1". This is SSH tearing down the ProxyCommand connection and is nothing to be alarmed about. You can get rid of it by adding LogLevel Quiet to your bastion host config stanza.

Solution 2:

PIPES!

If the internet is a series of tubes, Unix is a series of pipes -- something like:

cat ginormous-file | ssh user@host1 "cat | ssh user@host2 \"cat >out\" "

should work.

If you need to traverse more hosts, add more pipes (and more nested layers of \-escaped quotation) as needed. (Note however that if the pipeline/escaping gets so complex that you have to draw a diagram or resort to counting on your fingers to determine how many times you have to double up on escapes it's probably time to admit defeat and set up a proper VPN!)