Do I need to check for file corruption once scp is done?

scp verifies that it copied all the data sent by the other party. The integrity of the transfer is guaranteed by the cryptographic channel protocol. So you don't need to verify the integrity after the transfer. That would be redundant, and very unlikely to catch any hardware error since the data you're comparing against would probably be read from the cache. Verifying data periodically can be useful, but verifying immediately after the transfer is pointless.

You do however need to ensure that scp isn't telling you that something went wrong. There should be an error message, but the reliable indicator is that scp returns a nonzero exit code if something went wrong.

More precisely, you know that the file was transmitted correctly if scp returns 0 (i.e. the success status code). Checking that the exit status is 0 is necessary when you run any command anyway. If scp returns an error status, or if it's killed by a signal, or if it never dies because the system crashes or loses power while it's running, then you have no guarantees. In particular, since scp copies the file directly to its final name, this means that you can end up with a partial file in case of a system crash. The part that was copied is guaranteed to be correct but the file may be truncated.

For better reliability, use rsync instead of scp. Unless instructed otherwise, rsync writes to a temporary file, and moves it into place once it's finished. Thus, if rsync returns a success code, you know the file is present and a correct, complete copy; if rsync hasn't returned an error code then no file will be present (unless there was an older version of the file, in which case that older version won't be modified).


I've never had a problem with corruption after I scp something but if you're worried about it you can always run md5sum <filename> on both systems to make sure they're the same.


Per @david-king 's suggestion, this is an md5 based solution for checking the integrity of files after the transfer. Run the following command once after cding to /source/folder on the local machine and once after cding to /destination/folder on the remote host: find . -type f -print0 | xargs -0 -I {} md5sum {} | md5sum. The resulting hash should be identical after a successful transfer.

Update: According to this answer to a similar question on ServerFault, scp does not guarantee file integrity (Please check this answer by @Gilles for details). Alternative to post-transfer checking of file hashes, you can use rsync to transfer files and check its return code.

Update 2: The following only checks if the files and their respective sizes match after the transfer: find . -type f -print0 | xargs -0 -I {} stat --printf="%n %s\n" {} | sort | md5sum

Tags:

Scp