Does Robocopy SKIP copying existing files by default?

By default, Robocopy skips copying existing files if the specific metadata of the files match then those files will be skipped from the "file" copy operation (/COPY:DAT).

Robocopy Defaults

  • Skips file copy if last write time, file name, and file size match
  • Copies file if last write time, file name, or file sizes don't match

As indicated by @mklement0, the implied default /COPY:DAT will not copy over files with different data if the timestamp [LastWriteTime] and the file size is the same and thus this is a default skip.

So if for some reason you have two files you are syncing that have matching file size, file name, and last modified attributes even though the data is different, it will not copy the source file.

Robocopy Default Options: /COPY:DAT /R:1000000 /W:30

As confirmed by log file or command window running robocopy c:\source c:\dest *

Robocopy or Robocopy /?

 /COPY:copyflag[s] : What to COPY (default is /COPY:DAT)
                      (copyflags : D=Data, A=Attributes, T=Timestamps
                       S=Security=NTFS ACLs, O=Owner info, U=aUditing info).


robocopy SOURCE DESTINATION FILE(S) /IS

Where IS stands for Include Same file(s). Using this switch causes to overwrite existing files. See below:

::
:: File Selection Options :
::
                 /A :: copy only files with the Archive attribute set.
                 /M :: copy only files with the Archive attribute and reset it.
    /IA:[RASHCNETO] :: Include only files with any of the given Attributes set.
    /XA:[RASHCNETO] :: eXclude files with any of the given Attributes set.

 /XF file [file]... :: eXclude Files matching given names/paths/wildcards.
 /XD dirs [dirs]... :: eXclude Directories matching given names/paths.

                /XC :: eXclude Changed files.
                /XN :: eXclude Newer files.
                /XO :: eXclude Older files.
                /XX :: eXclude eXtra files and directories.
                /XL :: eXclude Lonely files and directories.
                /IS :: Include Same files.
                /IT :: Include Tweaked files.

  • Unfortunately, the official documentation doesn't describe the logic behind what files are skipped by default.

  • The unofficial ss64.com documentation, however, provides the crucial pointer (emphasis added):

    • By default Robocopy will only copy a file if the source and destination have different time stamps or different file sizes.

Note: time stamp refers to the last-modified time stamp (only).

In other words: Robocopy considers two files to be the same based only on whether their last-modified time stamps and file sizes are identical and therefore skips copying in that event.

From what I can tell:

  • This behavior is not affected by modifying what aspects of a file / directory to copy (parameters /copy / /dcopy)

    • These arguments are only applied if RoboCopy has fundamentally deemed a given file / directory in need of copying, based on sameness / inclusion logic.

  • If you want to extend the default sameness detection (same last-modified timestamp and same file size) to the following properties, use the /it (Include Tweaked) option:

    • file attributes (represented as A for the /copy parameter)
    • ACL (represented as S for the /copy parameter)
    • file ownership (represented as O for the /copy parameter)
    • auditing information (represented as U for the /copy parameter)

  • Robocopy seems offers no option to detect sameness of files based on their content (which is typically implemented via a cryptographic hash function).
    Therefore, in the rare event that there could be files with identical last-modified time stamps and identical file sizes that nonetheless differ in content, your only option is to use the /is (Include Same) option:

    • This causes the files that are considered the same to be copied unconditionally - and therefore potentially unnecessarily.

    • Caveat: Curiously, files that are the same with respect to last-modified file stamp and file size but differ with respect to the /it-related properties mentioned above are not included by default with /is alone; therefore, to cover all eventualities, use /is and /it combined.

The following Pester tests demonstrate the behaviors; save the code as Tests.ps1 and invoke it as Invoke-Pester ./Tests.ps1:

Describe RoboCopySkippedFilesTests {
  BeforeAll {
    Push-Location TestDrive:
  }
  AfterAll {
    Pop-Location
  }
  BeforeEach {
    # Set up a source and a destination folder and make their
    # Content the same.
    if (Test-Path dest) { Remove-Item -Force -Recurse dest }
    $null = New-Item -Type Directory -Force src
    'file1' > src\file1
    'file2' > src\file2
    Copy-Item -Recurse src dest
  }

  It "Does not copy anything with identical folders." {
    robocopy.exe src dest
    $LASTEXITCODE | Should Be 0
  }
  It "Does recognize a changed last-modified filestamp" {
    (gi src\file1).LastWriteTime = [datetime]::now
    robocopy.exe src dest # | Out-Host
    $LASTEXITCODE | Should Be 1
  }
  It "Does recognize a change in file size" {
    '!' | Add-Content dest\file1
    robocopy.exe src dest # | Out-Host
    $LASTEXITCODE | Should Be 1
  }
  It "Does not recognize a change in file content, with size and last-modified date identical" {
    'file!' > dest\file1
    (Get-Item dest\file1).LastWriteTime = (Get-Item src\file1).LastWriteTime
    robocopy.exe src dest # | Out-Host
    $LASTEXITCODE | Should Be 0
  }
  It "Does not recognize a change in file attributes, with size and last-modified date identical" {
    (Get-Item dest\file1).Attributes = 'System'
    robocopy.exe src dest # | Out-Host
    $LASTEXITCODE | Should Be 0
  }
  It "Does recognize an attribute-modified-only file as tweaked (/IT)" {
    (Get-Item dest\file1).Attributes = 'System'
    robocopy.exe src dest /IT # | Out-Host
    $LASTEXITCODE | Should Be 1
  }
  It "Does not include an attribute-modified-only file with /IS" {
    Remove-Item src\file2, dest\file2
    (Get-Item dest\file1).Attributes = 'System'
    robocopy.exe src dest /IS # | Out-Host
    $LASTEXITCODE | Should Be 0
  }
}

You should see something like the following, indicating that all tests passed (as of Robocopy.exe with file version 10.0.16299.15 (WinBuild.160101.0800)):

Describing RoboCopySkippedFilesTests
 [+] Does not copy anything with identical folders. 231ms
 [+] Does recognize a changed last-modified filestamp 112ms
 [+] Does recognize a change in file size 68ms
 [+] Does not recognize a change in file content, with size and last-modified date identical 69ms
 [+] Does not recognize a change in file attributes, with size and last-modified date identical 83ms
 [+] Does recognize an attribute-modified-only file as tweaked (/IT) 65ms
 [+] Does not include an attribute-modified-only file with /IS 70ms
Tests completed in 589ms
Passed: 7 Failed: 0 Skipped: 0 Pending: 0 Inconclusive: 0

Tags:

Windows

Robocopy

Recent Posts