rsync mkstemp failed Invalid argument (22) with davfs mount of Box.com cloud
The problem occurs because of rsync
making temporary files with filenames that the box.com and/or davfs do not understand. Thus the file .01_Track_1.mp3.YVmFI9
does not exist on your system, but is a temporary artefact of rsync
. Some guesswork from my side: if you don't get the error on all files, you probably only get the errors on files that were already uploaded (and changed).
It used to be impossible to switch off this temporary file generation, but you might nowadays have more luck by adding the option --inplace
.
However the advantages of using rsync if you are not talking to a rsync-daemon (which you are not if you are using davfs), are unclear to me.
Therefore, as an alternative, you could try cp --update
, which only copies a file when the source is newer than the destination. New files and any files with changes in ID3 tags will get copied, others will not.
Or if you need to have more control use find
:
cd /home/me/Music/
find * -size -250M -print0 | cpio -pdmv0 /home/me/Cloud/Box/Music
this preserves the hierarchy structure and cpio
does not overwrite existing files that are not older.
1. Issues with special characters in filenames
Are there any special characters in the filenames? Depending on the filesystem you're writing these files to, they may not allow you to prefix files with a dot (.
) for example.
2. Problems with rsync modification times and webdav2
I came across this blog post where an issue is described with rsync
having a problem writing/tracking file modification times across webdav2 mounted box.com directories.
The problem shows up like this in the mounted file system:
david@sydney:~/Pictures$ ls -l /mnt/box/bwca/08/09/IMG_3084.CR2
-rw-r--r-- 1 david david 12564061 Aug 14 16:08 /mnt/box/bwca/08/09/IMG_3084.CR2
david@sydney:~/Pictures$ ls -l 2012/08/09/IMG_3084.CR2
-rw-rw-r-- 1 david david 12564061 Aug 9 13:00 2012/08/09/IMG_3084.CR2
That same article showed a workaround:
$ rsync -avhP --size-only --bwlimit=64 2012/08 /mnt/box/bwca/
This is an OK way to use rsync
, but it's only comparing files based on their size now, not their checksums.
3. Issues with davfs2 (WebDAV)
I came across this thread titled: rsync via davfs2? in the WebDAV (davfs) forum over on sourceforge. Someone was inquiring about a similar situation where they wanted to use WebDAV to mount an online storage provider and perform rsync's to the mounted storage via WebDAV. This is what one of the developers (Werner Baumann) of WebDAV had to say about this topic.
excerpt of Werner's response
davfs2 will only upload complete files. It can not do the incremental stuff rsync usually does, and that makes rsync very efficient.
davfs2 uses a local cache on disk. This will make it more responsive and your application should profit from this too. But it needs local disk space for this. You should allow for a large cache size, so rsync can do most of its work with the local cache, and davfs2 will upload most of the files in the background, when rsync has already finished.
Werner goes on to suggest the following
This could be a disadvantage in this case. When rsync reads a file on the remote host, it must be transfered by davfs2 into the local cache first (if it is not already there). This could make the process really and unnecessary slow. As rsync only works as a sophisticated copy-program in your case, it might be better to use cp instead. cp has an options (-u) to copy only files that are newer than the ones in the davfs2 file system (= smartdrive) and it would not need to read the files, but only reads file meta data like mtime.
A command like "cp -pru directory/to/backup dav/" might do the job. It shold not download files (like rsync might do, but I am not sure) (please look at the manuals of cp and rsync).
Options?
So as @Anthon has suggested, you can use the cp -u
method to copy the files. Realizing that this method solely looks at a file's size as a factor in comparison, so it's not completely reliable.
You should not use anything that only looks at modification times when comparing files, cp -pru
. Werner explains why in this thread:
excerpt on issue with mod times
When you unmount a davfs2 file system and mount it again at some later time, file times may have changed according to the time information from the server. Tools like cp -pu and rsync can not rely on these times to determine what files have changed.
So given the various issues surrounding modification times an approach using purely checksums seems like a better fit:
$ rsync -avvz --omit-dir-times --checksum --human-readable --progress <local dir> <remote dir>
To stop the Invalid argument (22)
error I had to stop rsync from creating its temp files on the davfs destination.
rsync --temp-dir=/tmp
I think what happens is rsync's temp file names start with .
and davfs does not allow this. So I went further and told rsync to ignore source files with names starting with .
. Since I'm using --delete
, I also told it not to try to delete the lost+found
directory in the davfs destination.
rsync --temp-dir=/tmp --exclude lost+found --exclude '.*'