Getting `write too long` error when trying to create tar.gz file from file and directories
I was having a similar issue until I looked more closely at the tar.FileInfoHeader doc:
FileInfoHeader creates a partially-populated Header from fi. If fi describes a symlink, FileInfoHeader records link as the link target. If fi describes a directory, a slash is appended to the name. Because os.FileInfo's Name method returns only the base name of the file it describes, it may be necessary to modify the Name field of the returned header to provide the full path name of the file.
Essentially, FileInfoHeader isn't guaranteed to fill out all the header fields before you write it with WriteHeader, and if you look at the implementation the Size field is only set on regular files. Your code snippet seems to only handle directories, this means if you come across any other non regular file, you write the header with a size of zero then attempt to copy a potentially non-zero sized special file on disk into the tar. Go returns ErrWriteTooLong to stop you from creating a broken tar.
I came up with this and haven't had the issue since.
if err := filepath.Walk(directory, func(path string, info os.FileInfo, err error) error {
if err != nil {
return check(err)
}
var link string
if info.Mode()&os.ModeSymlink == os.ModeSymlink {
if link, err = os.Readlink(path); err != nil {
return check(err)
}
}
header, err := tar.FileInfoHeader(info, link)
if err != nil {
return check(err)
}
header.Name = filepath.Join(baseDir, strings.TrimPrefix(path, directory))
if err = tw.WriteHeader(header); err != nil {
return check(err)
}
if !info.Mode().IsRegular() { //nothing more to do for non-regular
return nil
}
fh, err := os.Open(path)
if err != nil {
return check(err)
}
defer fh.Close()
if _, err = io.CopyBuffer(tw, fh, buf); err != nil {
return check(err)
}
return nil
})