How to create a large file in UNIX?
yes "Some text" | head -n 100000 > large-file
With csh
/tcsh
:
repeat 10000 echo some test > large-file
With zsh
:
{repeat 10000 echo some test} > large-file
On GNU systems, see also:
seq 100000 > large-file
Or:
truncate -s 10T large-file
(creates a 10TiB sparse file (very large but doesn't take any space on disk)) and the other alternatives discussed at "Create a test file with lots of zero bytes".
Doing cat file >> file
would be a bad idea.
First, it doesn't work with some cat
implementations that refuse to read files that are the same as their output file. But even if you work around it by doing cat file | cat >> file
, if file
is larger than cat
's internal buffer, that would cause cat
to run in an infinite loop as it would end up reading the data that it has written earlier.
On file systems backed by a rotational hard drive, it would be pretty inefficient as well (after reaching a size greater than would possibly be cached in memory) as the drive would need to go back and forth between the location where to read the data, and that where to write it.
You can create a large file on Solaris using:
mkfile 10g /path/to/file
Another way which works on Solaris (and Linux):
truncate -s 10g /path/to file
It is also possible to use:
dd if=/dev/zero of=/path/to/file bs=1048576 count=10240
The fastest way possible to create a big file in a Linux system is fallocate
:
sudo fallocate -l 2G bigfile
fallocate
manipulates the files system, and does not actually writes to the data sectors by default, and as such is extremely fast. The downside it is that it has to be run as root.
Running it successively in a loop, you can fill the biggest of filesystems in a matter of seconds.
From man fallocate
fallocate is used to manipulate the allocated disk space for a file, either to deallocate or preallocate it.
For filesystems which support the fallocate system call, preallocation is done quickly by allocating blocks and marking them as uninitialized, requiring no IO to the data blocks. This is much faster than creating a file by filling it with zeros.
Supported for XFS (since Linux 2.6.38), ext4 (since Linux 3.0), Btrfs (since Linux 3.7) and tmpfs (since Linux 3.5).