How to create a large file in UNIX?

yes "Some text" | head -n 100000 > large-file

With csh/tcsh:

repeat 10000 echo some test > large-file

With zsh:

{repeat 10000 echo some test} > large-file

On GNU systems, see also:

seq 100000 > large-file

Or:

truncate -s 10T large-file

(creates a 10TiB sparse file (very large but doesn't take any space on disk)) and the other alternatives discussed at "Create a test file with lots of zero bytes".


Doing cat file >> file would be a bad idea.

First, it doesn't work with some cat implementations that refuse to read files that are the same as their output file. But even if you work around it by doing cat file | cat >> file, if file is larger than cat's internal buffer, that would cause cat to run in an infinite loop as it would end up reading the data that it has written earlier.

On file systems backed by a rotational hard drive, it would be pretty inefficient as well (after reaching a size greater than would possibly be cached in memory) as the drive would need to go back and forth between the location where to read the data, and that where to write it.


You can create a large file on Solaris using:

mkfile 10g /path/to/file

Another way which works on Solaris (and Linux):

truncate -s 10g /path/to file

It is also possible to use:

dd if=/dev/zero of=/path/to/file bs=1048576 count=10240

The fastest way possible to create a big file in a Linux system is fallocate:

sudo fallocate -l 2G bigfile

fallocate manipulates the files system, and does not actually writes to the data sectors by default, and as such is extremely fast. The downside it is that it has to be run as root.

Running it successively in a loop, you can fill the biggest of filesystems in a matter of seconds.

From man fallocate

fallocate is used to manipulate the allocated disk space for a file, either to deallocate or preallocate it.
For filesystems which support the fallocate system call, preallocation is done quickly by allocating blocks and marking them as uninitialized, requiring no IO to the data blocks. This is much faster than creating a file by filling it with zeros.
Supported for XFS (since Linux 2.6.38), ext4 (since Linux 3.0), Btrfs (since Linux 3.7) and tmpfs (since Linux 3.5).

Tags:

Shell

Files