How to remove X bytes from the end of a large file without reading the whole file?
use the function truncate
http://linux.die.net/man/2/truncate
int truncate(const char *path, off_t length);
int ftruncate(int fd, off_t length);
truncate takes the file name
ftruncate takes an open file descriptor
both of these set the file length to length
so it either truncates or elongates (in the latter case, the rest of the file will be filled with NULL/ZERO)
[edit]
truncate (linux shell command) will work also
**SYNTAX**
truncate -s integer <filename>
**OPTIONS**
-s number specify the new file length. If the new length is smaller than the current filelength data is lost. If the new length is greater the file is padded with 0. You can specify a magnitude character to ease large numbers:
b or B size is bytes.
k size is 1000 bytes.
K size is 1024 bytes.
m size is 10^6 bytes.
M size is 1024^2 bytes.
g size is 10^9 bytes.
G size is 1024^3 bytes.
**EXAMPLES**
To shrink a file to 10 bytes:
truncate -s 10 /tmp/foo
To enlarge or shrink a file to 345 Megabytes:
truncate -s 345M /tmp/foo
[/edit]
Although there were plenty of references to the truncate
function in this thread, no one really answered the OP's question about reducing a file by a fixed amount from a scripting environment. Kevin's answer used truncate
to resize the target file to a fixed amount, but of course the correctness of this solution requires the user to first know the size of the target file minus the extraneous data at the end. So, we have:
-s, --size=SIZE set or adjust the file size by SIZE bytes
Truncate
actually supports removing data from the end of a target file directly by prefixing SIZE with a -
character.
For example, to resize a file by 314 bytes you can do:
truncate --size=-314 target_file.bin
Using "truncate" is best way, I just post some examples:
I have a file "PNav-h.db", it has 50176 bytes.
-rw-r--r-- 1 user user 50176 Mar 8 23:43 PNav-h.db $truncate -s 1000 PNav-h.db
it set the file size to 1000 byptes
-rw-r--r-- 1 user user 1000 Mar 9 00:02 PNav-h.db
For your case, use
$truncate --size=xxx xxfilename
, using-<size number>
to reduce the file size$truncate --size=-300 PNav-h.db -rw-r--r-- 1 user user 700 Mar 9 00:07 PNav-h.db
final file size = 1000-300=700
using
+<size number>
to increase the file size$truncate --size=+500 PNav-h.db -rw-r--r-- 1 user user 1200 Mar 9 00:09 PNav-h.db
final file size = 700 + 500 = 1200
if there is no
-
or+
, it is to set the file size.$truncate --size=60000 PNav-h.db -rw-r--r-- 1 user user 60000 Mar 9 00:12 PNav-h.db
final file size set to 60000