How to remove X bytes from the end of a large file without reading the whole file?

use the function truncate

http://linux.die.net/man/2/truncate

int truncate(const char *path, off_t length);
int ftruncate(int fd, off_t length); 

truncate takes the file name
ftruncate takes an open file descriptor

both of these set the file length to length so it either truncates or elongates (in the latter case, the rest of the file will be filled with NULL/ZERO)

[edit]
truncate (linux shell command) will work also

**SYNTAX**

truncate -s integer <filename>  
**OPTIONS**

-s number specify the new file length. If the new length is smaller than the current filelength data is lost. If the new length is greater the file is padded with 0. You can specify a magnitude character to ease large numbers:
b or B size is bytes.
k size is 1000 bytes.
K size is 1024 bytes.
m size is 10^6 bytes.
M size is 1024^2 bytes.
g size is 10^9 bytes.
G size is 1024^3 bytes.


**EXAMPLES**

To shrink a file to 10 bytes:

truncate -s 10 /tmp/foo

To enlarge or shrink a file to 345 Megabytes:

truncate -s 345M /tmp/foo

[/edit]


Although there were plenty of references to the truncate function in this thread, no one really answered the OP's question about reducing a file by a fixed amount from a scripting environment. Kevin's answer used truncate to resize the target file to a fixed amount, but of course the correctness of this solution requires the user to first know the size of the target file minus the extraneous data at the end. So, we have:

   -s, --size=SIZE
          set or adjust the file size by SIZE bytes

Truncate actually supports removing data from the end of a target file directly by prefixing SIZE with a - character.

For example, to resize a file by 314 bytes you can do:

truncate --size=-314 target_file.bin


Using "truncate" is best way, I just post some examples:

  1. I have a file "PNav-h.db", it has 50176 bytes.

    -rw-r--r--  1 user user 50176 Mar  8 23:43 PNav-h.db
     $truncate -s 1000 PNav-h.db
    

    it set the file size to 1000 byptes

     -rw-r--r--  1 user user  1000 Mar  9 00:02 PNav-h.db
    
  2. For your case, use $truncate --size=xxx xxfilename, using -<size number> to reduce the file size

    $truncate --size=-300 PNav-h.db
    -rw-r--r--  1 user user   700 Mar  9 00:07 PNav-h.db
    

    final file size = 1000-300=700

  3. using +<size number> to increase the file size

    $truncate --size=+500 PNav-h.db
    -rw-r--r--  1 user user  1200 Mar  9 00:09 PNav-h.db
    

    final file size = 700 + 500 = 1200

  4. if there is no - or +, it is to set the file size.

    $truncate --size=60000 PNav-h.db
    -rw-r--r--  1 user user 60000 Mar  9 00:12 PNav-h.db 
    

    final file size set to 60000

Tags:

Linux

Shell