Tell fs to free space from deleted files NOW

Check with lsof to see if there are files held open. Space will not be freed until they are closed.

sudo /usr/sbin/lsof | grep deleted

will tell you which deleted files are still held open.

Use lsof to find the deleted, but open, file still consuming space:

lsof | grep deleted | grep etilqs_1IlrBRwsveCCxId
chrome     3446       user  128u      REG              253,2              16400       2364626 /var/tmp/etilqs_1IlrBRwsveCCxId (deleted)  

Find the entry in /proc/<pid>/fd/ that cooresponds to the filehandle:

ls -l /proc/3446/fd/etilqs_1IlrBRwsveCCxId
lrwx------. 1 user unix 64 Feb 11 15:31 128 -> /var/tmp/etilqs_1IlrBRwsveCCxId (deleted)

Now, just cat /dev/null into the fd:

cat /dev/null > /proc/3446/fd/128

Note that the inode is still open, but now it's 0 length

chrome     3446       user  128u      REG              253,2         0    2364626 /var/tmp/etilqs_1IlrBRwsveCCxId (deleted)

df will not show space reserved for root (even when run as root):

# df -h
Filesystem            Size  Used Avail Use% Mounted on
...
/dev/optvol           625G  607G     0 100% /opt
...

How to change "reserved block percentage"

1. Reduce reserved space to 4%

   # tune2fs -m4 /dev/sda4

df -h now showed 45M free.

2. Saved my files quickly

3. Put it back to 5%

   # tune2fs -m5 /dev/sda4