df in linux not showing correct free space after file removal

Solution 1:

Deleting the filename doesn't actually delete the file. Some other process is holding the file open, causing it to not be deleted; restart or kill that process to release the file.

Use

lsof +L1

to find out which process is using a deleted (unlinked) file.

Solution 2:

as Ignacio mentions, deleting the file won't free the space until you delete the processes that have open handles against that file.

Nevertheless, you can reclaim the space without killing the processes. All you need to do is to remove the file descriptors.

First execute lsof | grep deleted to identify the process holding the file

[hudson@opsynxvm0055 log]$ /usr/sbin/lsof |grep deleted
java       8859   hudson    1w      REG              253,0 3662503356    7578206 /crucible/data/current/var/log/fisheye.out (deleted)

Then execute:

cd /proc/PID/fd

then

[hudson@opsynxvm0055 fd]$ ls -l |grep deleted
total 0
l-wx------ 1 hudson devel 64 Feb  7 11:48 1 -> /crucible/data/current/var/log/fisheye.out (deleted)

The "1" will be the file descriptor. Now type "> FD" to reclaim that space

> 1

You might need to repeat the operation if there are other processes holding the file.

---

Solution 3:

One possibility is that the file(s) you deleted have more references in the filesystem. If you've created hardlinks, several filenames will point to the same data, and the data (the actual contents) won't be marked as free/usable until all references to it has been removed. Before you delete files, either stat them (Entry named Links) or do ls -l on them (should be the second column).

If it does turn out that the files are referenced elsewhere, I guess you'll have to ls -i the file(s) to find the inode-number, and then do a find with -inum <inode-number> to find the other references to that file (you probably also want to use -mount to stay within the same filesystem as well).

---

Solution 4:

If partition has been configured to reserve certain portion of disk space only for root usage, df will not include this space as available.

[root@server]# df -h
Filesystem            Size  Used Avail Use% Mounted on
...
/dev/optvol           625G  607G     0 100% /opt
...

Even after space will be reclaimed by deleting files/directories, non-root user won't be able to write to particular partition.

You can easily check if that's your case by trying to create a file on a device as root and non-root user.

Additionally you can check filesystem configuration by running

tune2fs -l <device> | egrep "Block count|Reserved block count

and calculating actual % on your own.

To change disk % reserved for root-only usage, execute

tune2fs -m <percentage> <device>

---

Solution 5:

The file is still locked by the process opening it. To free up space, do these steps:

1. Run sudo lsof | grep deleted and see which process is holding the file. Example result:

   $ sudo lsof | grep deleted
   COMMAND   PID USER   FD   TYPE DEVICE SIZE/OFF      NODE NAME
   cron     1623 root    5u   REG   0,21        0 395919638 /tmp/tmpfPagTZ4 (deleted)
   

2. Kill the process using sudo kill -9 {PID}. In above sample, the PID is 1623.

   $ sudo kill -9 1623
   

3. Run df to check if space is already freed up. If it's still full, maybe you need to wait a few seconds and check again.