Deleted Files Hold Disk Space

Posted on | Edited on | In

This is a case spotted recently, the file system disk usage was high and triggered the alert.

From df -hT we see the utilization of this file system has 52GB used, but checking from du -s -BG * | sort -nr I did not see any big file or folder that can explain for the big usage.

Even checking with hidden option du -sch .[!.]* * |sort -h, no answer.

Then I realized that probably the files have been deleted but they were held by live process, for example, thread.

To identify the pending deleted files:

1
2
3
4
5
6
7
8
# +L1: list files that have fewer than 1 link, so likely is deleted
# you can use | head to find the column meaning, specifically the file size
lsof -nP +L1 | grep '(deleted)'

# or using find command to get deleted file hold by process
# but this will not show the file size, the pid is in /proc/<pid>/fd
# so you which process is it.
find /proc/*/fd -ls | grep  '(deleted)'

Then kill(if not needed) or restart the process or service, for example by systemctl restart, after that checked again with df -Th, the disk space got released.

0%