How to diagnose missing disk space? (disk usage analyzer not corresponding to df)

Question

I have a very mysterious problem. Today, I was not able to log in. There was no "incorrect-password" message or anything, just the symptoms described in this question.

So I logged into the GUI-less shell and the first thing that shocked me was that when I hit tab, the bash auto-completion failed, because there was not enough space to create a temporary file. I was shocked, since I expected that about 90% of disk is still free. After removing some redundant docker volumes I was able to login and now I am trying to see where all my disk space went:

It seems that I have about 5 Giga of stuff in my home folder and another 5 is used by the system. This seems to correspond to reality. Since my disk is about 100 Giga I have nothing to worry about. However, df thinks otherwise:

Filesystem                   Size  Used Avail Use% Mounted on
udev                         3,9G     0  3,9G   0% /dev
tmpfs                        788M  9,9M  778M   2% /run
/dev/mapper/ubuntu--vg-root  102G   96G  847M 100% /
tmpfs                        3,9G   57M  3,8G   2% /dev/shm
tmpfs                        5,0M  4,0K  5,0M   1% /run/lock
tmpfs                        3,9G     0  3,9G   0% /sys/fs/cgroup
/dev/sda1                    464M  121M  315M  28% /boot
tmpfs                        788M  172K  788M   1% /run/user/1000
/home/askbe4/.Private        102G   96G  847M 100% /home/askbe4

Or more visually:

That is, the other tools show that I have only about 800 Megabytes free. This is a drastic difference. Also, it stands to mention that the disk and the home folder are both encrypted. I encrypted them just by clicking 'yes' during Ubuntu installation.

I feel completely dumbfounded and have no Idea what to make of this. Any hints are most welcome.

@ravery yes, I did sudo rm -rf ~/root/.local/share/Trash/* and sudo rm -rf ~/home/me/.local/share/Trash/* — Martin Drozdik, Dec 19 '17 at 19:16
those are incorrect it is sudo rm -rf /root/.local/share/Trash/* and sudo rm -rf ~/.local/share/Trash/* — ravery, Dec 19 '17 at 19:56
@ravery Yupps, you're right. I used your commands, but unfortunately the issue remains. — Martin Drozdik, Dec 20 '17 at 09:49

score 10 · Accepted Answer · answered Dec 20 '17 at 11:54

From this guide on recovering lost disk space:

Permissions. Users of Ubuntu are allowed access to files as set by permissions. These file permissions may prevent a user from seeing files created or stored in system folders and those owned by other users. If using a graphical file browser don't forget to enable hidden file viewing (usually CTRL+H). Run the commands/applications as 'root'. For command line operations, precede the command with sudo. For graphical applications such as nautilus and baobab/Disk Usage Analyzer start the command to launch the application with gksudo.

So I ran the disk usage analyzer with:

sudo baobab

and suddenly it showed 90 Giga of data in /var/lib/docker. This was a docker container that ran astray and that I forgot to delete.

Apparently I misunderstood a fundamental principle of Linux. I always thought that you can see the size of a file even when you do not have read permissions. I even tested this with sudo touch file-with-no-access. I could see the size with ls -l file-with-no-access. The thing I did not realize was that things are different when there is a directory to which I have no access.

who could think this program is calle baobab, what are they smoking? if it is called disk usage analizer in ui, why it cannot be called similarly in command line — Darius.V, Apr 28 '22 at 05:57
@Darius.V Baobab is an African tree, it is a word play and apparently the original name of the tool, which has been around for many years. — Konrad Höffner, Jul 26 '23 at 08:07

How to diagnose missing disk space? (disk usage analyzer not corresponding to df)

1 Answers1