3

Very complex situation.

My main server is in a 5TB raid.

I'm currently on holiday so I use Teamviewer and ssh to remote into my server.

2 Days ago I ran a backup which was configured to be stored to my external hard drive. (10TB)

Backup started successfully and I left it to backup.

However later at night I had errors everywhere, here is what I noticed.

External drives aren't mounted. No connection to network. (though I can ssh) Teamviewer stopped working. All critical applications stopped working. (VirtualBox for example) Package Manager stopped working. I ran df and all kinds of disk utilities in ssh to see what was taking up the drive.

But the confusing part is that the drive isn't even full. The Trash is empty. All partitions aren't even past 50% full.

Now back to when I ran that backup. I thought maybe I accidentally routed the backup to store on the main drive, but there is no signs of the backup at all.

I've scanned and rebooted multiple times, however it shows that the disk isn't full, but Linux still says it's full.

I have attached some images to give a better understanding.

1

2

I was wondering if the /dev/loopx partitions in the image could be the issue but I'm not sure what to do with those.

karel
  • 114,770
SirSterben
  • 39
  • What does df (without -i) print? – tkausl Apr 18 '19 at 21:15
  • 1
    df without -i reports that /dev/sdb1 (mounted on /) is 100% in use, however all the folders in / all total up to around 2-3TB. (and the drive is 5TB)

    Edit: I'm using su so I don't think there is any hidden folders which it can't read.

     – SirSterben Apr 18 '19 at 21:21
  • your du report wont show files hidden under a mount point like /home, yet they will still fill up your filesystem. Use cut and paste text instead of images. The loop devices are not the issue. – ubfan1 Nov 06 '19 at 18:17
  • 1
    The du report will also not show any files that have been deleted but are still opened by some process. You would need to run lsof to find these files. – doneal24 Nov 06 '19 at 18:42
  • I've the same problem on 21.04 A reboot fixes it but still afterwards it fills up. I'm using the xanmod kernel.. – france1 Nov 18 '21 at 11:54
  • /var/log is according to du -h 9.4GB but all files are only ~1GB – france1 Nov 18 '21 at 11:55

1 Answers1

0

According to doneal24 you need to run lsof to find files that are deleted but still opened by a process. In my case I deleted /var/log/syslog* but it didn't free any space on my ssd

But if I do echo>/var/log/syslog it frees up a lot

In my case this information helped me - finally after a reboot the space is freed because the process that's using this file's shut down

Maybe something used your hd space and removed the files afterwards but they were still used or it didn't free it because of some bug

france1
  • 174