0

I know, that when I create empty catalog it has his own size:

mkdir test && du -h test

gives:

4,0K    test

But this case is another. I had catalog with 1e6 files (74 GB) on external HDD (ext4). I processed these files, saved in other catalog and deleted from this location in parallelized script. Now this catalog should be empty. So it is ok that

ls -a raw/all

print

.  ..

But 

du -h raw/all

gives

19M     raw/all

Without -h we have exactly 19288B. And this is also strange result:

ls -la raw/all

total 19292
drwxr-xr-x 2 daniel daniel 19746816 wrz 18 19:23 .
drwxrwxr-x 4 daniel daniel     4096 wrz 17 13:21 ..

What is going on? Is there any cache in du command? Or after processing such many files there are any rest after these files? I deleted filed by unlink in perl5 script.

Update - answers to questions from comments

ls -aiR raw/all
raw/all:
51647758 .  51647760 ..

lsof +L1

prints 44 lines but nothing connected with this catalog. But lsof gives something with numbers like 19746816.

lsof | grep raw
bash      5355                daniel  cwd       DIR               8,17  19746816   51647758 /media/daniel/f2971ccd-4841-4da3-8ce3-d9f2cc03fc27/daniel/pro/scraper/raw/all
lsof      5677                daniel  cwd       DIR               8,17  19746816   51647758 /media/daniel/f2971ccd-4841-4da3-8ce3-d9f2cc03fc27/daniel/pro/scraper/raw/all
grep      5678                daniel  cwd       DIR               8,17  19746816   51647758 /media/daniel/f2971ccd-4841-4da3-8ce3-d9f2cc03fc27/daniel/pro/scraper/raw/all
lsof      5679                daniel  cwd       DIR               8,17  19746816   51647758 /media/daniel/f2971ccd-4841-4da3-8ce3-d9f2cc03fc27/daniel/pro/scraper/raw/all

Laptop was rebooted. External drive was disconnected. But du still shows 19M.

Daniel
  • 387
  • 1
    try to ls -aiR raw/all(https://askubuntu.com/questions/15419/how-to-list-all-the-files-in-a-tree-a-directory-and-its-subdirs) – Redbob Sep 18 '17 at 18:12
  • 1
    I'm curious if you run sync if that changes the size reported by du. Also any chance some file(s) are help open and can't be fully removed? Maybe check lsof for open files?
    lsof +L1 to list open deleted files, found here: https://serverfault.com/questions/232525/df-in-linux-not-showing-correct-free-space-after-file-removal#232526     – virullius Sep 18 '17 at 19:00
  • I think this is related to the fact that the directory own now 78GB worth of Inodes and have to keep a huge list of thoses, which size is 19MB – Dark Sinus Sep 18 '17 at 23:07
  • Maybe there are hidden files, listed by ls -a (part of the suggestion in Redbob's comment) – sudodus Sep 21 '17 at 10:10
  • @sudodus Not, there was no any hidden file. I shows ls -a output in my question. – Daniel Sep 21 '17 at 10:16
  • Then I think muru's comment contains the answer. – sudodus Sep 21 '17 at 10:21
  • @sudodus Yes. In my opinion too. – Daniel Sep 21 '17 at 10:27

1 Answers1

2

Directories in ext4 filesystems grow, but don't shrink. So, if at some point it had to take up 19M for storing details of files in it, it will retain that much space even if those files were deleted. If you want to shrink a now-empty directory, delete and recreate it:

rmdir foo && mkdir foo

Or, retaining permissions:

mkdir foo2
chown --reference=foo foo2
getfacl foo | setfacl --set-file=- foo2
rmdir foo
mv foo2 foo

Source:

muru
  • 197,895
  • 55
  • 485
  • 740