23

This question is related to this one.

I work with animation, which generates a LOT of files (+/- 1,000,000) typically stored on a single directory. On Mac Os X, some bugs came up with more than +/-30,000 files, so I used to break the animation into various directories.

On Ubuntu, is there a limit for the number of files a single directory can hold?

clami219
  • 103
H_7
  • 661

1 Answers1

35

Ubuntu does not limit the size of a directory, it's imposed by the file system. Each file and directory is an so-called inode. You can use df -i to check the number of inodes in use and available for all mounted filesystems.

I've just created 1 million and one files without issues because my inode limit for my ext4 home partition of 50 GB (46 GiB) is large enough.

I used shell expansion for creating the files, combined with the touch utility:

mkdir test
cd test
touch {0..300000}
touch {300000..600000}
touch {600000..900000}
touch {900000..1000000}

This creates 1000001 files which can be verified with ls | wc -l. Why 300000..600000 and not 300001..600000? Because I was too lazy to put that 1 at the end.

df -i looks like:

/dev/sda6            3055616 1133635 1921981   38% /home

Now remove the test files (cd ..&&rm -f test took much longer, so use rm with the filenames):

rm {0..300000}
rm {300000..600000}
rm {600000..900000}
rm {900000..1000000}
cd ..
rmdir f

and the number of inodes in use decreased immediately after removal of the files:

/dev/sda6            3055616  133634 2921982    5% /home

Note that even if the filesystem allows such large numbers of files, it's a horrible idea to store such large files in a single directory. At least use some subdirectories with a structure like f/i/l/e/filename.ext. Programs do often not expect such large quantities of files.

Lekensteyn
  • 174,277
  • wow. I am learning a lot from your answer, many thanks. But my programs do expect a large quantities of files (After Effects, Autodesk Combustion) and handles them as a unique file. (seqs of jpgs, tifs, pngs, dpxs, etc.. ). This should work with ext3 extension too? – H_7 Oct 21 '11 at 20:56
  • 1
    ext4 is a filesystem type, not a file extension. Can't you create a directory for queuing edits? I doubt that those programs can run nicely with such large file quantities. – Lekensteyn Oct 21 '11 at 21:04
  • 1
    Sure I can. I'm just imagining problems before I get to them. You're right about Adobe After Effects, 'll crash certainly, but don't doubt Autodesk guys, they make amazing pieces of code... =) looks like using a Ferrari. Very easy to crash, very fast and powerfull because is blocked on ground. Hope my english makes sense. – H_7 Oct 21 '11 at 21:14
  • 1
    I understand it. Yay, exactly 17.000 reputation ^^ – Lekensteyn Oct 21 '11 at 21:27
  • 1
    Before deleting the files, try and open this directory up in Nautilus and see what happens ;). – Mario Oct 26 '11 at 17:25
  • Let's assume that the system (for some reason) uses nearly all iNodes available... what can you do to increase it? How can I tell Ubuntu to use more iNodes for a certain EXT4-mount? – bzero Mar 19 '15 at 08:50
  • 1
    @bzero Please open a new question for that. – Lekensteyn Mar 20 '15 at 16:06
  • @Lekensteyn done: http://askubuntu.com/questions/600159/how-can-i-increase-number-of-files-in-an-ext4-partition – bzero Mar 23 '15 at 07:48
  • "This creates 1000001 files which can be verified with ls | wc -l. Why 300000..600000 and not 300001..600000? Because I was too lazy to put that 1 at the end." love the explanation that's longer than the actual change. :) – Hartator Nov 15 '18 at 18:19