UFS How many files could be stored in a directory?

benpptung

Member

Reaction score: 1
Messages: 49

I've googled this question, but got no clear answer.

I would plan a centralized file storage for all the files uploaded ( Mostly, jpg or jpeg file, I guess )

I know it might be only thousands files at this moment. However, I really hope to know what is the limitation of the FreeBSD directory. only around 30000 files?

Thank you for any help. :)
 

Beastie

Daemon

Reaction score: 532
Messages: 2,272

The file count limit in UFS2 depends on the number of inodes made available when newfs(8)ing the disk.
You can check the inode count on your current system with df -i.

I don't know how Oracle's version of ZFS compares to the one currently implemented in FreeBSD, but it may be a more appropriate choice for heavy use.
 

usdmatt

Daemon

Reaction score: 561
Messages: 1,482

It's generally a bad idea to design an application so an unlimited number of files are going to end up in the same directory. How are the files being stored/retrieved?

For web applications, the usual method is to record each file in a database, which gives the file an ID number. You then store the file in a directory based on its ID number. File 1 will end up in 0/0/1/filename, file 48583 in 5/8/3/filename, etc. How many levels of sub directories you decide to use depends on how many files you expect to store. 3 levels allows you to store 1 million files but only have 1000 in any single directory. If you don't have something allocating ID numbers you can also use characters from the filename or various other methods to choose directory names.

If your users are storing and accessing the files directly (samba/ftp/etc) then obviously a system like that isn't possible. If they are accessing the files directly though, having a huge number of files in one place will definitely become a problem. Opening a directory with thousands of files can easily take several minutes and isn't particularly user friendly (Whatever the user is accessing the files with, console/windows explorer/etc will just appear to hang while it generates the file listing).

Eventually you will start running out of inodes on UFS although that's a fairly high number and you can see how many are used/free by running df -hi
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 9,283
Messages: 33,826

For web applications, the usual method is to record each file in a database, which gives the file an ID number. You then store the file in a directory based on its ID number. File 1 will end up in 0/0/1/filename, file 48583 in 5/8/3/filename, etc. How many levels of sub directories you decide to use depends on how many files you expect to store. 3 levels allows you to store 1 million files but only have 1000 in any single directory. If you don't have something allocating ID numbers you can also use characters from the filename or various other methods to choose directory names.
I've seen a lot of variations on this theme, not just for web applications. ISPs use it a lot too for their users' home directories. It's quite common to see home directories like /home/d/dice and /home/u/usdmatt.
 
Top