Hi everybody,
We are using FreeBSD on our servers, and we need to store around 2.000.000 files (there is around 3000 uploaded files per day), 1 file is around 50kb. Please suggest how to format hard drive, which file system we should use to get the best performance (IO) or some other approach not build on java, storing files in FS is preferable.
For now we are using
and storing files in:
/files/1/2/3/123456.gz
with this directory hierarchy we got around 2000 files in 1 directory.
I got in mind to change it to:
/files/8/1/d/81dc9bdb52d04dc20036dbd8313ed055.gz
storing filename as md5($filename) and we finish with around 500 files per directory.
Also I am considering this hierarchy:
/files/81/dc/81dc9bdb52d04dc20036dbd8313ed055.gz
which will be 30 files in directory.
How you would store 2.000.000 files ?
We are using FreeBSD on our servers, and we need to store around 2.000.000 files (there is around 3000 uploaded files per day), 1 file is around 50kb. Please suggest how to format hard drive, which file system we should use to get the best performance (IO) or some other approach not build on java, storing files in FS is preferable.
For now we are using
Code:
/dev/mfid0s1d on /usr (ufs, local, noatime, soft-updates)
and storing files in:
/files/1/2/3/123456.gz
with this directory hierarchy we got around 2000 files in 1 directory.
I got in mind to change it to:
/files/8/1/d/81dc9bdb52d04dc20036dbd8313ed055.gz
storing filename as md5($filename) and we finish with around 500 files per directory.
Also I am considering this hierarchy:
/files/81/dc/81dc9bdb52d04dc20036dbd8313ed055.gz
which will be 30 files in directory.
How you would store 2.000.000 files ?