Space overhead is much higher on ashift=12 raidz

belon_cfy · #1 August 19th, 2012, 06:37

Hi,
My current server consume almost double of space on the ashift=12 raidz especially storing a lot of small files in the server even the recordsize has been set to 128K, may I know is there any zfs update to resolve this issue, or should I need to choice mirror instead of raidz because space saving seems to be more noticable on mirror in this case.

Savagedlight · #2 August 19th, 2012, 14:38

Your RaidZ with ashift=12 is consuming twice as much space compared to what? Do you store the exact same data in both places?

Do you have drives with a 4k sector size? ashift=12 instructs ZFS to use record sizes which are a multiple of 4kiB (2^12), which prevents several performance problems with such drives.

~~If you set record size to 128k, minimum space taken by each file will be 128kiB regardless of the ashift setting. (I'm not entirely on this one)~~
Quoting the zfs manpage, regarding recordsize; emphasis is mine:

Code:

For databases that create very large files but access them in small 
random  chunks,  these  algorithms  may be suboptimal. Specifying a
recordsize greater than or equal to the record size of the database
can	result	in significant performance gains. Use of this property
for general purpose file systems is strongly discouraged,  and  may
adversely affect performance.

gkontos · #3 August 19th, 2012, 14:49

The overhead is always an issue when dealing with very small files on 4K drives. You could choose not to align them for 4K but there would be a performance impact. Sometimes it is a matter of choice, speed versus capacity.

http://lists.freebsd.org/pipermail/f...ay/014311.html

t1066 · #4 August 19th, 2012, 15:49

I had some data about changes from 512 to 4k in the thread. The changes on the sizes of the filesystems /usr/src and /usr/ports are particularly interesting since they have lots of small files.

belon_cfy · #5 August 20th, 2012, 03:19

OK, so I would rather use 512b on a 4k disk for storing backup but sacrifice some of the performance, it will be better in space saving.

**wblock@** · #6 August 20th, 2012, 04:51

Benchmark it first. Doing misaligned operations can destroy performance. Or maybe pick a filesystem that handles fragments better. Setting UFS to optimize for space rather than speed might do it. I'd think ZFS would do block suballocation also, maybe optionally.

**phoenix** · #7 August 22nd, 2012, 01:03

Or, enable compression and not worry about it.

Sebulon · #8 August 23rd, 2012, 14:03

Quote:

Originally Posted by phoenix

Or, enable compression and not worry about it.

Well, if you had compression enabled before- when there was ashift=9, and have compression enabled now with ashift=12, the space consumed will have increased. I too have noticed this, which to me was to be expected. Especially on filesystems like t1066 said, /, /usr, /usr/local, /usr/ports, /var and so on. And no, or a very small increase on other filesystems where most files are larger, like /home e.g.

But I agree with you that between performance and a space increase on small files, performance is preferable.

/Sebulon

gkontos · #9 August 23rd, 2012, 17:46

You can always use SAS drives instead.

Sebulon · #10 August 23rd, 2012, 19:35

Quote:

Originally Posted by gkontos

You can always use SAS drives instead.

Whaaat?

/Sebulon

gkontos · #11 August 24th, 2012, 01:02

Quote:

Originally Posted by Sebulon

Whaaat?

/Sebulon

They are faster

Sebulon · #12 August 24th, 2012, 07:17

Quote:

Originally Posted by gkontos

They are faster

You have got to be janking my chain, right. You´re joking or something, cause I don´t follow. Speed has absolutely nothing to do with compression.

/Sebulon

gkontos · #13 August 24th, 2012, 10:26

Quote:

Originally Posted by Sebulon

You have got to be janking my chain, right. You´re joking or something, cause I don´t follow. Speed has absolutely nothing to do with compression.

/Sebulon

I was not referring to you nor to compression. I simply pointed out that SAS drives are much faster than SATA.

Sebulon · #14 August 24th, 2012, 13:04

Quote:

Originally Posted by gkontos

I was not referring to you nor to compression. I simply pointed out that SAS drives are much faster than SATA.

Let me just point out to you that the topic discussed here is:
"Space overhead is much higher on ashift=12 raidz"

I´m not trying to be rude or anything. It´s just that your comment had absolutely nothing to do with the subject. If you have a point, please explain further how that correlates to the subject. Would SAS drives somehow have a smaller space overhead than SATA with ashift=12?

/Sebulon

gkontos · #15 August 24th, 2012, 16:48

Quote:

Originally Posted by Sebulon

Let me just point out to you that the topic discussed here is:
"Space overhead is much higher on ashift=12 raidz"

I´m not trying to be rude or anything. It´s just that your comment had absolutely nothing to do with the subject. If you have a point, please explain further how that correlates to the subject. Would SAS drives somehow have a smaller space overhead than SATA with ashift=12?

/Sebulon

The answer regarding the overhead is here.

My later comment regarding SAS drives has to do with the fact that SAS drives will have better performance with ASHIFT=9 without the overhead.

Is that clear enough for you now?

Sebulon · #16 August 24th, 2012, 17:34

Quote:

Originally Posted by gkontos

The answer regarding the overhead is here.

My later comment regarding SAS drives has to do with the fact that SAS drives will have better performance with ASHIFT=9 without the overhead.

Is that clear enough for you now?

Yes, and I'm sorry, I totally missed that post! That was the missing piece

/Sebulon

gkontos · #17 August 24th, 2012, 17:42

Quote:

Originally Posted by Sebulon

Yes, and I'm sorry, I totally missed that post! That was the missing piece

/Sebulon

belon_cfy · #18 October 12th, 2012, 03:22

How about switching all the HDD to Seagate with smart align technology adopted so that I can maintain the same 512 block size but without sacrifice additional disk space and performance. Running raidz2 on 4k drive natively really waste more than double of space in my scenario.

Anyway , will try to benchmark once i got it to see what is the performance impact.

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode

Similar Threads
Thread	Thread Starter	Forum	Replies	Last Post
Using labels overhead? or something else	fgordon	General	4	May 30th, 2012 18:23
[Solved] ZFS raidz space	garig0	Storage	8	January 26th, 2012 14:16
Can't boot from ZFS on 8.2-STABLE with ashift=12	mainland	General	1	July 22nd, 2011 07:33
Jail performance overhead	kniveton	Web & Network Services	4	July 4th, 2010 22:33
Adding a new raidz to zpool does not give zfs create more space	audunfr	Installing & Upgrading	0	April 19th, 2009 12:39

The Following User Says Thank You to phoenix For This Useful Post:
xibo (August 22nd, 2012)