ZFS ZFS compression with email

U

usdmatt

Anyone else storing email on ZFS using lz4 compression?

I have a couple of mail servers and seem to find that compression ratios are fairly low when using default ZFS settings (apart from enabling compression).

One server is using 415GB of space and has a 1.25x compression ratio (so theoretically around 518GB of raw data).
I've just set up a new server which only has 2G of email at the moment, but a 1.08x ratio.

Maybe the way emails are split across records with the default settings isn't conducive to good compression, but it just seems like a dataset of nothing but text should compress pretty well. My root dataset, which contains a full FreeBSD installation has a 2.23x ratio...
 
The reason seems to be that most size factor in mail is attachments of binary data (usually compressed data like mp3, video, jpeg) encoded in base64. There is not much to compress.
 
Yeah I did consider attachments, but considering it's still text and usually much larger than the binary source file, was hoping that half decent compression would still be achievable.

I copied the 2GB dataset to a new gzip compressed dataset and got 1.45x, which is a decent improvement. As such I've decided to try changing the compress setting on the live dataset and see how it goes as mail starts to get moved to it.
 
usdmatt said:
Yeah I did consider attachments, but considering it's still text and usually much larger than the binary source file, was hoping that half decent compression would still be achievable.
Click to expand...

These are random patterns with few repeating strings. I just had a look at a mailfile that contains only mp3 attachments: lz4 compression factor is 1.015.
gzip will do better - as it takes a lot more CPU.
 
You must log in or register to reply here.
Back
Top