ZFS Some questions about adding an SSD to my zpool

mariourk

mariourk

Hi,

I want to increase the performance of my zpool. One way of doing this, is adding an SSD for ZIL and/or L2ARC. However, I wonder if this will be effective. And if it is, how much effect will it have?

To figure this out, getting a good idea of the ARC and ZIL usage would be very nice. But how does this work with FreeBSD? Most ZFS tutorials assume Solaris is used. And those commands often don't work with FreeBSD.

The whole ZIL business is a bit confusing. Some say a ZIL needs to be mirrored. But according to this guy, mirroring a ZIL isn't really needed anymore. But, what happens if the SSD dies? Does the zpool continue to operate normally? Will ZFS fallback to ZIL in the zpool itself? Do I need to replace the SSD immediately? Or can I take my time to order a new one and replace it at whatever time fits me best, taking the lower performance I have in the meantime for granted? That isn't very clear from the answer. And how do I check if my zpool meets the needs (ZFS v19) for a non-mirrored ZIL?

I have only one spare tray and SAS connector, where I can place 1 SSD. I was thinking to divide it into 2 partitions and use 1 partition as ZIL and the other as L2ARC. The SSD I have in mind is a Samsung Pro 850. That is a SATA SSD, not SAS. Is this ok? Or is it a horrible idea? Does someone have experience with this, or a similar, setup?

I really want to be well informed, before getting down this road. Any tips, suggestions, explanations or pointing to good reading material would be most welcome.

Thanks in advance :)
 
It really depends on what your server is doing right now and where do you see the bottlenecks. In general, L2ARC helps for reads and ZIL for sequential writes. I would first upgrade my ZFS first though because 19 is a very old version. It would also help to know what version of FreeBSD you are running.
 
I want to get a good view what my server is doing. But I'm not sure how to do this with FreeBSD. That was one of the questions ;)

According to zpool get version Tank I have ZFS v28. I'm currently running FreeBSD 9.3
 
I want to get a good view what my server is doing.But I'm not sure how to do this with FreeBSD
Click to expand...

I shouldn't presume, but I'm pretty sure gkontos meant: What is the actual purpose of your server? If you're running something that gets a lot of sequential writes, like SQL databases that get a lot of traffic, then a ZIL can be helpful. If you're doing something that involves reading the same set of files over and over, like a file server, an L2ARC cache can be helpful. If you're running a home server for personal use and the only traffic it gets is from you, then an SSD might not be worth it.

Some say a ZIL needs to be mirrored. But according to this guy, mirroring a ZIL isn't really needed anymore. But, what happens if the SSD dies? Does the zpool continue to operate normally?
Click to expand...

The system should continue to work, but ZIL is effectively the ZFS implementation of a filesystem journal ("ZIL" stands for "ZFS Intent Log," as in a log of data intended to be written to disk in the future). If the SSD becomes unreadable during operation, any data waiting in the queue to be written will be lost. That's why mirroring is usually recommended.
 
It's not so much sequential writes that are helped by ZIL, but synchronous writes - i.e. writes where the application is going to wait until the data in physically on-disk before continuing. If you don't run many applications that heavily use sync writes, like databases or NFS*, then a separate ZIL (SLOG) device will probably not do much. (*In most NFS clients you can specify to use async, but some clients like ESXi use sync on pretty much every write)

It's actually quite common for a decent pool to perform sequential writes faster than an SSD would, but when your application is waiting for many small sync writes to complete, the difference in latency of an SSD can make a big difference. In a small pool (maybe a couple of disks or less) an SSD SLOG may make sequential writes faster (if they also happen to be written in sync mode) due to that one device having a higher write throughput than the pool disks, but that's not really its purpose.

Regarding losing a SLOG device. When ZFS writes, every single write gets cached in memory, then flushed to disk later as a transaction, regardless of whether it's sync or async. Because ZFS has to confirm sync writes have been completed to the application, and can't really wait until the entire transaction is done, it stores a copy of each sync write in the ZIL. If the server crashes and the in-memory transaction is lost, it loses all async writes that were waiting, but the sync writes (which it has already confirmed to the application were done) can be replayed from ZIL. With the ZIL on an SLOG device, if you lose the SLOG on a running system, the writes are still in RAM and you lose nothing. You actually need the server to lose power and for the SLOG to fail at the same time in order to lose data; I.e., If the server has sync writes waiting, then reboots, but comes back up without the SLOG device that had the backup copy of those sync writes, they are lost. The pool will complain and you'll have to forcefully import the pool and 'rewind' it to the last successful transaction.

When the SLOG is gone (either on a running system with no data loss, or a pool you've had to rewind), ZFS will just fall back to storing the ZIL on pool disks. You can run like that for as long as you want, as if you never had an SLOG in the first place, then add a new one later.

For a file server you may be better off with L2ARC (especially if you get lots of small random reads). If your clients are write heavy and do a lot of sync writes (such as VMs on ESXi over NFS) then an SLOG may be useful.

If you can stand to lose 5-30 seconds of sync data in the event of a crash, you can actually turn off ZIL which will make sync writes no different to async writes:
Code: 
zfs set sync=disabled pool/dataset
(not such a good idea if you're running something like VMs or databases off it that will be sensitive to small inconsistencies in their data)
 
There is a dtrace script out in the wild called zilstat. You can download and run to see if you will benefit with SSD based ZIL. I also usually load up zfs-stats package to see how my ARC is running.

I agree with others that have said upgrading your pool version to current is a really good idea. I think v19 is from about 2008. A lot of improvements have been made since then.
 
All right. That puts L2ARC higher on the todo-list. A crashing L2ARC SSD doesn't cause any problems, right? So, no need to put this in mirror?

As far as the benefit of a ZIL goes, this fileserver is used by macs (using AFP, with Netatalk-2.2.5) and PC's (using SMB, with Samba-4.1). From what I see with zpool iostat, the writes (operations) are higher than the reads.

Code: 
[mario@server ~]$ zpool iostat
               capacity     operations    bandwidth
pool        alloc   free   read  write   read  write
----------  -----  -----  -----  -----  -----  -----
Tank      4.58T  19.8T     40    154  1.04M   955K

Any other way I could monitor this?

And is a SATA SSD ok for this? From what I read on the internet, the IOPS of a Samsung Pro 850 is significantly higher than conventional SAS-drives. So, that shouldn't be a problem, despite not being SAS?

Can I easily remove the ZIL (or SLOG) and the L2ARC from the zpool, should I change my mind?
 
I agree with others that have said upgrading your pool version to current is really good idea. I think v19 is from about 2008. Alot of improvements have been made since then.
Click to expand...
My zpool is v28, from what I can tell from zpool get version. I ment that at least v19 is needed, to run a non-mirrored SLOG. If I'm running v28, I seem to meet the requirements to run a non-mirrored SLOG.

Code: 
[mario@server ~]$ zpool get version
NAME    PROPERTY  VALUE    SOURCE
Tank  version   28       local
 
Well said from usdmatt, synchronous rather than sequential, I sometimes tend to mix those words. Don't forget that you partition and use both ZIL and CACHE on the same SSD. As a rule of thumb ZIL should be approximately 1/2 or less of your RAM which is not much.

BTW. You might want to upgrade your pool unless you are keeping compatibility with Solaris.
 
Actually, the zpool is complaining about not being up-to-date

Code: 
status: The pool is formatted using a legacy on-disk format.  The pool can
    still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
    pool will no longer be accessible on software that does not support feature
    flags.

But since there seem to be some potentially severe consequences (the pool will no longer be accessible on software that does not support feature flags), I want to make sure that all the software is indeed compiled with the feature-flag and stay out of trouble.

Speaking of upgrading, to which version should I upgrade? I have no idea what the latest version of ZFS is, that FreeBSD 9.3 supports. Or is simply running zpool upgrade sufficient?
 
mariourk said:
Actually, the zpool is complaining about not being up-to-date

Code: 
status: The pool is formatted using a legacy on-disk format.  The pool can
    still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'.  Once this is done, the
    pool will no longer be accessible on software that does not support feature
    flags.

But since there seem to be some potentially severe consequences (the pool will no longer be accessible on software that does not support feature flags), I want to make sure that all the software is indeed compiled with the feature-flag and stay out of trouble.
Click to expand...

No, this means that the pool will no longer be able to be imported by systems runnings a lower version of ZFS. It has nothing to do with software compilation or packages.

mariourk said:
Speaking of upgrading, to which version should I upgrade? I have no idea what the latest version of ZFS is, that FreeBSD 9.3 supports. Or is simply running zpool upgrade sufficient?
Click to expand...

Yes, that would be enough and it will bring you to version 5000 with some extra feature flags.
 
No, this means that the pool will no longer be able to be imported by systems runnings a lower version of ZFS. It has nothing to do with software compilation or packages.
Click to expand...
Aha! Actually, that makes a lot more sense. Thanks for clearing that up. :beer:
 
You must log in or register to reply here.
Back
Top