Solved ZFS - SLOG ramifications it if fails?

Hello,

I'm in the process of rebuilding a FreeBSD server which primarily functions as a heavily used database. One of the tuning recommendations from FreeNAS and others that use ZFS is to have a SLOG device.

Can anyone tell me if it will kill my pool if the device fails?

thanks,
 
(edit) Oh dear, so much assumptions in this post ;)

The whole SLOG concept is mostly a FreeNAS devised scheme and FreeNAS is not necessarily FreeBSD; it's a derivative. Personally I find the concept to be ridiculous but... please be warned that heavy bias on my end is involved here.

So.. SLOG basically means Separated LOG device. And a log device, within the concept of ZFS, is (from zfs(8)):

Code:
     log     A separate-intent log device. If more than one log device is
             specified, then writes are load-balanced between devices. Log
             devices can be mirrored. However, raidz vdev types are not
             supported for the intent log. For more information, see the
             "Intent Log" section.
This is why I pick up the whole "SLOG concept" as ridiculous. It's nothing more but a log device, and a log device is already intended to be separate. So why the new "flashy" description?

So, about ZFS log devices... from that same manualpage:
Code:
     Log devices can be added, replaced, attached, detached, imported and
     exported as part of the larger pool.  Mirrored devices can be removed by
     specifying the top-level mirror vdev.
The whole idea behind log devices is synchronous transactions. So... are you using those? If yes, then this device becomes essential because it basically helps validating write requests; if that fails then your pool will definitely fail as well.

But if not... then there wasn't really a necessity for a log device in the first place ;)

Why would you want to use synchronous transactions in the first place though? What is the goal here?
 
My goal is to try to keep the pool from degrading as quickly. Since ZFS won't likely have defragmentation ability for the next 10-20 years, I'm trying to be preemptive.

For example, quoting vermaden:

ts not a database problem, its ZFS problem for all data.

Typical write is:

Code:
POOL: WRITE METADATA1
POOL: WRITE DATA1
POOL: ERASE METADATA1
POOL: WRITE METADATA2
POOL: WRITE DATA2
POOL: ERASE METADATA2

As You can imagine there are blocks of free space after METADATA
has been erased, which beautifully leads to fragmentation. This can be
limited by adding a ZIL device, even on a single disk, then it would
look like that:

Code:
ZIL: WRITE METADATA1
POOL: WRITE DATA1
ZIL: ERASE METADATA1
ZIL: WRITE METADATA2
POOL: WRITE DATA2
ZIL: ERASE METADATA2

So there would be no 'holes' ... but casual fragmentation is as usual.

I'm aware that FreeNAS is a separate project and I can't remember if it's a hard fork, but regardless. I'm just trying to research ways to help reduce performance degradation for a constantly growing database.

This approach seemed reasonable, but they did not bother to discuss the risks of using this approach. (Link for those interested.) So in summary, if I wanted to do this, It would be preferable to have a mirrored (S)eparate LOG? Also, can you tell me if I'm properly understanding vermaden's point?

Thanks for the prompt response. I'm just wanting to make sure that my rebuild is more optimized aside from "As much memory and hard drive space you can muster on a mirrored + stripped pool."
 
Listen to those two jgreco and cyberjock. They know what they're talking about.
As jgreco pointed out, ZFS uses a ZIL whether its on the disks or on SSDs. A SLOG is just a a seperate ZIL. As far as a SLOG making a pool slightly less fragmented; I've never heard of that, I'd say don't worry about it.

The internet is plagued with ZFS, SLOG, L2ARC copypasta, so I'll continue...

if that fails then your pool will definitely fail as well.
No. The consistency of the application data may be compromised, but the pool will not fail. As stated, yes it is a log of synchronous transactions. Though, in normal operation data never flows "through" the SLOG. Data is only read from the SLOG on the next import after a power failure. Therefore, if your SLOG fails catastrophically during regular operation, your pool will not. ZFS will learn the device is'nt accepting writes anymore and mark the pool as degraded. If your SLOG fails during or right after a power failure, you will lose whatever data is in the SLOG, but your pool will certainly not "definitely fail as well". If the SLOG is mirrored as it should be, ZFS will figure which mirror device holds the correct data.

risks of using this approach

If you implement a SLOG incorrectly, it will be no safer than simply setting "sync=disabled" pool wide. Correct implementation means mirrors of capacitor or battery backed flash. These are not suggestions, these are functional requirements. That said, you do it right, yeah you'll be pretty safe, and benefit from the performance of SSDs for synchronous writes.

Why would you want to use synchronous transactions in the first place though? What is the goal here?
Synchronous writes exist because they are your most important data. Otherwise an application has no guarantee of consistency. Databases issue a lot of synchronous writes.

freebsdinator , since you are running a database, I absolutely advise you to use a mirrored SLOG. This is a good application for the feature.
 
nihr43, so just to confirm, a failed SLOG would have an impact on the database data, but not the pool integrity itself? Or is it just unlikely to affect the pool?

jgreco noted that a poor choice would be the same as setting sync=disabled.
If you're not going to get a suitable device for SLOG, then you are putting your pool at risk, and you might as well just turn on sync=disabled and be done with it. Lightning speed without all the pesky ZIL concerns.

From my understanding, it would mean that some devices won't properly sync the data before writing, not that if a SLOG failed, your pool is done. Do you agree?

I'm not trying to split hairs; I just want to make sure I understand the implications before I finalize the pieces that I need for the rebuild, but yes, I am a fan of the idea of the mirrored SLOG even if it just ensures the machine can keep running without much degradation until the next maintenance window.

Thanks again!
 
If a SLOG fails while the system is running, the pool will continue without issue. If the system crashes and you lose SLOG, you may lose several seconds of sync writes (writes that were written just before the crash). The pool will still be completely consistent but those missing writes could easily cause issues with software such as databases.

I personally would mirror the logs as I'd rather just have a degraded pool and replace a disk after the unlikely event of a combined crash and disk failure. The alternative is to have to force import the pool and have missing data; Data which could be an important part of some database.

and FreeNAS is not necessarily FreeBSD; it's a derivative. Personally I find the concept to be ridiculous but... please be warned that heavy bias on my end is involved here.

Might want to reconsider your bias as SLOG is a fundamental part of ZFS and nothing to do with FreeNAS

This is why I pick up the whole "SLOG concept" as ridiculous. It's nothing more but a log device, and a log device is already intended to be separate. So why the new "flashy" description?

ZFS always has an intent log (ZIL). This is normally on the pool disks. You have the option to use an independent disk for the ZIL, hence the name separate log device - because you're separating it from the pool data disks.

The whole idea behind log devices is synchronous transactions. So... are you using those? If yes, then this device becomes essential because it basically helps validating write requests; if that fails then your pool will definitely fail as well.

But if not... then there wasn't really a necessity for a log device in the first place ;)

Why would you want to use synchronous transactions in the first place though

It's down to the application to choose sync or async writes. Even a general purpose computer probably gets a mix of the two.

Reading the edit I think you already realise most of what you posted is complete nonsense, but there's little point replying if you don't understand the subject and are just spreading misconceptions.

The information in nihr43's post is correct.
 
I would recommend to do a test run first. I use ZFS on a desktop. But there is still some activities on ZIL. In fact, one-seventh of the writes came from ZIL.

zpool iostat -v

Code:
tank                       744G  1.45T      4      5  1.02M  78.2K
  raidz1                   744G  1.45T      4      5  1.02M  68.0K
    gpt/disk0                 -      -      3      1   344K  38.1K
    gpt/disk1                 -      -      3      1   349K  38.0K
    gpt/disk2                 -      -      3      1   351K  38.0K
logs                          -      -      -      -      -      -
  gpt/log0                 400K  9.94G      0      0      0  10.2K
 
Back
Top