ZFS on geli, bad intent log on reboot

[Tassilo]

Hi,

I'm discovering ZFS at the moment and was missing encryption, so combining it with geli sounded pretty flexible, however I ran into a subtle problem. My hard drive has one slice that I encrypted with geli, and the resulting .eli device is then set[]up as a ZFS storage pool. To be precise:

geli init -b -B /mnt/ada0s3d.eli -e Blowfish-CBC -s 4096 -a HMAC/SHA256 /dev/ada0s3d
zpool create -m /encrypted tank /dev/ada0s3d.eli

I decided to have /var, /usr and /tmp under /encrypted and symlink to those folders. This by itself works great and booting asks me correctly for the passphrase, mounts the ZFS, etc.. However, when I then reboot, it drops me into single user mode with a bunch of GELI errors that it can't authenticate some data correctly. Running zpool status -v tank shows a FAULTED state and:

status: An intent log record could not be read.

zpool clear takes care of it and I can restart and everything is fine. However, the same happens on the next reboot.

Looking into this a bit more and wrapping my head around the ZIL, it kinda of sounded like as if the reboot (shutdown+cold start) happens without waiting for the last few transactions to be written. According to the ZFS docs the sync happens apparently every 5 seconds (?), so I was wondering if geli detaches without waiting for ZFS to flush it' pending writes? Unfortunately, I wasn't able to find a lot of info about this, however, setting

sync=always

, thus sacrificing performance, fixed it for good (as I assumed it will):

zfs set sync=always tank

Anybody any idea what's causing this? I don't want to keep the sync set to "always". I consider setting it to "always" before shutdown and standard on startup, as a workaround, however, I'm not sure what the problem is at first hand.

I'm on 9.1 with ZFS 5 and zpool 28.

Thanks!

 

[Savagedlight]

I had some issues with GELI incorrectly closing/detaching when used in combination with ZFS on FreeBSD 10.0-RELEASE. It manifested in some seemingly random errors, including being unable to remove log devices (all zpool-related commands would hang afterwards, and log device wouldn't be removed). The fix I applied was to not have GELI automatically detach devices on last close.

This can be achieved by adding geli_autodetach="NO" to /etc/rc.conf, and rebooting. There may be less intrusive ways of applying the change, but I didn't check as rebooting was acceptable when I did this.

I hope it helps!

 

[Tassilo]

Hi,

thanks for your answer and suggestion. I tried it, but unfortunately, it didn't make a difference...

I actually had the same problem this morning although using sync=always, however this was the first and only times in days, whereas with sync=standard it happens on every reboot. So, might be something more subtle, after all...