UFS gjournal vs soft updates journaling

jarek6

jarek6

Hi
I'm now deploying new File Server running FreeBSD 10. I a little confused which journaling should be used for new file systems. newfs supports:
a) Soft updates journaling
b) Journaling on the new file system via gjournal(8)

New file systems have different sizes, from 200GB (/home/mail) where we will have a lot of small files (400-500k) up to 60TB (/vol1) where we will have mostly large files (600MB+ each)

Please advise
 
wblock@ said:
Whoa, back up. Aren't these two different things? SUJ journals filesystem metadata, and gjournal(8) is for the files stored?
Click to expand...
SU+J does both. Soft updates (SU) prevent damage in the filesystem metadata while Journalling (J) cares for the files that are stored on the file system.

gjournal should not be used anymore.
 
I disabled gjournal then, just leaved SU+J. Also I got some errors like below when I used gjournal.
What below messages means ? I'm 100% sure that HDD are in great condition, so it's not hardware problem especially with HDDs.

Code: 
Jan 11 11:46:37 Amnesiac kernel: fsync: giving up on dirty
Jan 11 11:46:37 Amnesiac kernel: 0xfffff80014c42ce8: tag devfs, type VCHR
Jan 11 11:46:37 Amnesiac kernel: usecount 1, writecount 0, refcount 80 mountedhere 0xfffff8005793f400
Jan 11 11:46:37 Amnesiac kernel: flags (VI_ACTIVE)
Jan 11 11:46:37 Amnesiac kernel: v_object 0xfffff8001ee40200 ref 0 pages 610 cleanbuf 77 dirtybuf 1
Jan 11 11:46:37 Amnesiac kernel: lock type devfs: EXCL by thread 0xfffff8004d7ef000 (pid 82346, g_journal switcher, tid 100342)
Jan 11 11:46:37 Amnesiac kernel:      dev da3.eli.journal
Jan 11 11:46:37 Amnesiac kernel: fsync: giving up on dirty
Jan 11 11:46:37 Amnesiac kernel: 0xfffff80014c42ce8: tag devfs, type VCHR
Jan 11 11:46:37 Amnesiac kernel: usecount 1, writecount 0, refcount 80 mountedhere 0xfffff8005793f400
Jan 11 11:46:37 Amnesiac kernel: flags (VI_ACTIVE)
Jan 11 11:46:37 Amnesiac kernel: v_object 0xfffff8001ee40200 ref 0 pages 610 cleanbuf 77 dirtybuf 1
Jan 11 11:46:37 Amnesiac kernel: lock type devfs: EXCL by thread 0xfffff8004d7ef000 (pid 82346, g_journal switcher, tid 100342)
Jan 11 11:46:37 Amnesiac kernel:      dev da3.eli.journal
Jan 11 11:46:37 Amnesiac kernel: GEOM_JOURNAL: Cannot suspend file system /home (error=35).
Jan 11 11:51:18 Amnesiac kernel: fsync: giving up on dirty
Jan 11 11:51:18 Amnesiac kernel: 0xfffff80014c42ce8: tag devfs, type VCHR
Jan 11 11:51:18 Amnesiac kernel: usecount 1, writecount 0, refcount 80 mountedhere 0xfffff8005793f400
Jan 11 11:51:18 Amnesiac kernel: flags (VI_ACTIVE)
Jan 11 11:51:18 Amnesiac kernel: v_object 0xfffff8001ee40200 ref 0 pages 610 cleanbuf 77 dirtybuf 1
Jan 11 11:51:18 Amnesiac kernel: lock type devfs: EXCL by thread 0xfffff8004d7ef000 (pid 82346, g_journal switcher, tid 100342)
Jan 11 11:51:18 Amnesiac kernel:      dev da3.eli.journal
Jan 11 11:51:18 Amnesiac kernel: fsync: giving up on dirty
Jan 11 11:51:18 Amnesiac kernel: 0xfffff80014c42ce8: tag devfs, type VCHR
Jan 11 11:51:18 Amnesiac kernel: usecount 1, writecount 0, refcount 80 mountedhere 0xfffff8005793f400
Jan 11 11:51:18 Amnesiac kernel: flags (VI_ACTIVE)
Jan 11 11:51:18 Amnesiac kernel: v_object 0xfffff8001ee40200 ref 0 pages 610 cleanbuf 77 dirtybuf 1
Jan 11 11:51:18 Amnesiac kernel: lock type devfs: EXCL by thread 0xfffff8004d7ef000 (pid 82346, g_journal switcher, tid 100342)
Jan 11 11:51:18 Amnesiac kernel:      dev da3.eli.journal
Jan 11 11:51:18 Amnesiac kernel: GEOM_JOURNAL: Cannot suspend file system /home (error=35).
 
lme@ said:
gjournal should not be used anymore.
Click to expand...
So can I assume you only mean this in the context of UFS? Because it's really nice not having to wait for a mirror rebuild.
When gjournal is configured on top of gmirror(8) or graid3(8) providers,
it also keeps them in a consistent state, thus automatic synchronization
on power failure or system crash may be disabled on those providers.
Click to expand...
Or are you saying gjournal should never be used for anything, in which case I'll put in a PR to update the manpage.

I'm guessing the OP enabled both J and j with newfs which seems to, at the least produce spurious warnings and at the worst, who knows? Maybe they should be mutually exclusive? I'd be happy to test if it's possible and raise a PR suggesting this if you think it worthwhile.
 
I used to be a big fan of SU+J, but after running into a lot of problems with it at home and at work, I would strongly recommend sticking with gjournal or ZFS for large filesystems.

The journal in SU+J I believe only covers filesystem metadata, as wblock@ noted. In practice, it almost never kicks in, especially after extended power outages. Additionally, SU+J does not support UFS snapshots, and last time I checked, it was not recommended for solid state drives. When it is unable to use the journal, the fsck is extremely long, in particular when geom_mirror is used (not sure how it compares against regular soft updates since I haven't done any formal benchmarks).

UFS (Regular) and UFS (SU) make sense on SSDs and small file systems.

I haven't found it especially easy to change a filesystem from SU+J or SU to use gjournal, because you either need to set aside a partition for the journal provider, or explicitly set aside space when creating the journal on a new data provider:

Code: 
gjournal label -s [SIZE] [DATA_PROVIDER]

According to the handbook, "Your RAM size should fit in 30% of the journal provider's space", and the manual page says "recommended minimum is twice the size of the physical memory installed".
 
Also, I've noticed a trend with SU+J in the case where the filesystem uses the journal correctly, it marks the file system as clean, but all too often when followed with a forced non-journaled fsck also in single user mode, it finds many filesystem errors. It should not mark the filesystem as clean in this case, but it does anyway. This makes me suspect that data loss could be occurring without a full fsck performed, which defeats the entire purpose of a journal.

It just happened last week, and this is not the first time. It has happened on a wide variety of server hardware, this time on a machine with hardware RAID and a battery backup. I haven't seen this happen with the other filesystems on FreeBSD, so I am very uncomfortable about recommending SU+J to anyone.
 
Last edited:
lme@ said:
SU+J does both. Soft updates (SU) prevent damage in the filesystem metadata while Journalling (J) cares for the files that are stored on the file system.

gjournal should not be used anymore.
Click to expand...

Actually, I've been looking into this recently and from all observations and everything I can see in the Kernel source and Marshal McKusick's documentation about the implementation it looks like the journal is only used to assist with repairing the metadata on a crash and does not actually cover the file data.

http://www.mckusick.com/softdep/suj.pdf
https://www.usenix.org/legacy/publi...gs/usenix99/full_papers/mckusick/mckusick.pdf

As far as anything saying that gjournal is no longer recommended, I can't find anything other than this thread saying that and hope that gjournal - if it is the only option for journalling of data blocks as well as metadata - isn't getting rusty through lack of use.

I can confirm that with 10.3-RELEASE gjournal is alive and kicking and with a little coercion it is possible to make the installer create a bootable root file system using a command line to partition, setting the fstab up in /tmp/bsdinstall_etc and the loader.conf up in /tmp/bsdinstall_boot.

Note that the stock, 10.3-RELEASE Kernel build has UFS_GJOURNAL enabled so it's still available without a special build.
 
Andrew Smith said:
Actually, I've been looking into this recently and from all observations and everything I can see in the Kernel source and Marshal McKusick's documentation about the implementation it looks like the journal is only used to assist with repairing the metadata on a crash and does not actually cover the file data.

http://www.mckusick.com/softdep/suj.pdf
https://www.usenix.org/legacy/publi...gs/usenix99/full_papers/mckusick/mckusick.pdf

As far as anything saying that gjournal is no longer recommended, I can't find anything other than this thread saying that and hope that gjournal - if it is the only option for journalling of data blocks as well as metadata - isn't getting rusty through lack of use.
Click to expand...

I think the "no longer recommended" claim really just amounts to Michael W. Lewis's advice/editorial in FreeBSD Mastery: Storage Essentials. I cannot help but wonder if he has since changed his recommendation, given that at the time he wrote the book, I also thought SU+J was the best choice. Unlike his experience, I've actually switched from SU+J to gjournal in many cases, and to regular SU (or without either) where it was not applicable (SSD, etc).

I can confirm that with 10.3-RELEASE gjournal is alive and kicking and with a little coercion it is possible to make the installer create a bootable root file system using a command line to partition, setting the fstab up in /tmp/bsdinstall_etc and the loader.conf up in /tmp/bsdinstall_boot.
Click to expand...

I think the main reason why the installer does not include an extra option for gjournal, at least when I last checked, is only because of the extra complexity of creating a large journal provider. I agree that gjournal remains a very important filesystem option on FreeBSD, and in my view the only serious one (other than ZFS) if you want to use a multi-terabyte drive.
 
beyert said:
Also, I've noticed a trend with SU+J in the case where the filesystem uses the journal correctly, it marks the file system as clean, but all too often when followed with a forced non-journaled fsck also in single user mode, it finds many filesystem errors. It should not mark the filesystem as clean in this case, but it does anyway. This makes me suspect that data loss could be occurring without a full fsck performed, which defeats the entire purpose of a journal.
Click to expand...

I just saw the same thing after frequent panics while using the UEFI version of 10.3-Release on a Mac Mini (3,1). A manual fsck found errors every time. Not very confidence building as the installer does su+j by default :(
 
Well, a quick me-too here. I have stability problems with the virtualbox-vm I use right now for poudriere (sometimes panics while waiting for HDD IO), but that's another story, point is, during reboot, the journal recovery says all done and marks the fs "clean", but booting from USB stick and fsck'ing the fs manually without journal always finds some corruptions to fix. I'll definitely not use SU+J on any production installation.
 
lme@ said:
SU+J does both. Soft updates (SU) prevent damage in the filesystem metadata while Journalling (J) cares for the files that are stored on the file system.

gjournal should not be used anymore.
Click to expand...
Why gjournal should not be used? What's wrong with it?
I don't find it is deprecated in Handbook https://www.freebsd.org/doc/handbook/geom-gjournal.html

I have issues with long fsck on file server after sporadic write errors on SU+j and seek for proper settings.
 
LiSergey said:
Why gjournal should not be used? What's wrong with it?
I don't find it is deprecated in Handbook https://www.freebsd.org/doc/handbook/geom-gjournal.html
Click to expand...
Well, first please keep the age of the thread in mind; it's nearly 2 years old so many things changed between then and now.

But having said that I don't share their opinion at all. I even have an older server running (386 based hardware) which uses both gmirror and gjournal and it has yet to fail me so far. The critter even survived a blown out PSU and a broken HD and a regular power failure. Nothing wrong with gjournal.
 
seems like I have to start a new thread "gjournal vs soft updates journaling in 2018", link this thread, and ask what have had changed since then :-/

I would like to learn real use case for gjournal.
What is beneficial vs SU+j and why or when.
Recommendations with reasons.
 
ShelLuser these old dinosaurs are, as the other kind, only bothered by a direct asteroid impact.

And gjournal has many uses, you can f.e. journal a msdos-fs. FAT with jourrnaling, yay!
Not being recommended does not mean being useless. And you can put the journal on a different drive, and indeed you should.
 
Any changes to SU+J since 2016?

Do you recommend SU or SU+J for production server using gmirror?

Maybe I avoid gmirror rebuild after a crash using SU+J?
 
CyberCr33p said:
Any changes to SU+J since 2016?

Do you recommend SU or SU+J for production server using gmirror?

Maybe I avoid gmirror rebuild after a crash using SU+J?
Click to expand...
Gmirror rebuild is acceptable. Do you see any reasons to avoid gmirror rebuild besides it's a low risk, still risk?

I still use SU+j in servers.
I didn't find reasons to switch to SU+geom_journal (J).
 
At the moment I use SU.

Today a server crash and rebooted. Then it crash again few times (after running the fsck automatically at boot). I have seen the same issue in the past and I was able to solve it by booting in rescue mode (mfsbsd) and manually run fsck. Fsck took a long time as the gmirror was rebuilding.

I try to find if SU+J will solve this issue.
 
You must log in or register to reply here.
Back
Top