Solved Guidance for Windows/Linux User new to FreeBSD

V

Virtualisor

Hi,

I hope I'm creating this Post in the correct area.

I'm looking to learn more about the capabilities of FreeBSD, I've attempted to do significant research but some answers have not been forthcoming.

It is well known that ZFS is one of if not the best FS available but it is not supported by Windows or Linux in kernel at this point.

While Linux FS are missing a number of key features that are present in ZFS, Linux Storage technologies are continuously and quickly evolving.

Linux is able to use any or all of the following features as layers in almost any order as appropriate and needed - VDO (De-Duplication and Compression) RAID, LVM (including flexible discrete RAID of different Logical Volumes), Encryption, and a number of FS that can be used to format each Logical Volume. It is also possible to leave a Logical Volumes unformatted so it can be used for applications like databases OR shared to and formatted by a remote computer.

I'm looking to use FreeBSD to create a 1 or more NAS, to provide storage for other computers.

I want to, where possible use the extensive capabilities (raid, compression, de-duplication, protection from bitrot, etc) of ZFS to share general files to multiple networked computers using methods like NFS.

I also want the option to be able to define volumes that have most of the benefits and protections of ZFS but volumes would be shared to any networked computer using iSCSI and appear as an unformatted volume/disk.
The remote computer (initiator) would format the volume using whatever filesystem was appropriate for that Operating System (NTFS/FAT32 for Windows, XFS/Ext2-4/Reiser/Btrfs for Linux), and may include encryption

Is it possible to have most of the benefits of ZFS in an iSCSI volume that appears to be to the target computer as an unformatted volume?

I may choose to run Virtual Machines on separate hosts where the system and data storage may be partly or completely located on a NAS but the iSCSI volumes are attached to the remote host and the run on that remote host.

I may be using 1/2.5/5/10Gbps Network Interfaces between the NAS and target Computers to allow for near SATA speeds and share by use of iSCSI NAS Volumes to these networked computers?
----
Do you have what I believe is known in Linux as B-Caching in FreeBSD where a fast substantial SSD (0.5-2TB) is used to cache commonly accessed data from a number of larger volume disks or disk arrays so that following an initial period of caching commonly used data, the majority of read requests may be fulfilled from the Cache, and Writes may be also cached for a period of time to reduce the need to have disks continuously spinning, and prevent disks from frequently spinning up and down, perhaps arranging for a commit every 1-7 days OR at a point where the Write Cache is almost full.
----
Can you share links to documents, articles and howto's that demonstrated how I might achieve the above

I may have other questions as a follow-up to whatever responses I get here

I hope someone can help me here as whether I choose to use FreeBSD depends largely on what I can do with it.

Thanks in advance
 
Welcome to FreeBSD Forums.

Virtualisor said:
… Do you have what I believe is known in Linux as B-Caching in FreeBSD where a fast substantial SSD (0.5-2TB) is used to cache commonly accessed data …
Click to expand...

I'm not familiar with the phrase B-Caching. Closest might be this feature of OpenZFS:
  • persistent L2ARC.
<https://openzfs.github.io/openzfs-docs/man/7/zpoolconcepts.7.html#Cache_Devices>

fast substantial SSD (0.5-2TB)
Click to expand...

My own experience with far less substantial devices: L2ARC is amazing.

OpenZFS: L2ARC: CACHE vdev devices

Low end, but I'm endlessly delighted with the performance boost from L2ARC. Recent measurements graphed below.

Mid-to-high end is surely better. Enjoy.

1644786680989.png
 
  • Like
Reactions: drr
GrahamPerrin, SirDice,

Many thanks for this information, I will look into those suggestions and if I have any further questions I will come back to you.

Very much appreciated
 
Hi,
I have looked into these suggestions and they do indeed meet the criteria I listed above.

Right now, I have one supplementary question.

From my initial impressions, L2ARC is very resource intensive and can impact on performance of a other functions of a server including any standard caching and applications that might co-exist on the same server and greatly increase the cost to setup and operate such a server.

While exploring options for using Linux for some aspects of NAS storage, in particular it's more substantial support for it's own filesystems and possibly having newer implementations of NTFS Driver support that I would face the same potential issues with B-Caching.

The initial 2x NAS I setup are not spectacularly powerful or new machines so implementing and testing L2ARC/B-Cache in the same machine would be problematic!

Additionally it might be more costly and complicated to implement L2ARC/B-Cache in each and every NAS that I would use.

I have considered therefor that it might be worth de-coupling L2ARC and B-Cache from the main NAS server(s) and potentially have an intermediate L2ARC/B-Cache Server that sits between all NAS and servers.

By having a Dedicated L2ARC/B-Cache Server, it can be specified with more suitable CPU/RAM to fulfill the function with out necessarily impacting on normal storage capability.
Such a L2ARC/B-Cache Server can:
-selective cache individual NAS and not cache other NAS depending on content - For example streaming an AV Media NAS is unlikely to need such Caching
-selective cache areas of the NAS and pass-through other areas where Caching is not required
-install Medium in a single Caching Server rather than dividing it between multiple individual NAS giving better integrity and durability of the Cache particularly for Write Caching
-Options to split Caching Medium into Frequent Access/Infrequent Access and Write caching medium

This allows for some NAS to be idle/suspended or even shutdown for significant periods of time and only requiring them to be active for updates or access to infrequently accessed data that may have not reached the cache or have been pushed out

I realize this is rather an unusual and unconventional approach and that it might never have been done before, and it might be sometime before I could get to experiment with the approach, but if you have seen any information on previous attempts, I would appreciate any guidance

Thanks in advance for any guidance you can provide
 
Hello & welcome to this FreeBSD community!

Virtualisor said:
While Linux FS are missing a number of key features that are present in ZFS, Linux Storage technologies are continuously and quickly evolving.
Click to expand...
Oh yeah... this is one of the reasons why I really don't miss using Linux anymore. You set up a machine, you look away for five seconds, come back and the entire ecosystem changed and you have to sit through hours of reading documentation to understand all the new acronyms & buzz words before you figure that it's just the same shit only different.

From personal experience I can assure you that you will eventually appreciate having migrated to FreeBSD. One of the things I really appreciate about FreeBSD is that the stability of interfaces, technologies and such. I can look up notes, documentation, blog posts and so on from 10 or even 20 years ago and it still mostly holds true. The drawback of this is that FreeBSD "lacks behind" on the technology hype-train but luckily that's almost a non-issue for any personal to mid-professional usage. While this is partly due to man-power I'd also argue that development processes just take the time to do things properly rather than rushing some feature implementation as quickly as possible. But in return you get something that you can setup (such as a small home NAS), come back a couple of months later and there are not 18460 new things, 985 deprecated things and 274 vulnerabilities to evaluate.
However, the initial "getting up & running" time investment is somewhat larger. Totally worth it on the long run tho.
 
Thanks again SirDice,

Hi jbodenmann,

I totally get what your saying.

Unfortunately Linux and Windows are prevalent and for my next job there is a significant chance that I would have to work with one or both as well as the more popular business OS AIX/HP-UX/etc, so I cannot totally swear off these things.

I'm not a programmer or a Hard Core SysAdmin, so as a bit of a Jack of All Trades, I can quite easily Assemble a PC and install almost any OS on it and do some basic configuration, I don't have the more in depth experience!

I seek to improve my knowledge of these Operating Systems and setup some systems at home as a means to gain those and other skills on the off-chance that my next job requires such skills

I also seek to setup various aspects of the Operating Systems and software that can be run on them and may well use such tools based on BSD like pfsense/OPNSense, but I get much more satisfaction in trying to research and figure things out once I know the right tools and have a good idea of how to do it.

I will be attempting to use FreeBSD for much of that BUT I will also have to continue unfortunately to do the same sort of exercises with Linux so I have as much experience as possible and a wider pool of job prospects for the future.

I appreciate everyone sharing their experience so in time I might do the same.

🖖
 
Virtualisor said:
It is well known that ZFS is one of if not the best FS available but it is not supported by Windows or Linux in kernel at this point.
Click to expand...
You are wrong about that one:
On Linux ZFS is stable. You can do with ZFS whatever you can do under FreeBSD, and vice versa. Under Windows much of the functionality is already there as well, but it's still under heavy development.

Aside that FreeBSD is a boring OS, meaning it's rarely hops on certain hype trains. Which is a great thing.
 
Virtualisor said:
Can you share links to documents, articles and howto's that demonstrated how I might achieve the above
Click to expand...
I would suggest becoming familiar with the FreeBSD Handbook, and there's other good books available in the Handbook's parent folder. The Handbook has a whole chapter devoted to ZFS (Ch. 20), and the Forums do have their share of users who can help with explanations of how to work individual commands. I benefited from that - see this thread if you're curious.
hardworkingnewbie said:
FreeBSD is a boring OS
Click to expand...
I'd quibble with that assessment. FreeBSD is an interesting OS, because of its consistency and stability. With the ZFS example - FreeBSD went all-in on that, and seems to be sticking with it, which is a lot of effort. Various Linux distros/projects seem to be going after the hot thing of the day - BTRFS, JFS, ExtFS, whatever's convenient. Yeah, I know this makes me sound like a fanboi, but back in the day, I did get frustrated with trying to find coherent, consistent, and standards-compliant information that actually applies to my version of whatever I'm running. :p
 
In terms of OSes boring is good, because it means no unexpected gifts which keep on giving - like systemd on Linux - or unpleasant hiccups and crashes. FreeBSD is doing its job, is rock solid and therefore boring.

BTW JFS on Linux never took off. It's more like a file system which still happens to be there because it is still actively maintained, but nobody uses it.
 
hardworkingnewbie said:
You are wrong about that one:
On Linux ZFS is stable. You can do with ZFS whatever you can do under FreeBSD, and vice versa. Under Windows much of the functionality is already there as well, but it's still under heavy development.

Aside that FreeBSD is a boring OS, meaning it's rarely hops on certain hype trains. Which is a great thing.
Click to expand...
Thanks,

I'm aware of ZFS on Linux, but as a kernel module it will never achieve the full potential performance of one built into kernel. It is a tragedy that they cannot sort out all these incompatible license scheme so that could change

I wasn't aware of ZFSin, so I will have a look into that

The important thing here is that which ever platform and version of ZFS is used by someone, that they are ALL basically compatible and conform to OpenZFS v2 and succesive standards. The biggest problem with earlier ZFS releases was that every operating system and distribution took the basic code and modified it to include their own preferred extras which meant that it was very difficult to have portability and compatibilty that means you could take a ZFS volume and safely access it with FreBSD, any other BSD Distro, OpenIndiana and related distributions, Linux Distributions and Windows without fear of corruption.

Data integrity and portability is more important to me than anything else and the lack of a proper standard previously gave me pause in considering the File System for usage
 
Virtualisor said:
I'm aware of ZFS on Linux, but as a kernel module it will never achieve the full potential performance of one built into kernel.
Click to expand...
You do understand there's no difference? The Linux kernel is a monolithic kernel[*]. Just because you can load kernel modules dynamically doesn't change this fact. You seem to be confused with something like a microkernel.

[*] FreeBSD's kernel is a monolithic kernel too. And ZFS is loaded as a module on FreeBSD.

Monolithic kernel - Wikipedia

en.wikipedia.org en.wikipedia.org
en.wikipedia.org

Microkernel - Wikipedia

en.wikipedia.org en.wikipedia.org

Virtualisor said:
Reading about it, the documented requirements on RAM for example seem quite substantial.
Click to expand...
ZFS likes memory. More is better. But that doesn't mean it's not usable with less. I've had systems with only 2GB of memory running ZFS without problems. You just don't get the best performance, but it will still work without issues.
 
I do understand the nature of the Linux kernel, and in principle there is a performance penalty between running a service from within the kernel and running it as a module.

I was not aware that ZFS was run as a module in Free BSD, and while there may be the same penalty for FreeBSD, FreeBSD may be more efficient so this is less of an issue.

I'm aware that ZFS will function with smaller systems though certain features might not be supported without more memory such as Deduplication.

I am new to FreeBSD and not a HardCore Linux Admin, so I am simply attempting to get the right information, which you have provided, and will be evaluating it over the coming months.

My evaluation will be run on my older machines which are less powerful and have less RAM but ultimately I would plan to replace these with newer machines that are fully capable of taking advantage of the more demanding aspects of ZFS and give the best balance between storage capacity of HDD and speed of SSD while maximising the potential lifespan of these resources

Thanks for your Guidance 🖖
 
modules are to kernel what shared libraries are to userland binaries
once everything is loaded performance is the same
 
SirDice said:
No, there isn't. There is no difference between a module that's loaded and a module that's "builtin".
Click to expand...
I accept your assertion,

I'm not interested in disputing it, I can only comment that based on reports I have read previously have lead me to my prior conclusion. I cannot speak to their accuracy!

Primarily this is around the performance of the NTFS3.5 Kernel FUSE Module as used in Linux, and the New NTFS3 (Paragon) Driver that is being released into Linux Kernel 5.15.

Reports indicate that that latter is many times faster than the former, reported to be due to the need for extra interactions between the kernel and the module - however this may speak more to the code in the two drivers, than the differences in the use and function of kernel integrated drivers and kernel modules.

Perhaps there is some question in the reliability and accuracy of the reports here.

Honestly it is not that bigger deal - Where I will use ZFS, I will still choose FreeBSD over other Operating Systems.

I would like to close this specific sub-topic and concentrate going forward on learning about FreeBSD and OpenZFS 2, so I will get on with my evaluation and raise further question here as and when they arise.

Thank you all for your help todate.
 
Virtualisor said:
I see no option here to mark this thread as solved!
Click to expand...

For future reference, it's pictured here:

forums.freebsd.org

Hint: how to edit the title (subject line) of a topic (thread), or add the 'Solved' prefix

A number of topics are solved, but not marked as such. There seems to be no guideline. Please, can one be added? With a screenshot, or two, to make XenForo less ambiguous. https://forums.freebsd.org/forums/forum-rules-and-guidelines-required-reading.49/ Thanks
forums.freebsd.org forums.freebsd.org
 
SirDice said:
No, there isn't. There is no difference between a module that's loaded and a module that's "builtin".
Click to expand...
There's no performance or functional difference, but there is a political difference. Linus and his lieutenants are openly hostile towards ZFS, and are not above spreading FUD about it.

Given the above, I would not use Linux for ZFS because it's likely it will be broken in future kernel releases. Then again breakage and churn are par for the course in Linux as others point out so eloquently upthread.
 
Virtualisor said:
Primarily this is around the performance of the NTFS3.5 Kernel FUSE Module as used in Linux, and the New NTFS3 (Paragon) Driver that is being released into Linux Kernel 5.15.
Click to expand...
Sorry for staying a bit off-topic, but I wanted to clarify something:
In Linux, there are(*) two ways to run ZFS: with FUSE or with a kernel module. FUSE is very slow, being run in userspace, while the kernel module was running alright.
And FUSE being slow isn't specific to ZFS, but to all others FS, like the NTFS3.5 module you were talking about :)

(*) or at least there were, when I used ZFS on Debian, about 12 years ago.
 
You must log in or register to reply here.
Back
Top