Which file system do you prefer?

danbi said:
Then... discovered that ZFS on an USB FLASH is way faster (because of aggregated writes), than UFS. In situations like this, you realize that UFS is sort of primitive...

Why would you want to WRITE on a CF card?

It's why most of us use UFS for base system on CF/Pendrive/Flash, install on UFS, configure, mount as READ ONLY and all the rest (including all writes) goes to ZFS pools.
 
Because, the typical life cycle of a current USB FLASH drive is way longer than the typical life cycle of an FreeBSD install. There is no serious reason to 'save' it from writing.

Of course, all my special-purpose flash based systems are read-only (for the OS part), but this is because I make extra effort for these to be replaced, not upgraded. For application servers, including my own workstation, I see no reason to make it's boot volume read only. Especially with ZFS, where the chance of corrupted filesystem is much less significant then it is with UFS.

One reason to write on a flash drive is during upgrades. On a typical not-so-fast USB FLASH drive an 'make installworld' may take many, many minutes, sometimes hours. This is because it has to write lots of small files. During this time, if anything breaks, your system may end up unusable. You may sort of speed it up with mounting async for the duration of the upgrade, but not significantly.

With ZFS, it writes as fast as the flash part can and you are safe from crashes, both because of the nature of ZFS and also because you may use snapshots on your boot/root filesystem and revert back at any time, should something fail. Also, I think using ZFS on top of flash will wear the flash part less, because of the aggregate writes (remember, when you write to flash based storage, you have to rewrite an entire flash block each time -- this is why it is so slow).
 
Sorry for the off-topic.

danbi said:
Then, I started building most of my servers to boot from USB FLASH drives.
You mean you boot and work all day long with a system stored on a USB pendrive? How long have you been doing this and have you ever had problems due to this prolonged use?
 
I've always only ever used UFS with FreeBSD. Until ZFS becomes the de facto standard, I'll continue to use UFS. I may use ZFS for my special project in the near future (an autonomous robot that uses FreeBSD for its brain) but for my desktop systems, UFS it is for now.
 
Beastie said:
You mean you boot and work all day long with a system stored on a USB pendrive? How long have you been doing this and have you ever had problems due to this prolonged use?

My home computer uses a 2 GB USB stick (Cruzer Micro U3). Started with FreeBSD 7.0, upgraded with each release to 8-STABLE (post 8.1). No problems with it so far. Just the OS is on the USB stick.

I originally ran our secondary backup storage server using 2x 2 GB USB sticks in gmirror. They were horrible no-name sticks, but gmirror saved me and they were replaced with 4 GB CompactFlash disks. :)

For just the base OS (/{usr/}{lib|bin|sbin|libexec|share|boot}), which gets hardly any writes except during OS upgrades or kernel installs, even a 2 GB USB/CF disk will last for years.
 
I have used flash media for the OS for many years (early fbsd 4.x and bsd/os before that). Initially CF because USB support was not that good. I have never had failed out drive. Of course, just to be safe (because most such systems were far away) /var and /tmp were usually (if possible) ram disks.

The only time this has failed on me was with a pair of 512MB Kingston CF cards "lifetime warranty".. The weird thing was that those were always used read-only. At some point, they failed to read certain sectors. Writing over, they were happily working again.

Of course, you can be hit really bad by poor quality USB media. I had recently two 16GB HP v165w USB sticks fail on me, while scrubbing their ZFS. Bad is they were in mirror and both failed at the same time -- probably controller died, might be issue with the motherboard. The replacements work so far, but I am wary of this particular model. The "same" (v165w) 4GB sticks are ok -- I have few dozens in production servers.
 
danbi said:
On a typical not-so-fast USB FLASH drive an 'make installworld' may take many, many minutes, sometimes hours. This is because it has to write lots of small files.
For the initial setup I would suggest to use a file based MD and transfer that one en-block to the slice on the stick. You can keep that image around and update it, backup it, version control it... Updating the system currently running would not be possible in this way, but using another boot stick and switching them before rebooting can seriously save your lower back by having the previous, known to work, boot medium still at hand in case something went total wrong, like dropping some important devices from the kernel.

Another plus side for the paranoid would be that once ZFS is mounted the image can be compared and restored should someone decide to mess with your boot media.
 
What I always fear about live systems on pendrives is the high temperatures the drives can reach in just minutes. But I guess it is unfounded fear since internal disks get very hot too. :)
 
(if You need SWAP, omit on CF/Pendrive/SSD disks)
ZFS/SWAP zfs create -V 2g pool/swap

Why omit SWAP on an SSD? Write cycles on a small amount of space on a limited life disk? I am getting a laptop with SSD. Would you still recommend not having SWAP? thanks for your help.

Dave
 
I already toasted one USB stick, one of the small ones who only stick out about 4mm.
Placed it in the hub, checked later and it is completely passive now. So using sticks which have sufficient cooling surface seems to be in order for boot media.

richardpl said:
I prefer tmpfs or ufs on mdX.
I dream about having zettabytes of RAM.

... and a bullet proof suspend/resume I guess?
 
Crivens said:
I already toasted one USB stick, one of the small ones who only stick out about 4mm.
Placed it in the hub, checked later and it is completely passive now. So using sticks which have sufficient cooling surface seems to be in order for boot media.
Have you tried buying a waterproof one and plunging everything but the connectors into a cup of water (USB cord required)? LOL
 
Beastie said:
Have you tried buying a waterproof one and plunging everything but the connectors into a cup of water (USB cord required)? LOL

No, that would result in long discussions about the WAF of such a construction.

BTW, doing such things may result in cooling, but also in having hardware which does not work while being cooled. Even when the cooling media is not conductive it has different characteristics for magnetic or electric fields.
 
Now I am running PCBSD installation dvd on my laptop. Yes. PCBSD have option to install FreeBSD on zfs :) And have also 2 installs. PCBSD and FreeBSD :D
 
For me this is zfs all the way outside swap.
Zfs was not there when I moved from linux to FreeBSD but now I would be annoyed if I had to live without it.
I'm on my way to setup a new storage server and my choice is between FreeBSD and OpenIndiana(OpenSolaris) just because I can have native zfs on them. Linux is not an option with lvm or zfs on fuse.
 
danbi said:
Because, the typical life cycle of a current USB FLASH drive is way longer than the typical life cycle of an FreeBSD install. There is no serious reason to 'save' it from writing.

Of course, all my special-purpose flash based systems are read-only (for the OS part), but this is because I make extra effort for these to be replaced, not upgraded. For application servers, including my own workstation, I see no reason to make it's boot volume read only. Especially with ZFS, where the chance of corrupted filesystem is much less significant then it is with UFS.

One reason to write on a flash drive is during upgrades. On a typical not-so-fast USB FLASH drive an 'make installworld' may take many, many minutes, sometimes hours. This is because it has to write lots of small files. During this time, if anything breaks, your system may end up unusable. You may sort of speed it up with mounting async for the duration of the upgrade, but not significantly.

With ZFS, it writes as fast as the flash part can and you are safe from crashes, both because of the nature of ZFS and also because you may use snapshots on your boot/root filesystem and revert back at any time, should something fail. Also, I think using ZFS on top of flash will wear the flash part less, because of the aggregate writes (remember, when you write to flash based storage, you have to rewrite an entire flash block each time -- this is why it is so slow).

Then you should read the experience of a FreeBSD developer:

http://www.leidinger.net/blog/2010/11/02/are-usb-memory-sticks-really-that-bad/
 
Not all USB sticks are created equal (see my posts earlier in the thread). There are some 'no name' sticks that are rock solid, while some from reputable sources fail miserably. This has nothing to do with the 'USB FLASH' as such -- just poor quality control on commodity stuff. There are of course industrial USB FLASH drives out there :)

The thing that typically fails in the today's USB FLASH media is the controller. It may be due to overheating, bad design, bad power supply or whatever, but not because you 'write too much' to the media. Flash media has limited write cycles, but these are far, far more than any OS install will ever need. This was my point. You need not cripple the OS, when using flash media.

The issue with the way flash media works are more related to the fact that you do full page overwrites and an filesystem that groups writes, like ZFS does, helps great detail.
 
oliverh said:
Hm, I just got a shipment of 5 WD hard drives from newegg. 2 were DOA. Does that mean hard drives are really that bad or does it mean anecdotal evidence isn't really evidence?

danbi is correct, a USB drive preforming to spec is a great option. Like with any other hardware, preforming to spec is not always something to take for granted unfortunately.

http://www.bress.net/blog/archives/114-How-Long-Does-a-Flash-Drive-Last.html
 
Galactic_Dominator said:
Hm, I just got a shipment of 5 WD hard drives from newegg. 2 were DOA. Does that mean hard drives are really that bad or does it mean anecdotal evidence isn't really evidence?
Actually, it means that the supplier you got them from sucks... well, eggs. :p I've avoided newegg for years now because of their long-standing reputation for selling crap.
 
Eponasoft said:
Actually, it means that the supplier you got them from sucks... well, eggs. :p I've avoided newegg for years now because of their long-standing reputation for selling crap.

What does the supplier have to do with it? They got it from the same vendor. It's not like any of them take special care to take it out the box and gravity test it. Newegg is great, low prices and the shipments are actually what you ordered.
 
phoenix said:
PC-BSD installer can install to ZFS. And it can be used to install vanilla FreeBSD.

This needs to be more widely publicized. I, for one, had no idea about this. I looked at PC-BSD but turned away from it because their package management system struck me as totally ridiculous, and I didn't want KDE. And to be honest, sysinstall is a royal pain.

A graphical install of normal FreeBSD is probably what it would take to convince a lot of my BSD-curious (hahahaha) friends to take the proverbial plunge.

Perhaps we should steal their installer and allow people to download an .iso from freebsd.org that only installs normal FreeBSD by default? Seeing as finstall is going about as well as the war in Iraq.
 
The only thing I don't use ZFS for is swap. If someone has some information that says that the problems with using ZFS for swap are fixed, I'll switch that over too.
 
Galactic_Dominator said:
Hm, I just got a shipment of 5 WD hard drives from newegg. 2 were DOA. Does that mean hard drives are really that bad or does it mean anecdotal evidence isn't really evidence?

danbi is correct, a USB drive preforming to spec is a great option. Like with any other hardware, preforming to spec is not always something to take for granted unfortunately.

http://www.bress.net/blog/archives/114-How-Long-Does-a-Flash-Drive-Last.html

Well, he isn't starting a flamewar, he asks for response. As you can see, it's a FreeBSD-dev and I do think he is aware of the fine differences between different brands. There are no real numbers about the wear-out or quality per se of USB thumb drives, so comparing it to harddrives is like comparing Apple and Oranges.
 
Back
Top