HOWTO: Modern FreeBSD Install RELOADED (vermaden way)

@zeroseven

I have one sparc64 machine [*] available, but its in production at least till 2010/09/1 so I may try the same setup under sparc64, but not until that date (maybe even later, projects, as they often do slip little or more in time).


[*] Sun Ultra 45 if it comes to details.
 
I think I will give it a shot today then, mine is only in hobby status, so I can't really do any damage. I have a question about the AHCI mode though, because it's sparc and older architecture do I assume that part isn't applicable? Also tuning zfs in your guide is i386 specific, I'm wondering if others do not need to do this or if I should dig around for sparc specific tuning to accomplish the same thing?

I appreciate the quick response, I'll post what I encounter if it's relevant, definitely if I get it to work. Thanks.
 
@zeroseven

I havent seen any SPARC64/SPARC related info about AHCI, it propably depends what controller You have there and if it supports AHCI mode, if You kldload AHCI module and You will not have AHCI hardware, nothing bad will happen and You still will have /dev/ad* disks instead of /dev/ada* disks, but my guide operates on labels and ZFS which are not based on device names, so You cant shoot yourself in the foot here.

This Sun Ultra 45 box has some SATA drives, so if SATA is not i386/amd64 sprcific, then so is AHCI, but dunno if some developers put any love to sparc64 to add code for AHCI on it.

Anyway, please report how this guide 'works' on SPARC, propably no one tried it on Sparc ;)
 
@vermaden

I'll start with a little background here, I'm trying to accomplish this install on a Netra T1 105 with two scsi hdd. So, naturally, because it's headless I'm having to work over a serial line, no problem. I am fortunate enough to have a cdrom drive included stock with the system, however it obviously doesn't read a dvd. That's the first snag. I thought maybe there would be a method to umount the livefs disk in order to mount disk1 when I need to install the base system and boot. Unfortunately I don't know how to / a safe way to do this.

So, I decided I would settle for a modified version of the "vermaden way" for now. Installing a base system with disk1 on a 512M root, swap and a third partition as a place holder for zfs.

I know this defeats some of the purpose of disk labels instead of disk references, i.e. rootfs/root, basefs/usr, etc... However, the only affected mount point is the rootfs/root. Once in single user mode I:
Code: 
zpool create basefs da0e da1
zfs create basefs/usr
zfs create basefs/var
zfs create basefs/home
Then set the appropriate mount points to /usr, /var, /home and export the pool. Then:
Code: 
zpool import -f basefs
Everything mounted correctly. However, if you are quick to analyze, I made a fundamental mistake when I imported. I overlapped /usr with basefs/usr so when I left single user mode, whatever was needed in /usr was no where to be found. I'm pretty sure that's easily fixed, by just setting the basefs/usr to a temporary mount point other than /usr and a mv /usr /tempbasefs/usr then reset the basefs/usr mount point back to /usr.

I didn't have enough time to try this, I'll give it a shot tomorrow.
 
zeroseven said:
I am fortunate enough to have a cdrom drive included stock with the system, however it obviously doesn't read a dvd. That's the first snag.
Click to expand...

I assume that Your machine does not allow You to boot from USB/pendrive?

I know this defeats some of the purpose of disk labels instead of disk references, i.e. rootfs/root, basefs/usr, etc... However, the only affected mount point is the rootfs/root.
Click to expand...
That is not a problem, You may LATER add those labels and modify /etc/fstab to boot from them, so You do not loose anything here.

As for 'problematic' /usr stuff, You may as well delete the whole /usr partition (along with data), and extract again base dataset, so it will (again) extract everything to / and to needed /usr partition.
 
zeroseven said:
I didn't have enough time to try this, I'll give it a shot tomorrow.
Click to expand...
Good luck, I see that all 'major' problems are solved here.

You may also want to go phoenix (longtime FreeBSD community member) with installing both / and /usr on UFS2, and put all the rest on ZFS, IMHO phoenix way is better for server while mine is better for laptop/workstation/desktop.

TO ADMINS: I needed to spread my reply into 2 posts, because I got 503 error from varnish.
 
@vermaden

Thanks for all the tips and the good write up. Eventually I just settled for a UFS2 / and /usr like you suggested. It makes more sense I suppose. I'm working with two very small 4 gig scsi drives and it would have been nice to have /usr on the 6.8 gig zfs tank so it could expand as needed.

Really this was all just an exercise to see if I could actually do this. If I had a dvd drive I would have been able to get it done with no problems. No worries. The whole process is very similar on sparc from what I could tell. You don't have to worry about setting up the master boot record and you can use sunlabel instead of bsdlabel to setup the partitions and for the most part I'm certain the rest of your write up is the same.
 
Am I right in thinking that, starting from a single-disk setup, you can simply add vdevs to the tank to increase the space available?

Specifically, can I add another drive to the system and have zfs expand /usr and use the space on the new drive?
 
loop said:
Am I right in thinking that, starting from a single-disk setup, you can simply add vdevs to the tank to increase the space available?

Specifically, can I add another drive to the system and have zfs expand /usr and use the space on the new drive?
Click to expand...
Yes, its 100% expandable, ZFS by addinganother mirror/raidz/raidz2 (which will make RAID0 of them), and also by GEOM Mirror which will add another partititons to current mirror setup.
 
After FreeBSD 8.1-RELEASE has been released, we can now upgrade our ZPOOL(s) to newer (v13 --> v14) version, below You will find simple way to achieve that.

First check what is currently on You system:
Code: 
# [color="Blue"][B]uname -m -r[/B][/color]
8.1-RELEASE amd64

# [B][color="Blue"]zfs get version basefs[/color][/B]
NAME    PROPERTY  VALUE    SOURCE
basefs  version   3        -

# [B][color="#0000ff"]zpool list -o version[/color][/B]
VERSION
     13

Now lets proceed with the zpool/zfs upgrade procedure:
Code: 
# [B][color="#0000ff"]zpool upgrade[/color][/B]
This system is currently running ZFS pool version 14.

The following pools are out of date, and can be upgraded.  After being
upgraded, these pools will no longer be accessible by older software versions.

VER  POOL
---  ------------
13   basefs

Use 'zpool upgrade -v' for a list of available versions and their associated
features.

# [color="#0000ff"][B]zpool upgrade basefs[/B][/color]
This system is currently running ZFS pool version 14.

Successfully upgraded 'basefs' from version 13 to version 14

# [B][color="#0000ff"]zfs upgrade[/color][/B]
This system is currently running ZFS filesystem version 3.

All filesystems are formatted with the current version.

After the upgrade we can check again our ZPOOL(s) version:
Code: 
# [B][color="#0000ff"]zpool list -o version[/color][/B]
VERSION
     14

Your pool is now upgraded to newest 'release' zpool/zfs version.
 
Nice guide.

but is setting / to be mounted read in the last step necessary? I get a bunch of errors when booting, like "not able to write to /etc/hosts.conf, file system is read only" ect.

And it doesn't seem practical having to remount the file system as rw everytime I wanna edit something like /etc/rc.conf. Or am I just doing it wrong?
 
jj said:
And it doesn't seem practical having to remount the file system as rw
everytime I wanna edit something like /etc/rc.conf. Or am I just doing it wrong?
Click to expand...

How often do You edit /etc/rc.conf (or anything else in /etc)? :)

I do it that way:
Code: 
# mount -w /
# vi /etc/rc.conf
# mount -r /

... You can even creata alias/function for that in Your shell.
 
I just created a mirrored USB pen setup by following this guide. It works great, except that when I pull out one of USB pens and reconnect, it doesn't notice that I've reconnected the USB pen. With another setup I have with a regular FreeBSD install (with only UFS partitions) this works fine. What could be wrong?
 
olav said:
I just created a mirrored USB pen setup by following this guide. It works great, except that when I pull out one of USB pens and reconnect, it doesn't notice that I've reconnected the USB pen. With another setup I have with a regular FreeBSD install (with only UFS partitions) this works fine. What could be wrong?
Click to expand...

Provide outputs of these commands BEFORE and AFTER unlugging/attaching the USB pendrives:
Code: 
# gmirror status
# zpool status
 
Before outplugging:
Code: 
zbtankX# gmirror status
         Name    Status  Components
mirror/rootfs  COMPLETE  da0s1
                         da1s1
zbtankX# zpool status
  pool: basefs
 state: ONLINE
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        basefs      ONLINE       0     0     0
          mirror    ONLINE       0     0     0
            da1s3   ONLINE       0     0     0
            da0s3   ONLINE       0     0     0

errors: No known data errors

When I unplug I get this message on screen:
Code: 
ugen4.2: <Corsair> at usb4 (disconnected)
umass0: at uhub4, port 2, addr 2 (disconnected)
(da0:umass-sim0:0:0:0): lost device
GEOM_MIRROR: Device rootfs: provider da0s1 disconnected.

After unplug
Code: 
zbtankX# gmirror status
         Name    Status  Components
mirror/rootfs  DEGRADED  da1s1
zbtankX# zpool status
  pool: basefs
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        basefs      DEGRADED     0     0     0
          mirror    DEGRADED     0     0     0
            da1s3   ONLINE       0     0     0
            da0s3   REMOVED      0   335     0

errors: No known data errors

Reattach:

Same as after unplug. No message on screen(with the other setup I get a message telling me that the usb pen has been reattached)
 
olav said:
Code: 
zbtankX# gmirror status
         Name    Status  Components
mirror/rootfs  DEGRADED  da1s1
Click to expand...

For gmirror, first try activate, if that does not helps, then forget/insert should do, check man gmirror for more details.

Code: 
# gmirror activate rootfs da0s1
# gmirror rebuild rootfs da0s1

Code: 
# gmirror forget rootfs da0s1
# gmirror insert rootfs da0s1
# gmirror rebuild rootfs da0s1

olav said:
Code: 
zbtankX# zpool status
  pool: basefs
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
        attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
        using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://www.sun.com/msg/ZFS-8000-9P
 scrub: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        basefs      DEGRADED     0     0     0
          mirror    DEGRADED     0     0     0
            da1s3   ONLINE       0     0     0
            da0s3   REMOVED      0   335     0

errors: No known data errors
Click to expand...

I would try that:
Code: 
# zpool attach basefs da0s3




Reattach:

Same as after unplug. No message on screen(with the other setup I get a message telling me that the usb pen has been reattached)
Click to expand...
What about dmesg messages after plugin?
 
There is no /dev/da0 when I replug, dmesg says nothing. I really don't think this has anything to do with gmirror nor zfs.

When I use camcontrol I get this:
Code: 
zbtankX# camcontrol rescan all
Re-scan of bus 0 was successful
Re-scan of bus 1 returned error 0xa
Re-scan of bus 2 was successful
Code: 
zbtankX# camcontrol devlist
<TOSHIBA MK1234GSX AH002E>         at scbus0 target 0 lun 0 (pass0,ada0)
<Corsair Flash Voyager 1100>       at scbus2 target 0 lun 0 (pass2,da1)

Any other ideas about what I can do? Is it possible to restart the usb service/module?
 
@olav

Does these disks appear after reboot? (I mean if they are attached before You start the system)
 
Yes. Before I start the system they're both attached and appear with dmesg. I find this really weird, I've tried several other usb pens(different brands too), but none of them show up in dmesg if I attach them after the boot process.
 
mystique said:
I found this link:
http://www.fscker.ca/rc/2010/05/20/migrate-zfs-mirror-to-raidz-on-freenas/

but it doesn't look like they have /usr and /var and /home running on their pool..
Click to expand...
The guide is ok, following it would allow You to 'migrate' from RAID1 into RAID5, as for /usr, /var and so, its just a matter of datasets, just add needed datasets with appreciate -o mountpoint=/where arguments and put whatever You like on the pool.
 
@vermaden

Hello again, it has been a while. I recently acquired an intel machine a compact flash drive and a sata hdd. I was following your recent post pertaining to that sort of setup, however I am having difficulty with the zfs pool being persistent through a reboot. After a reboot only the root label is mounted and it kicks me to single user mode. From single user mode, the zfs pool is no longer importable or even recognized.

If I jump back into the fixit mode on the install cd, I can import and re-export the zfs pool, reboot and then re-import through single user mode, exit and the system runs.. until another reboot. I have the appropriate /boot/loader.conf and /etc/rc.conf with zfs enabled and loaded.

I will continue to scour over your guide to see if I missed something and look around, but I was wondering if you could think of anything off the top of your head I might be missing.
 
I figured it out.. conflicting mounts. the UFS /var/db/pkg in /etc/fstab creates a problem because it tries to mount before the zfs pool. Var doesn't exist yet and results in an error and boots to single user mode. I removed the entry from fstab and everything booted and mounted normally. Now my question is.. Other than manually mounting the /var/db/pkg every time I reboot.. is there a way to make it the last partition mounted?
 
You must log in or register to reply here.
Back
Top