Currently my FreeBSD 8.2-RELEASE/amd64 box is setup with the following disk structure:
- 3 1TB WD Black HDDs
- 2 500 G SATA2 HDDs
I have them sliced up as follows:
- ada0 (1 TB)
---/dev/label/slash 1.9G 488M 1.3G 27% /
---/dev/label/tmp 3.9G 795M 2.8G 22% /tmp
---/dev/label/usr 893G 61G 761G 7% /usr
---/dev/label/var 1.9G 408M 1.4G 22% /var
- ada1 (500 GB)
--- part of /backup raidz1 zpool
- ada2 (1 TB)
--- /dev/label/video 902 249 580 30% /video
- ada3 (500 GB)
--- part of /backup raidz1 zpool
- ada4 (1 TB)
--- ada4s1a (~500 GB)
----- part of /backup raidz1 zpool
---ada4s1d (~500 GB slice)
----- /dev/label/media 451 190 224 46% /media
I have another 1 TB WD Green drive and 2 400 G drives I'm planning to use (1 in a zpool, 1 as a spare). What I'm planning to do is set things up as follows:
- 4x1 TB (raidz) to be used for /usr and /data (all the stuff under /usr/data (~70G), /video and /media will go there after the migration)
- 3x400G (2 400G slices from 500G disks + 1 400G disk) (raidz) to be used for /backup with the other 400G drive as a spare
- 2x100G (2 100G slices from 500G disks) (gmirror, raid1) to be used for /, /var and /tmp
If I didn't have this SATA optical drive, I'd use that other 400G disk and have 2 parity disks in the backup raidz, but at least I'll have the spare.
So given that, I'm trying to plan what all I will need to do to move the data around/etc to re-organize the disks. I wanted to run my plan by you all and see if it looks good before I start.
Currently I'm using all 6 SATA ports on this motherboard (Asus P5Q-Pro) on the Intel controller - the 5 SATA disks, then a SATA bluray/dvd/cd writer. I'm trying to decide which disks to put on the other 2-port controller (Silicon Image Sil5723). Actually, I need to see if that controller is even supported. Assuming it is, I'm thinking I'll put the bluray drive and the 400 drive on there, although perhaps I shouldn't trust one of my disks in the /backup raidz to be on that 3rd party controller?
I'm booting from ada0's MBR (I think), so that means I will need to install the boot loader on the gmirror once that's setup then move one of those disks to the first SATA port (or change the BIOS drive order so that disk is first), right? With that in mind, here's my plan of attack:
1. make sure the other SATA controller works. If not I'll have to wait until I can purchase a PCI-E controller add-on, or forgo these plans for a while. Update fstab to temporarily remove the mounts for /video and /media (will mount them manually later). Question: when is the best time to remove any glabels I have since I'll want to re-create them on the new gmirror later?
2. Install the 2 new drives. Move the disks around such that the 1st 500 G drive is ada0, 2nd 500 G drive ada1. Then the next 4 (ada2 - ada5) for the 1 TB drives. On the Silicon Image controller, I'll put the 400G drive on port0 then the bluray drive on port1. I'll confirm in the BIOS that the old ada0 that I'm currently booting from is set as the "first" disk (until later).
3. Boot normally and copy the existing stuff in /backup to /video. Then destroy the zpool to free up the two 500 G drives and the 500 G slice on the one 1TB drive.
4. Slice up the 500G disks such that I have two 400 G partitions equal to the size of a full-disk slice on the 400G disk. Then calculate whatever is left and create those for the gmirror. I think this will mean a ~100G ada0s1 and ada1s1 and 400G ada0s2 and ada1s2, right? So I'll gmirror ada0s1 and ada1s1 then create my slices in gm0s1 for /, /tmp and /var) - Question: can I create gm0s1b for swap or do I have to waste a few G on both ada0 and ada1 to create swap?
5. Format and create /, /tmp and /var on the gmirror, then install the boot loader to the gmirror (or ada0). Copy over the existing /, /tmp and /var and (temporarily) /usr (except /usr/data) to / on the gmirror since we'll need a usable /usr when creating the "data" zpool.
6. Setup the 400 G disk such that I now have three 400 G partitions (ada0s2, ada1s2, ada6s1) of equal size. Then create the backup zpool from these 3 400G chunks and mount it at /backup.
7. Copy all of my data from /usr, /media and /video (temporarily) to the /backup ZFS mount.
8. Reboot and change the disk order in the BIOS so the 500G disk is booted from (the new ada0) and boot with the gmirror setup as the root fs (/ (including temporary /usr), /tmp, /var).
9. The data that was on the 1 TB disks is now all copied to /backup, clear the partition tables on those (dd ...) then create new disk-spanning slices to use for the 4-disk "data" zpool. Once the zpool is created, copy the /usr/data, /video and /media stuff from /backup back to /usr/{data,video,media} on the new zpool and copy the old backup data from the video sub-dir back to /backup, then remove the usr/data, video and media data from the new /backup.
10. Update fstab on gmirror's /etc and (re)create glabels for slash (/), var (/var), tmp (/tmp) and swap. Ensure zpools are setup and mounting properly and set mountpoints. Delete the /usr from the gmirror / then ensure /usr is the mountpoint for the "data" zpool.
Am I missing anything major here? I think my only uncertainty is dealing with the glabels, but worst case scenario I can mount/use things directly from adaX instead of the labels.
- 3 1TB WD Black HDDs
- 2 500 G SATA2 HDDs
I have them sliced up as follows:
- ada0 (1 TB)
---/dev/label/slash 1.9G 488M 1.3G 27% /
---/dev/label/tmp 3.9G 795M 2.8G 22% /tmp
---/dev/label/usr 893G 61G 761G 7% /usr
---/dev/label/var 1.9G 408M 1.4G 22% /var
- ada1 (500 GB)
--- part of /backup raidz1 zpool
- ada2 (1 TB)
--- /dev/label/video 902 249 580 30% /video
- ada3 (500 GB)
--- part of /backup raidz1 zpool
- ada4 (1 TB)
--- ada4s1a (~500 GB)
----- part of /backup raidz1 zpool
---ada4s1d (~500 GB slice)
----- /dev/label/media 451 190 224 46% /media
I have another 1 TB WD Green drive and 2 400 G drives I'm planning to use (1 in a zpool, 1 as a spare). What I'm planning to do is set things up as follows:
- 4x1 TB (raidz) to be used for /usr and /data (all the stuff under /usr/data (~70G), /video and /media will go there after the migration)
- 3x400G (2 400G slices from 500G disks + 1 400G disk) (raidz) to be used for /backup with the other 400G drive as a spare
- 2x100G (2 100G slices from 500G disks) (gmirror, raid1) to be used for /, /var and /tmp
If I didn't have this SATA optical drive, I'd use that other 400G disk and have 2 parity disks in the backup raidz, but at least I'll have the spare.
So given that, I'm trying to plan what all I will need to do to move the data around/etc to re-organize the disks. I wanted to run my plan by you all and see if it looks good before I start.
Currently I'm using all 6 SATA ports on this motherboard (Asus P5Q-Pro) on the Intel controller - the 5 SATA disks, then a SATA bluray/dvd/cd writer. I'm trying to decide which disks to put on the other 2-port controller (Silicon Image Sil5723). Actually, I need to see if that controller is even supported. Assuming it is, I'm thinking I'll put the bluray drive and the 400 drive on there, although perhaps I shouldn't trust one of my disks in the /backup raidz to be on that 3rd party controller?
I'm booting from ada0's MBR (I think), so that means I will need to install the boot loader on the gmirror once that's setup then move one of those disks to the first SATA port (or change the BIOS drive order so that disk is first), right? With that in mind, here's my plan of attack:
1. make sure the other SATA controller works. If not I'll have to wait until I can purchase a PCI-E controller add-on, or forgo these plans for a while. Update fstab to temporarily remove the mounts for /video and /media (will mount them manually later). Question: when is the best time to remove any glabels I have since I'll want to re-create them on the new gmirror later?
2. Install the 2 new drives. Move the disks around such that the 1st 500 G drive is ada0, 2nd 500 G drive ada1. Then the next 4 (ada2 - ada5) for the 1 TB drives. On the Silicon Image controller, I'll put the 400G drive on port0 then the bluray drive on port1. I'll confirm in the BIOS that the old ada0 that I'm currently booting from is set as the "first" disk (until later).
3. Boot normally and copy the existing stuff in /backup to /video. Then destroy the zpool to free up the two 500 G drives and the 500 G slice on the one 1TB drive.
4. Slice up the 500G disks such that I have two 400 G partitions equal to the size of a full-disk slice on the 400G disk. Then calculate whatever is left and create those for the gmirror. I think this will mean a ~100G ada0s1 and ada1s1 and 400G ada0s2 and ada1s2, right? So I'll gmirror ada0s1 and ada1s1 then create my slices in gm0s1 for /, /tmp and /var) - Question: can I create gm0s1b for swap or do I have to waste a few G on both ada0 and ada1 to create swap?
5. Format and create /, /tmp and /var on the gmirror, then install the boot loader to the gmirror (or ada0). Copy over the existing /, /tmp and /var and (temporarily) /usr (except /usr/data) to / on the gmirror since we'll need a usable /usr when creating the "data" zpool.
6. Setup the 400 G disk such that I now have three 400 G partitions (ada0s2, ada1s2, ada6s1) of equal size. Then create the backup zpool from these 3 400G chunks and mount it at /backup.
7. Copy all of my data from /usr, /media and /video (temporarily) to the /backup ZFS mount.
8. Reboot and change the disk order in the BIOS so the 500G disk is booted from (the new ada0) and boot with the gmirror setup as the root fs (/ (including temporary /usr), /tmp, /var).
9. The data that was on the 1 TB disks is now all copied to /backup, clear the partition tables on those (dd ...) then create new disk-spanning slices to use for the 4-disk "data" zpool. Once the zpool is created, copy the /usr/data, /video and /media stuff from /backup back to /usr/{data,video,media} on the new zpool and copy the old backup data from the video sub-dir back to /backup, then remove the usr/data, video and media data from the new /backup.
10. Update fstab on gmirror's /etc and (re)create glabels for slash (/), var (/var), tmp (/tmp) and swap. Ensure zpools are setup and mounting properly and set mountpoints. Delete the /usr from the gmirror / then ensure /usr is the mountpoint for the "data" zpool.
Am I missing anything major here? I think my only uncertainty is dealing with the glabels, but worst case scenario I can mount/use things directly from adaX instead of the labels.
