How to grow up file systems and...

H

hannibal80

...how to make it properly?

I know that there is a lot of articles about growing partitions, slices and file systems and I read I think all of them but now I'm a bit confused and I don't want to harm my system that costed me a lot of time to configure.

As you can imagine I'm a newbie.

When I first installed FreeBSD on my 250G hd I decide to give FreeBSD only a small fraction of that space. Now I've realized that I was wrong.

This is the situation.

Code: 
# df -h
Filesystem     Size    Used   Avail Capacity  Mounted on
/dev/ad1s1a    963M    310M    576M    35%    /
devfs          1.0K    1.0K      0B   100%    /dev
/dev/ad1s1e    944M    1.0M    868M     0%    /tmp
/dev/ad1s1f     10G    4.5G    4.8G    48%    /usr
/dev/ad1s1d    3.7G    127M    3.3G     4%    /var

and

Code: 
# bsdlabel -A ad1s1
# /dev/ad1s1:
type: ESDI
disk: ad1s1
label:
flags:
bytes/sector: 512
sectors/track: 63
tracks/cylinder: 16
sectors/cylinder: 1008
cylinders: 484521
sectors/unit: 488397168
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0           # milliseconds
track-to-track seek: 0  # milliseconds
drivedata: 0

8 partitions:
#        size   offset    fstype   [fsize bsize bps/cpg]
  a:  2043904        0    4.2BSD        0     0     0
  b:  8073399  2043904      swap
  c: 41942817        0    unused        0     0         # "raw" part, don't edit
  d:  8019968 10117303    4.2BSD        0     0     0
  e:  2004992 18137271    4.2BSD        0     0     0
  f: 21800554 20142263    4.2BSD        0     0     0
bsdlabel: partition c doesn't cover the whole unit!
bsdlabel: An incorrect partition c may cause problems for standard system utilities

and

Code: 
# egrep 'ad[0-9]|cd[0-9]' /var/run/dmesg.boot
ad1: 238475MB <VB0250EAVER HPG0> at ata0-slave UDMA100 SATA
ad4: 1907729MB <ST2000DL003 9VT166 CC32> at ata2-master UDMA100 SATA 3Gb/s
Trying to mount root from ufs:/dev/ad1s1a

The 2T disk is waiting for a ZFS partition but that's another history.

What I would like to obtain is to grow up the space assigned to each file system according with needing. But first of all I don't know which one really need more space. For example I think that /dev/ad1s1f assigned to /usr need more space but I don't think so for /dev/ad1s1e (/tmp). What about /var?

Another thing I'm thinking about is a new separate partition (or slice? or, better, file system?) of about 50/70G that I'll use as a kind of temp for swapping big files in and out.

The final goal is to use about 100/120G for FreeBSD.

My questions are: which is the best way to do it? Which are the right file system (BSD4.2 or UFS or.. don't know) for installing ports and for swapping files?
And.. how to do that?

I read the growfs(8) man page but, as a newbie, I'm a bit scared. I don't want to lose my homework.

Can you please help me?

Thanks in advance,
Lo.
 
How I might do it: a... use fdisk's
Code: 
 -f file
(see posts about it) to create, say, a 100G space for USR (ufs2 or jgournal). Bsdlabel it, add it to fstab, rsync or tar /usr to its new space, delete everything on the old /usr, copy /tmp to where /usr was, edit fstab accordingly, be sure fstab matches the output of
Code: 
df
or pydf OR cdf , and reboot.
...
After making backups of critical data of course. Just writing from memory so I could be not explaining fully or leaving a step out or even getting something wrong... like leaving out newfs...
 
Okay, first step: Make A Backup. Seriously, do it. Do not mess with MBRs or GPTs or bsdlabels without it, it's just too easy to blow away critical information. Here's my article on that: Backup Options For FreeBSD
Put the backup files on another system or another disk that will not be connected to this system when you're changing disk layout. In other words, some place safe.

Back up anything else on that disk also. Anything you don't have backed up is at risk, and imagine how long it would take to recreate.

I have not used growfs(8). My understanding is that it could only grow the f partition, since the free space only comes after it.

The note about the c partition is a little concerning. Don't recall seeing that before.

For me, I'd back up as above, install a minimal FreeBSD to a new, blank drive, then restore the backups over that. After testing, I'd either use the new drive and keep the old drive as a backup, or maybe do the install/restore over the old drive.

For partitions, you don't really need 100G. Sizing of some partitions depends on how you use it. Some people keep mysql databases in /var, for example. As a general-purpose guess:

Code: 
/     2G
swap  4G
/var  4G
/tmp  4G to 8G
/usr  12G to 3000G, depending

As a side note, FreeBSD 9 only makes one big partition for / and one partition for swap. That's nice for space usage, but separate filesystems are more versatile.
 
Thanks guys.

So, the first step is to backup the system.

In my home network there's a NAS system (FreeNAS) that could hold the data.
I read your article wblock, what about using dump via ssh?

For the next step I think I'll use the jb_fvwm2 way but I need to understand all the step needed.
 
hannibal80 said:
Thanks guys.

So, the first step is to backup the system.

In my home network there's a NAS system (FreeNAS) that could hold the data.
I read your article wblock, what about using dump via ssh?
Click to expand...

That would be perfect. I would also suggest backing up the MBR and label.
% fdisk ada1 > /tmp/ad1.txt
% bsdlabel ada1s1 > /tmp/ad1s1.txt
Copy those files someplace safe in case you have to recreate them.

Edit: alternate form with gpart(8):
% gpart show ad1 ad1s1 > /tmp/ad1.txt
 
Ok, I did it!

Now, the next step.

I need some more space for /usr so I can install some other ports or packages but I really need more space (say at least 50G) for /tmp or, maybe, something like /tmp2...

I still have the 2T disk inside the system but, after the grow up, I'd like to have the 2T completely free for ZFS.

So, I don't know which is the best way for me. May be it's better to grow up the current /usr, leave the current /tmp and add a new /tmp2 (or, better, something like /swap) with the unused space in ad1.

Otherwise it could be an option to reinstall FreeBSD in the 2T disk with the right space, restore the dumps and then restore everything on the old disk but I'm a bit scared of managing the 2 concurrent OS... I don't know if I'm able to do that.

EDIT:
...and I don't want to make mistakes when I'll have to work with the 2 disks!
 
hannibal80 said:
Otherwise it could be an option to reinstall freeBSD in the 2T disk with the right space, restore the dumps and then restore everything on the old disk but I'm a bit scared of managing the 2 concurrent OS... I don't know if I'm able to do that.
Click to expand...

This is the best way to do it, much simpler than mucking about with growfs, etc.

Just dump your current ad1* to temporary partitions on ad4. Then you boot from ad4, rearrange ad1 as you want it, and dump back the data.

BTW, what in the world makes you think you need 50G for /tmp?
 
From my experience, best to lay out the entire process including diagrams and commands on paper, then if you change procedure midway or run into a problem you can figure out more exactly how to revise, backtrack, continue, or what problem to investigate solutions to; additionally consulting the Howto subforum prior to implementation for threads whose topics resemble the process somewhat, maybe small addons to the paper plan forestall huge revisions later on.
Forgot to mention, the majority of times I've done similar migrations, I ran into problems either another online computer and/or freebsd books were necc. to resolve. One may want to consider those also.
 
Thank you for your help.

Now, as jb said, I have to plan the next steps.

1. I have never touch ad4 disk so I think I have to do some work on it. What about sysinstall? Would be a good tool?
2. I have to figure out how to restore the ad1 dumps on the new ad4. I think I can use wblock guide.
3. I don't know how to switch the boot drive, I have to look around for it.
4. Then, another time I have to work with partitions, slices and file systems to prepare ad1. Which is the right tool (may be the simplest?) to do that?
5. Another time I'll switch the boot drive.

Ok, I'm going to googling a little bit.

For jalla, I need 50G on /tmp to do some testing managing big files like hd movies, videos and so on...
 
Hello all,

I'm working on the first point and I have the first problem. I'm using sysinstall to create one slice and the partitions to install dumps. I named the partitions with the same name the old (but still running) have (/, /var, /usr, /tmp).

I was following a guide - but I read lot of - and I freeze reading this: "don't use the name of a directory that already exists because sysinstall will mount the new partition on top of it". This is the guide: http://www.cyberciti.biz/faq/freebsd-adding-second-hard-disk-howto/

So, that's my problem: I thought I need the old directory names in the new drive to make all the path from the dumps working. But I cannot use the same name.

What did I miss?
 
The title of that article is "How To Add A Second Hard Disk". That would be for use at the same time as the first one. But you're setting up another disk to replace the first one. Let me point to a better (IMO) article: Disk Setup On FreeBSD

Some other things: using sysinstall to set up a second disk is dangerous. Given the chance, it will rewrite /etc/fstab on your main disk. That's bad. So is using sysinstall to create directories and mountpoints. After the system is installed, it's best to stop using sysinstall.

I would recommend the gpart(8) section of the tutorial above.
 
Thank you wblock, I'm following your guide.

This is the log of my operations:

Code: 
1.
# sysctl kern.geom.debugflags=16
kern.geom.debugflags: 0 > 16

2. 
# gpart create -s gpt /dev/ad4
ad4 created

3.
# gpart add -t freebsd-boot -l gpboot -s 64 ad4
ad4p1 added

4.
# gpart bootcode -b /boot/pmbr -p /boot/gptboot -i 1 ad4
bootcode written to ad4

5.
# gpart add -t freebsd-ufs -l gprootfs -b 2048 -s 2G ad4
ad4p2 added

6. 
# gpart add -t freebsd-swap -l gpswap -s 4G ad4
ad4p3 added
# gpart add -t freebsd-ufs  -l gpvarfs -s 8G ad4
ad4p4 added
# gpart add -t freebsd-ufs  -l gptmpfs -s 88G ad4
ad4p5 added
# gpart add -t freebsd-ufs  -l gpusrfs -s 32G ad4
ad4p6 added

7.
# gpart show -l ad4
=>      34  3907029101  ad4  GPT  		(1.8T)
        34          64	  1  gpboot  	(32k)
        98        1950       - free -  	(975k)
      2048     4194304    2  gprootfs  	(2.0G)
   4196352     8388608    3  gpswap  	(4.0G)
  12584960    16777216    4  gpvarfs  	(8.0G)
  29362176   184549376    5  gptmpfs  	(88G)
 213911552    67108864    6  gpusrfs  	(32G)
 281020416	3626008719       - free - 	(1.7T)

8.
# newfs /dev/gpt/gprootfs 
/dev/gpt/gprootfs: 2048.0MB (4194304 sectors) block size 16384, fragment size 2048
	using 12 cylinder groups of 183.72MB, 11758 blks, 23552 inodes.
super-block backups (for fsck -b #) at:
 160, 376416, 752672, 1128928, 1505184, 1881440, 2257696, 2633952, 3010208,
 3386464, 3762720, 4138976

# newfs -U /dev/gpt/gpvarfs 
/dev/gpt/gpvarfs: 8192.0MB (16777216 sectors) block size 16384, fragment size 2048
	using 45 cylinder groups of 183.72MB, 11758 blks, 23552 inodes.
	with soft updates
super-block backups (for fsck -b #) at:
 160, 376416, 752672, 1128928, 1505184, 1881440, 2257696, 2633952, 3010208,
 [...]
 12416608, 12792864, 13169120, 13545376, 13921632, 14297888, 14674144,
 15050400, 15426656, 15802912, 16179168, 16555424

# newfs -U /dev/gpt/gptmpfs 
/dev/gpt/gptmpfs: 90112.0MB (184549376 sectors) block size 16384, fragment size 2048
	using 491 cylinder groups of 183.72MB, 11758 blks, 23552 inodes.
	with soft updates
super-block backups (for fsck -b #) at:
 160, 376416, 752672, 1128928, 1505184, 1881440, 2257696, 2633952, 3010208,
 [...]
 180979296, 181355552, 181731808, 182108064, 182484320, 182860576, 183236832,
 183613088, 183989344, 184365600

# newfs -U /dev/gpt/gpusrfs 
/dev/gpt/gpusrfs: 32768.0MB (67108864 sectors) block size 16384, fragment size 2048
	using 179 cylinder groups of 183.72MB, 11758 blks, 23552 inodes.
	with soft updates
super-block backups (for fsck -b #) at:
 160, 376416, 752672, 1128928, 1505184, 1881440, 2257696, 2633952, 3010208,
 [...]
 65092448, 65468704, 65844960, 66221216, 66597472, 66973728

9. (disable soft update 'cause I think I don't need it now)
# tunefs -n disable /dev/gpt/gpvarfs 
tunefs: soft updates cleared
# tunefs -n disable /dev/gpt/gptmpfs 
tunefs: soft updates cleared
# tunefs -n disable /dev/gpt/gpusrfs 
tunefs: soft updates cleared

10. (log backup)
# ( tar cf - /usr/home/Lorenzo/newfs.log ) | ssh root@192.168.0.190 "cd /mnt/repository/Backup/freeBSD/20110916 && tar xvf -"
tar: Removing leading '/' from member names
root@192.168.0.190's password: 
x usr/home/Lorenzo/newfs.log

So, I'm at the "restore" step of your guide.

After the data restoring, as in your previous guide - via ssh, I'll have to modify the /etc/fstab.

So, what will happen next? My actual file systems (/, /var, /usr, /tmp) in ad0s1 will be replace by the new in ad4s1?

I'll have to choose something in the next boot for use ad0 or ad4? I think so because I followed all your steps so I made a boot partition.

Sorry for the silly questions but in this moment I'm really scared to miss my system.

Thank you.
 
I'm going on the idea that you're replacing the drive with a bigger one. Please say if that isn't the plan.

Turning off soft updates slows things down, but won't hurt anything and can be turned back on later.

What I like to do at this point is shut the system down and physically remove the old drive. Put it on a shelf where it's safe and can't be accidentally overwritten.

Connect the new drive, but boot from mfsBSD. You've already set up everything with gpart, so that doesn't need to be repeated.
Mount and restore the filesystems. Doesn't matter what order, they're just coming from backups. Based on the log,
# ssh -c blowfish [email=root@192.168.0.190]root@192.168.0.190[/email] gzcat root.dump.gz | (mount /dev/gpt/gprootfs /mnt&& cd /mnt && restore -rf - && umount /mnt)

(root ssh login... tsk.) This assumes that root.dump.gz is in root's home directory, so correct that. The output of gzcat(1) on the remote system is piped into the subshell on the local system. The subshell mounts the root filesystem by GPT label, then restores to it, then unmounts it.

Repeat for /var and /usr. There shouldn't have been anything in /tmp to back up, but do that too if there was.

Finally, mount /dev/gpt/rootfs again and edit /mnt/etc/fstab to refer to GPT labels (/dev/gpt/rootfs and so on).

Remember that all this time the "real" drive is sitting safely out of the system, and if anything goes wrong you can swap it back in and be back up and running.
 
Yes, you are right.

The current working disk is 250G, the new 2T.
I'm replacing the current with the new and, after that, I want to put back the 250G with the new bigger partitions and switch off the big one.

My box does not have a CD-ROM drive so I'm going to use a USB stick for mfsBDS.

Thank you!
 
You're really overcomplicating things here. With both disks attached you can just dump directly between them

As you have the partitioning of ad4 in place, proceed like this

Code: 
mkdir /2t
mount /dev/gpt/gprootfs /2t
cd /2t
dump 0Lf - / | restore -rf -
mount /dev/gpt/gpusrfs /2t/usr
cd /2t/usr
dump 0Lf - /usr | restore -rf -

etc...

Then edit /2t/etc/fstab and replace ad0 with the proper label where necessary. Next you're ready to reboot and select the second drive to boot from.
You can then repartition ad0 and finally dump everything back.
 
First of all, as usual, thanks a lot for you help.

Then, I followed jalla instructions and this is my log:

Code: 
# mkdir /2t
# mount /dev/gpt/gprootfs /2t
# cd /2t/
# dump 0Lf - / | restore -rf -
  DUMP: Date of this level 0 dump: Mon Sep 19 23:17:00 2011
  DUMP: Date of last level 0 dump: the epoch
  DUMP: Dumping snapshot of /dev/ad1s1a (/) to standard output
  DUMP: mapping (Pass I) [regular files]
  DUMP: mapping (Pass II) [directories]
  DUMP: estimated 314132 tape blocks.
  DUMP: dumping (Pass III) [directories]
  DUMP: dumping (Pass IV) [regular files]
warning: ./.snap: File exists
expected next file 189, got 188
  DUMP: DUMP: 314122 tape blocks
  DUMP: finished in 19 seconds, throughput 16532 KBytes/sec
  DUMP: DUMP IS DONE
# 
# mount /dev/gpt/gpusrfs /2t/usr
# cd /2t/usr/
# dump 0Lf - /usr | restore -rf -
  DUMP: Date of this level 0 dump: Mon Sep 19 23:20:34 2011
  DUMP: Date of last level 0 dump: the epoch
  DUMP: Dumping snapshot of /dev/ad1s1f (/usr) to standard output
  DUMP: mapping (Pass I) [regular files]
  DUMP: mapping (Pass II) [directories]
  DUMP: estimated 4778709 tape blocks.
  DUMP: dumping (Pass III) [directories]
  DUMP: dumping (Pass IV) [regular files]
warning: ./.snap: File exists
expected next file 872, got 868
  DUMP: 8.90% done, finished in 0:51 at Tue Sep 20 00:16:46 2011
  DUMP: 30.45% done, finished in 0:22 at Mon Sep 19 23:53:26 2011
  DUMP: 41.41% done, finished in 0:21 at Mon Sep 19 23:56:56 2011
  DUMP: 46.13% done, finished in 0:23 at Tue Sep 20 00:04:03 2011
  DUMP: 49.82% done, finished in 0:25 at Tue Sep 20 00:10:53 2011
  DUMP: 80.35% done, finished in 0:07 at Mon Sep 19 23:57:59 2011
  DUMP: DUMP: 4785372 tape blocks
  DUMP: finished in 1991 seconds, throughput 2403 KBytes/sec
  DUMP: DUMP IS DONE
# 
# mount /dev/gpt/gpvarfs /2t/var
# cd /2t/var/
# dump 0Lf - /var | restore -rf -
  DUMP: Date of this level 0 dump: Mon Sep 19 23:57:40 2011
  DUMP: Date of last level 0 dump: the epoch
  DUMP: Dumping snapshot of /dev/ad1s1d (/var) to standard output
  DUMP: mapping (Pass I) [regular files]
  DUMP: mapping (Pass II) [directories]
  DUMP: estimated 144840 tape blocks.
  DUMP: dumping (Pass III) [directories]
  DUMP: dumping (Pass IV) [regular files]
warning: ./.snap: File exists
expected next file 47105, got 4
  DUMP: DUMP: 144777 tape blocks
  DUMP: finished in 66 seconds, throughput 2193 KBytes/sec
  DUMP: DUMP IS DONE
# 
# ee /2t/etc/fstab
# Device                Mountpoint      FStype  Options         Dump    Pass#
/dev/ad4s1b             none            swap    sw              0       0
/dev/ad4s1a             /               ufs     rw              1       1
/dev/ad4s1e             /tmp            ufs     rw              2       2
/dev/ad4s1f             /usr            ufs     rw              2       2
/dev/ad4s1d             /var            ufs     rw              2       2

Then I rebooted but I couldn't find any message that make me choose between the ad1 and the ad4 disks. The system boots only with the ad1 disk.

What does it want from me?
 
On boot, press whatever it takes to get a boot menu, then choose the second disk. However, if both disks have the same labels, that could be bad. So again, I strongly suggest getting the original disk out of that machine while you're experimenting.

The difference between "won't" and "can't".

One way, "bad things won't happen" (a prediction).
The other way, bad things can't happen (a statement of fact).
 
The BIOS boot menu, shown by the BIOS on startup. Often available by pressing F12 on startup, some systems it's F10, F2, Esc, or something else. This would be before the FreeBSD boot options screen.
 
Ok, I'm missing something somewhere but I don't know what and I don't know where.

As you can see from the log, I followed the
Code: 
dump | restore
procedure.
After that I edited the /2t/etc/fstab replacing ad1 with ad4.
Then I rebooted but nothing happened.

So I detached the ad1 disk from my box and I rebooted again. After that I lost my USB keyboard and the system told me I had a problem with mounting option. The /etc/fstab was not consistent and it asked me to enter the right mount directives. Unfortunately the keyboard was not working.

Putting back the old small drive everything was all right. I re-mounted the partitions and I checked /2t/etc/fstab and it was as I left so with the right mount point:

Code: 
Device       Mountpoint  FSType   Options  Dumps  Pass
/dev/ad4s1b  none        swap     sw       0      0 
/dev/ad4s1a  /           ufs      rw       1      1
/dev/ad4s1e  /tmp        ufs      rw       2      2
/dev/ad4s1f  /usr        ufs      rw       2      2
/dev/ad4s1d  /var        ufs      rw       2      2

So, what I have to do? Do I need the mfsBSD system on a USB?

Thanks a lot for your help.
 
You don't mention trying the BIOS boot menu.

Depending on how the new drive was connected, it may not have been "ad4" any more. That was the point of using labels as we talked about before.

Check your BIOS for a "legacy USB emulation" setting. Enabled, that might let a USB keyboard work during booting.
 
wblock said:
You don't mention trying the BIOS boot menu.

Depending on how the new drive was connected, it may not have been "ad4" any more. That was the point of using labels as we talked about before.
Click to expand...
Yes, I tried switching drives from BIOS but was exactly the same as detaching the disks.

How can I check it?

Now I'm in this situation:
the /etc/fstab reports the same as before any action. The /2t/etc/fstab reports what I showed you.
Also# df -h reports the same as before so it seems that ad1 is still ad1.
 
With two drives attached, you might get ad0 and ad4. Move one to the other connector, and it will become ad0.

So use labels, where it doesn't matter. /2t/etc/fstab as shown is wrong, it refers to ad4, which will break when the drive number changes. From message #12, it should be like this:
Code: 
Device             Mountpoint  FSType   Options  Dumps  Pass
/dev/gpt/gpswap    none        swap     sw       0      0 
/dev/gpt/gprootfs  /           ufs      rw       1      1
/dev/gpt/gptmpfs   /tmp        ufs      rw       2      2
/dev/gpt/gpusrfs   /usr        ufs      rw       2      2
/dev/gpt/gpvarfs   /var        ufs      rw       2      2
 
I've never had ad0, always ad1. I think it's because I attached the first disk on a place made for CD-ROM.
Anyway I modified the /2t/etc/fstab replacing ad4 with ad1.
Then shut off the box, detached the old small drive, replaced with the new big one and booted up another time.
But there's something that is not working because it still say:
Code: 
ROOT MOUNT ERROR:
If you have invalid mount options, reboot, and first try the following from the loader prompt: 

  set vfs.root.mountfrom.options=rw

and then remove invalid mount options from /etc/fstab.

Loader viariables: 
vfs.root.mountfrom=ufs:/dev/ad1s1a
vfs.root.mountfrom.options=rw

Manual root filesystem specification: 
...

I'll try the same switching from ad1 to ad0 in /2t/etc/fstab.
 
You must log in or register to reply here.
Back
Top