HOWTO: keeping FreeBSD's base system and packages up-to-date [Archive]

View Full Version : HOWTO: keeping FreeBSD's base system and packages up-to-date

vermaden

September 5th, 2011, 12:36

Today, I would like to 'touch' an ungrateful topic of keeping both FreeBSD's base system and installed packages up-to-date.

After I started using FreeBSD at 5.4 times (2005) I have tried various methods of keeping my FreeBSD installations up-to-date, many of them terribly failed, but some recent ones seem to do the job as advertised. Even not so recently ago I thought, lets stick to RELEASE and do not compile newer versions of packages as there are available packages at FTP ... but there is a big problem with such attitude. First, once the RELEASE is completed, there are only security fixes for the base system, but there are no bug fixes for the RELEASE. Its even worse with packages for RELEASE since once they are built they are never later updated, even if they have security issues, not even mentioning bugs. So that is definitely not the right way.

The sollution seems to be tracking STABLE tree for the base system along with packages that are built every 2 weeks for the STABLE tree and compiling only when there are security issues in some of the installed packages, but there are for example 10 more days before their rebuilt versions would show up on the STABLE tree FTP. Below I would try to describe all that process of keeping FreeBSD up-to-date as simple as possible. In the first part I would focus on the base system and the second one will cover keeping packages up-to-date.

Some important information about keeping Your system this way. You would not rebuild the base system every day, not even every week, just when needed. Now what does it mean 'when needed' ... For example when there is a security issue, You would just follow the instructions in the SA (security advisory) to fix that issue, there is no need to rebuild whole world. The only reasons to rebuild the base system are that there has been found and fixed a bug in STABLE that affects You or that You need new features that has been merged into the STABLE branch (from CURRENT for example) like newer ZFS version or whatever.

As for the installation, You can install the RELEASE version and update to STABLE or install the daily STABLE snapshot so You would not have to build entire base system from source, the daily ISO images are available at http://pub.allbsd.org/FreeBSD-snapshots/ server.

EDIT:

Article also available as PDF here: http://bsdmag.org/magazine/1784-freebsd-get-up-to-date

If You want to have it separated from other articles, here's the way:
# pkg_add -r pdftk
% pdftk 'BSDmag 2012-01.pdf' cat 28-36 output 'BSDmag 2012-01 up-do-date.pdf'

... or as plain HTML at the bottom of this post.

vermaden

September 5th, 2011, 12:36

PART I. Keeping the FreeBSD base system up-to-date

Some facts about FreeBSD's base system:
-- once RELEASE is completed, there are only security fixes, there are no bug fixes
-- bugs in STABLE tree are fixed
-- security issues are also fixed in STABLE
-- the RELEASE branch allows to use binary updates via freebsd-update tool for security fixes
-- the STABLE branch requires compiling of the FreeBSD base system

We need to clone the current cource tree if we want to build up to date STABLE branch FreeBSD's base system, we will also need to update our sources to the current state so its quite handy to find fastest server for Your location, it can be easily done by using fastest_cvsup package.

# pkg_add -r fastest_cvsup
# fastest_cvsup -c all
(...)

>> Speed Daemons:
- 1st: cvsup.pl.freebsd.org
- 2nd: cvsup11.ua.freebsd.org
- 3rd: cvsup5.de.freebsd.org

For my location it is cvsup.pl.freebsd.org which in most cases will be different ther for Your location, so remember to put Your's fastest in the next steps.

Create simple supfile that will be used by csup to keep FreeBSD's base system sources up-to-date. There are useful examples under /usr/share/examples/cvsup/ if you want to 'dig more'.

# cat > /root/stable-supfile << EOF
*default host=cvsup.pl.freebsd.org
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=RELENG_8
*default delete use-rel-suffix
*default compress
src-all
EOF

Now lets get/update our sources to the current state, the list of edited/checked files will be quite different on Your box since I already have quite up-to-date sources, this will take more time if You do not have the sources on the disk.

# csup /root/stable-supfile
Connected to 188.125.237.138
Updating collection src-all/cvs
Edit src/bin/ed/buf.c
Edit src/sbin/fsck_ffs/main.c
Edit src/sbin/mdconfig/mdconfig.8
Edit src/sbin/mdconfig/mdconfig.c
Edit src/share/man/man4/ath.4
Edit src/share/man/man4/ath_hal.4
Edit src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_vnops.c
Edit src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_znode.c
Edit src/sys/dev/ahci/ahci.c
Edit src/sys/fs/msdosfs/msdosfs_vnops.c
Edit src/sys/fs/nfsclient/nfs_clbio.c
Edit src/sys/fs/nfsserver/nfs_nfsdserv.c
Edit src/sys/fs/nwfs/nwfs_io.c
Edit src/sys/fs/smbfs/smbfs_io.c
Edit src/sys/fs/tmpfs/tmpfs_vnops.c
Edit src/sys/gnu/fs/xfs/FreeBSD/xfs_vnops.c
Edit src/sys/kern/uipc_usrreq.c
Edit src/sys/kern/vfs_vnops.c
Edit src/sys/nfsclient/nfs_bio.c
Edit src/sys/sparc64/sbus/sbus.c
Edit src/sys/sys/vnode.h
Edit src/sys/ufs/ffs/ffs_inode.c
Edit src/sys/ufs/ffs/ffs_vnops.c
Finished successfully

Alternatively, You can grab the sources by SVN protocol, but You will need devel/subversion16 port/package for that purpose. Its generally a lot faster/easier to 'setup' then csup but the 'csup way' has one important advantage, its in the FreeBSD's base system, so its always available, anywhere. With SVN, You will have to add a package first which sometimes may be cumbersome. But as the FreeBSD source tree is kept under SVN it is possible that SVN will be part of the FreeBSD's base system one day.

Its also important to mention, that sources downloaded by subversion are not compatible with the sources grabbed by csup, so once You will decide which method to use, stick with it, unless You want to download the whole FreeBSD's source tree again. Below is the line needed to update the FreeBSD sources to 9-STABLE latest state.

# svn checkout svn://svn.freebsd.org/base/stable/9 /usr/src

Its the same no matter if You download the whole tree or just doing an update from yesterday. If svn will complain about anything, just delete the /usr/src and type the command again.

Now as we have the sources we can continue to building the FreeBSD's base system from source. As for editing the kernel config, You do not even have to bother about it, just use GENERIC, this guide is not about stripping the base system and kernel components, its about keeping everything up-to-date. Of course if You want to, then use Your tweaked kernel config, it will not interfere with the rest of this guide. You may want to put nice -n 20 in front of make buildworld ... line to make that build process less 'amusing' for your system. As instructions are completed, Your system will reboot.

# cd /usr/src
# rm -r -f /usr/obj
# make buildworld kernel KODIR=/boot/testing
# nextboot -k testing
# shutdown -r now

We are now proceeding to the second phase of the upgrade process, after normal boot (single user mode not required and definitely prohibited while doing upgrade over the network) stop all unneeded services (remember to keep sshd daemon alive if you are doing upgrade via network). If your system booted up properly, then You can make the new testing kernel the default one, at least there should not be any problems with the GENERIC kernel config ;)

# cd /boot
# rm -r -f OLD
# mv kernel OLD
# mv testing kernel

# killall dhclient moused
# /etc/rc.d/cron stop
# /etc/rc.d/devd stop
# /etc/rc.d/sshd stop
# /etc/rc.d/powerd stop
# /etc/rc.d/syslogd stop

The list of started processes will look something like that including (or not) sshd for keeping up the network connection.

# top -b
last pid: 64835; load averages: 0.00, 0.00, 0.00 up 0+03:11:51 10:24:37
119 processes: 2 running, 117 sleeping

Mem: 960M Active, 355M Inact, 4014M Wired, 6096K Cache, 8368K Buf, 2527M Free
Swap:

PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND
2178 root 1 54 0 10304K 2748K ppwait 0 0:00 0.00% csh
2174 root 1 45 0 21696K 1992K wait 0 0:00 0.00% login
2177 root 1 76 0 6912K 1284K ttyin 0 0:00 0.00% getty
2176 root 1 76 0 6912K 1284K ttyin 1 0:00 0.00% getty
2175 root 1 76 0 6912K 1284K ttyin 0 0:00 0.00% getty
114 root 1 76 0 2764K 1056K pause 0 0:00 0.00% adjkerntz

Now we can continue to type rest of needed instructions to finish the update, the mergemaster will ask You for the differences in startup scripts that You have modified and configuration files, type 'I' to install the new/default config and/or script and select 'D' to leave the version that you have in the system, remember that You can also add these changes later, it may be not appreciate to install default firewall config or customized OpenSSH config while doing the network upgrade.

# cd /usr/src
# mergemaster -p
# make installworld
# mergemaster -iU
# make delete-old
# shutdown -r now

After that second reboot You should have updates to STABLE branch FreeBSD's base system, I wrote 'should' because sometimes things do not go the way we want them to go, especially if you are doing it the first time as once Aerosmith sing "I know it's everybody's sin, You got to lose to know how to win". It would be best to do these instructions as exercise under virtual machine like VirtualBox or QEMU.

Also, if you do not feel that STABLE is 'production enought', then You may want to use STABLE packages along with RELEASE base system, You will need to define environment variable PACKAGESITE that will point to ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-8-stable/Latest/ at least for FreeBSD 8.x system.

vermaden

September 5th, 2011, 12:36

PART II. Keeping the FreeBSD packages up-to-date

Keeping packages up to date is little more tricky, we will also need the STABLE branch for them as these in RELEASE are not updated. Lets assume that You installed the FreeBSD STABLE snapshot a month ago, along with packages that were built by then, now there will be quite a lot of new versions for many packages which is not that important, but some of them can (and probably have) security issues and definitely should be updated. You can of course compile them from Ports using portmaster but why waste time for compiling, when You can use built every 2 weeks packages from the STABLE branch? The pkg_upgrade script from the bsdadminscripts package will be quite helpful here. It will fetch latest available packages from the STABLE FTP and there is a chance that the security issues will be solved by the newer versions, if not, we are forced to rebuild those packages from source using portmaster, but its a lot better and faster to recompile 1-2 packages instead of 30 or more.

As for updating the packages, I generally check them daily, mostly for security issues that would be reported with portaudit, there are often new versions reported, sometimes even quite lot, but as long as there are 0 problem(s) in your installed packages found. I do not bother. From time to time I fire up pkg_upgrade -a -C to fetch the latest packages from the STABLE branch FTP.

Some of You would certainly ask why use pkg_upgrade instead of updating with portmaster? Well, for example You have package z-1.0 installed in Your system, latest package available on the FTP is z-1.1 (newer) but version in Ports is z-1.2, so portmaster will omit that z-1.1 package no matter if its newer or not and will force You to compile the z-1.2 package from the Ports system.

Keeping FreeBSD packages up-to-date in short:
-- use packages from STABLE that are built every 2 weeks
-- use pkg_upgrade to update packages
-- use portmaster to rebuild packages that have security issues

Some facts about being up-to-date with FreeBSD's packages:
-- with every RELEASE packages are built and then they are never updated, even if they have security issues
-- for the STABLE tree packages are rebuilt every 2 weeks

First, we need to install tools that we will use to keep FreeBSD packages up-to-date.

# pkg_add -r bsdadminscripts portmaster portaudit

Optionally, we can allow users in group wheel to perform these task using sudo (You will have to add sudo package with pkg_add -r sudo commend) as they are already allowed to login on the root account, we can of course create separate group like maintainers that will be allowed to perform upgrades. You will need this line below in /usr/local/etc/sudoers file.

%wheel ALL=NOPASSWD: /usr/sbin/portsnap, /usr/local/sbin/pkg_upgrade, /usr/local/sbin/portmaster, /usr/local/sbin/portaudit

Here is the most important part, the commands put together into functions that will allow us easy checking for newer versions of the packages, security issues and updating them to newer/fixed versions. The ports-check function fetches latest Ports tree, then shows what new packages are available comparing to those installed on the system, next the security issues are checked with portaudit and last, the /usr/ports/UPDATING file is checked for various messages that can affect us. The ports-check does not rebuild or update any packages, only, as the name says, checks.

function ports-check {
# FETCH LATEST PORTS TREE
sudo portsnap fetch update

# CHECK WHAT NEW VERSIONS EXIST
sudo portmaster -L --index-only | awk '/ [Nn]ew / { print substr($0,9,9999) }'

# CHECK SECURITY ISSUES
sudo portaudit -Fda

# CHECK /usr/ports/UPDATING MESSAGES
pkg_updating -d $( ls -ltr -D '%Y%m%d' /var/db/pkg | awk 'END{print $6}' )
}

The second function, ports-update is for updating the packages using the STABLE branch, it uses pkg_upgrade from bsdadminscripts, but it will not compile from Ports.

function ports-update {
# UPDATE PACKAGES USING STABLE PATH ON FTP
sudo pkg_upgrade -a -C 2>&1 | grep --color=none --line-buffered -E "^(=+>|/usr/ports|/var/db)"

# FIX DEPENDENCIES AS NEEDED
sudo portmaster --check-depends
}

The last one named ports-build rebuilds the specified package or all of them if You do not specify one, used mostly to rebuild packages with security issues.

function ports-build {
# REBUILD SINGLE, SEVERAL OR ALL PORTS
case ${#} in
(0) sudo portmaster -y --no-confirm --packages-if-newer -m 'BATCH=yes' -d -a ;;
(*) sudo portmaster -y --no-confirm --packages-if-newer -m 'BATCH=yes' -d $@ ;;
esac

# FIX DEPENDENCIES AS NEEDED
sudo portmaster --check-depends
}

... and thats it generally, I would show some example of these functions usage below.

You will have to put these functions into Your shells startup files, it will be /etc/profile for sh shell and bash, /etc/zshrc for zsh. It will not work for C-shells like csh since they do not support functions and are retarded in many other ways: http://www.grymoire.com/Unix/CshTop10.txt

Drawbacks

Using this way of keeping the installed packages up-to-date You have to remember two things.

Customized packages. If You built some package with non-default options by compiling it, after upgrade it will 'revert' do the default options and You will have to build it again.

Kernel modules. Any package that comes with kernel modules can and probably will break at some point because the STABLE source tree is a 'moving target', that is one of the good reasons to update the base system and then update to latest packages. The packages that have kernel modules are for example emulators/virtualbox-ose (VirtualBox), sysutils/fusefs-kmod (FUSE implementation) and most notably x11/nvidia-driver (binary nVidia graphics driver).

vermaden

September 5th, 2011, 12:36

EXAMPLE II.1. Typical output about new/updated ports and new versions available.

% ports-check
Looking up portsnap.FreeBSD.org mirrors... 5 mirrors found.
Fetching snapshot tag from portsnap5.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Mon Sep 5 07:11:28 CEST 2011 to Mon Sep 5 08:51:01 CEST 2011.
Fetching 3 metadata patches.. done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
Fetching 10 patches.....10 done.
Applying patches... done.
Fetching 1 new ports or files... done.
Removing old files and directories... done.
Extracting new files:
/usr/ports/chinese/c2t/
/usr/ports/chinese/hc/
/usr/ports/devel/Makefile
/usr/ports/devel/p5-System-Command/
/usr/ports/german/mythes/
/usr/ports/math/p5-Statistics-R/
/usr/ports/polish/hunspell/
/usr/ports/textproc/es-mythes/
/usr/ports/textproc/nl-mythes/
/usr/ports/textproc/sk-mythes/
/usr/ports/textproc/sl-mythes/
Building new INDEX files... done.
New version available: arc-5.21p
New version available: ca_root_nss-3.12.11_1
New version available: expat-2.0.1_2
New version available: tinyxml-2.6.2
New version available: bash-4.1.11
New version available: gstreamer-plugins-0.10.35_1,3
New version available: gtk-2.24.6
New version available: gtk-update-icon-cache-2.24.6
New version available: libsamplerate-0.1.8_1
New version available: nas-1.9.3
New version available: nettle-2.4
New version available: p5-Date-Manip-6.25
New version available: p5-Mail-IMAPClient-3.29
New version available: p5-XML-Parser-2.41
New version available: xterm-273
New version available: filezilla-3.5.1
New version available: firefox-6.0.1,1
New version available: gtk-oxygen-engine-1.1.2
New version available: nginx-1.0.6,1
New version available: qemu-0.11.1_10
20 have new versions available
New database installed.
Database created: Thu Sep 1 21:20:00 CEST 2011
0 problem(s) in your installed packages found.

EXAMPLE II.2. A report that also shows some security issues.

% ports-check
Looking up portsnap.FreeBSD.org mirrors... 5 mirrors found.
Fetching snapshot tag from portsnap1.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Mon Sep 5 10:28:51 CEST 2011 to Mon Sep 5 12:07:23 CEST 2011.
Fetching 3 metadata patches.. done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
Fetching 20 patches.....10....20 done.
Applying patches... done.
Fetching 1 new ports or files... done.
Removing old files and directories... done.
Extracting new files:
/usr/ports/MOVED
/usr/ports/Mk/bsd.sites.mk
/usr/ports/cad/Makefile
/usr/ports/devel/Makefile
/usr/ports/devel/p5-Bread-Board-Declare/
/usr/ports/devel/p5-Curses-UI/
/usr/ports/devel/p5-Data-Peek/
/usr/ports/devel/p5-Scope-Upper/
/usr/ports/dns/pear-Net_DNS2/
/usr/ports/lang/p5-Try-Tiny/
/usr/ports/mail/p5-Email-Valid/
/usr/ports/math/p5-Math-BigInt/
/usr/ports/math/p5-Statistics-Descriptive/
/usr/ports/net/pear-Net_SMTP/
/usr/ports/ports-mgmt/portaudit-db/
/usr/ports/sysutils/py-supervisor/
/usr/ports/sysutils/zfsnap/
/usr/ports/www/nginx-devel/
/usr/ports/www/nginx/
/usr/ports/www/rubygem-passenger/
/usr/ports/x11/Makefile
Building new INDEX files... done.
New version available: arc-5.21p
New version available: ca_root_nss-3.12.11_1
New version available: expat-2.0.1_2
New version available: tinyxml-2.6.2
New version available: bash-4.1.11
New version available: gstreamer-plugins-0.10.35_1,3
New version available: gtk-2.24.6
New version available: gtk-update-icon-cache-2.24.6
New version available: libsamplerate-0.1.8_1
New version available: nas-1.9.3
New version available: nettle-2.4
New version available: p5-Date-Manip-6.25
New version available: p5-Mail-IMAPClient-3.29
New version available: p5-XML-Parser-2.41
New version available: xterm-273
New version available: filezilla-3.5.1
New version available: firefox-6.0.1,1
New version available: gtk-oxygen-engine-1.1.2
New version available: nginx-1.0.6,1
New version available: qemu-0.11.1_10
20 have new versions available
auditfile.tbz 100% of 69 kB 54 kBps
New database installed.
Database created: Mon Sep 5 12:35:01 CEST 2011
Affected package: ca_root_nss-3.12.9
Type of problem: ca_root_nss -- Extraction of unsafe certificates into trust bundle..
Reference: http://portaudit.FreeBSD.org/1b27af46-d6f6-11e0-89a6-080027ef73ec.html

Affected package: ca_root_nss-3.12.9
Type of problem: nss/ca_root_nss -- Fraudulent Certificates issued by DigiNotar.nl.
Reference: http://portaudit.FreeBSD.org/aa5bc971-d635-11e0-b3cf-080027ef73ec.html

2 problem(s) in your installed packages found.

You are advised to update or deinstall the affected package(s) immediately.

EXAMPLE II.3. Solving the security issues by rebuilding the problematic package.

% ports-build ca_root_nss

===>>> Currently installed version: ca_root_nss-3.12.9
===>>> Port directory: /usr/ports/security/ca_root_nss

(...)

===>>> Updating dependency entry for ca_root_nss-3.12.11_1 in each dependent port
===>>> Upgrade of ca_root_nss-3.12.9 to ca_root_nss-3.12.11_1 complete

% ports-check
Looking up portsnap.FreeBSD.org mirrors... 5 mirrors found.
Fetching snapshot tag from portsnap2.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Mon Sep 5 12:07:23 CEST 2011 to Mon Sep 5 12:25:09 CEST 2011.
Fetching 3 metadata patches.. done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
Fetching 2 patches.. done.
Applying patches... done.
Fetching 0 new ports or files... done.
Removing old files and directories... done.
Extracting new files:
/usr/ports/devel/p5-File-NFSLock/
/usr/ports/devel/p5-MooseX-Aliases/
Building new INDEX files... done.
New version available: arc-5.21p
New version available: expat-2.0.1_2
New version available: tinyxml-2.6.2
New version available: bash-4.1.11
New version available: gstreamer-plugins-0.10.35_1,3
New version available: gtk-2.24.6
New version available: gtk-update-icon-cache-2.24.6
New version available: libsamplerate-0.1.8_1
New version available: nas-1.9.3
New version available: nettle-2.4
New version available: p5-Date-Manip-6.25
New version available: p5-Mail-IMAPClient-3.29
New version available: p5-XML-Parser-2.41
New version available: xterm-273
New version available: filezilla-3.5.1
New version available: firefox-6.0.1,1
New version available: gtk-oxygen-engine-1.1.2
New version available: nginx-1.0.6,1
New version available: qemu-0.11.1_10
19 have new versions available
auditfile.tbz 100% of 69 kB 54 kBps
New database installed.
Database created: Mon Sep 5 12:40:01 CEST 2011
0 problem(s) in your installed packages found.

EXAMPLE II.4. Updating the installed packages using STABLE branch.

% ports-update
/var/db/uma/FTPINDEX 100% of 21 MB 1139 kBps 00m00s
/usr/ports/packages/All/nettle-2.4.tbz 100% of 1082 kB 332 kBps
/usr/ports/packages/All/gtk-2.24.6.tbz 100% of 9822 kB 253 kBps 00m00s
/usr/ports/packages/All/gstreamer-plugins-0.10100% of 4091 kB 942 kBps
/usr/ports/packages/All/gtk-oxygen-engine-1.1.100% of 509 kB 339 kBps
/usr/ports/packages/All/filezilla-3.5.1.tbz 100% of 3301 kB 232 kBps 00m00s
/usr/ports/packages/All/nas-1.9.3.tbz 100% of 487 kB 494 kBps
/usr/ports/packages/All/expat-2.0.1_2.tbz 100% of 130 kB 129 kBps
/usr/ports/packages/All/xterm-273.tbz 100% of 262 kB 104 kBps
/usr/ports/packages/All/p5-XML-Parser-2.41.tbz100% of 184 kB 381 kBps
/usr/ports/packages/All/nginx-1.0.6,1.tbz 100% of 225 kB 206 kBps
/usr/ports/packages/All/qemu-0.11.1_10.tbz 100% of 12 MB 359 kBps 00m00s
/usr/ports/packages/All/tinyxml-2.6.2.tbz 100% of 170 kB 121 kBps
/usr/ports/packages/All/p5-Date-Manip-6.25.tbz100% of 1301 kB 681 kBps
===> Update <tinyxml-2.6.1_1> to <tinyxml-2.6.2> (textproc/tinyxml)
=> Update <tinyxml-2.6.1_1> to <tinyxml-2.6.2> (textproc/tinyxml) succeeded
===> Update <p5-Date-Manip-6.24> to <p5-Date-Manip-6.25> (devel/p5-Date-Manip)
=> Update <p5-Date-Manip-6.24> to <p5-Date-Manip-6.25> (devel/p5-Date-Manip) succeeded
===> Update <nginx-1.0.5,1> to <nginx-1.0.6,1> (www/nginx)
=> Update <nginx-1.0.5,1> to <nginx-1.0.6,1> (www/nginx) succeeded
===> Update <nettle-2.2> to <nettle-2.4> (security/nettle)
=> Update <nas-1.9.2> to <nas-1.9.3> (audio/nas) succeeded
===> Update <expat-2.0.1_1> to <expat-2.0.1_2> (textproc/expat2)
=> Update <expat-2.0.1_1> to <expat-2.0.1_2> (textproc/expat2) succeeded
===> Update <xterm-271> to <xterm-273> (x11/xterm)
=> Update <xterm-271> to <xterm-273> (x11/xterm) succeeded
===> Update <qemu-0.11.1_9> to <qemu-0.11.1_10> (emulators/qemu)
=> Update <qemu-0.11.1_9> to <qemu-0.11.1_10> (emulators/qemu) succeeded
===> Update <gtk-2.24.5_1> to <gtk-2.24.6> (x11-toolkits/gtk20)
=> Update <gtk-2.24.5_1> to <gtk-2.24.6> (x11-toolkits/gtk20) succeeded
===> Update <firefox-6.0_1,1> to <firefox-6.0.1,1> (www/firefox)
=> Update <firefox-6.0_1,1> to <firefox-6.0.1,1> (www/firefox) succeeded
===> Update <filezilla-3.5.0_1> to <filezilla-3.5.1> (ftp/filezilla)
=> Update <filezilla-3.5.0_1> to <filezilla-3.5.1> (ftp/filezilla) succeeded
===> Update <bash-4.1.10> to <bash-4.1.11> (shells/bash)
=> Update <bash-4.1.10> to <bash-4.1.11> (shells/bash) succeeded
===> Update <arc-5.21o_1> to <arc-5.21p> (archivers/arc)
=> Update <arc-5.21o_1> to <arc-5.21p> (archivers/arc) succeeded
===>>> Checking 2bsd-vi-050325_1
===>>> Checking ImageMagick-6.7.1.10
===>>> Checking ORBit2-2.14.19
===>>> Checking OpenEXR-1.6.1_3
===>>> Checking Thunar-1.2.2_2
(...)

vermaden

September 5th, 2011, 12:37

Feel free to sent Your opinions/questions and Your ways of keeping both FreeBSD's base system and installed packages.

Kiiski

September 5th, 2011, 16:55

Hi

I use ports myself, but about this:

Some of You would certainly ask why use pkg_upgrade instead of updating with portmaster? Well, for example You have package z-1.0 installed in Your system, latest package available on the FTP is z-1.1 (newer) but version in Ports is z-1.2, so portmaster will omit that z-1.1 package no matter if its newer or not and will force You to compile the z-1.2 package from the Ports system.

From
man portmaster

--packages-if-newer
use package if newer than installed even if the package is not the
latest according to the ports tree

Shouldn't that take care of situation you described?

vermaden

September 6th, 2011, 05:50

@Kiiski

Maybe it has changed recently, portmaster always used the Ports tree version on my box, I also tried the -PP option and if the first package to update does not exist on the FTP, portmaster will end its work with an error instead of trying to get update for the next package.

vermaden

September 8th, 2011, 10:03

Shouldn't that take care of situation you described?

I have just checked how portmaster -a -PP again behaves and that is the result:

===>>> Checking package repository for latest available version

===>>> The newest available package (sudo-1.8.1_5)
is older than the version in ports (sudo-1.8.2)

===>>> Try --packages-if-newer, or do not use -PP/--packages-only
===>>> Aborting update

===>>> Update for security/sudo failed
===>>> Aborting update

The --packages-if-newer option is useless here since it will try to REBUILD/COMPILE all packages and we want to omit that.

alie

September 19th, 2011, 10:42

Awesome good job! this thread enters Distrowatch weekly issue

http://distrowatch.com/weekly.php?issue=20110919#news

Yampress

September 19th, 2011, 13:32

very good work Ver. ;D

nORKy

September 19th, 2011, 15:30

Hi, I have 1 question : What is the diffrence between portmaster and pkg_* ?

EDIT : and an other : where do you track STABLE bugfix/securityfix ??

wblock@

September 19th, 2011, 16:28

pors-mgmt/portmaster is an add-on tool for managing ports and packages. The various pkg_* programs are base system programs that only deal with packages.

Symbiosis

September 19th, 2011, 16:39

Hi,

First of all thanks for your useful post.

I want to comment something about this:

Also, if you do not feel that STABLE is 'production enought', then You may want to use STABLE packages along with RELEASE base system, You will need to define environment variable PACKAGESITE that will point to ftp://ftp.freebsd.org/pub/FreeBSD/po...stable/Latest/ at least for FreeBSD 8.x system.

I think that if you are on -RELEASE branch and you change to -STABLE packages you have to modify the environment variable 'BRANCH' for uma. If I don't add BRANCH=8-stable to my /usr/local/etc/uma.conf then the script ports-update doesn't work.

Best regards

vermaden

September 19th, 2011, 20:56

@alie;@Yampress;@Symbiosis

Thanks mates ;)

Hi, I have 1 question : What is the diffrence between portmaster and pkg_* ?

Beyond the difference that wblock already mentioned, pkg_* tools allows You to install/manage/delete binary packages from FTP and/or other source and portmaster is a tool tat automates (re)building from source using the FreeBSD's Ports tree.

EDIT : and an other : where do you track STABLE bugfix/securityfix ??

I check SA at the usual place, its the same for STABLE and RELEASE: http://www.freebsd.org/security/notices.html

... about BUGs, well, I browse from time to time this one: http://svnweb.freebsd.org/base/

Also reading FreeBSD's Mailing Lists sometimes helps where 'that and that has been merged from current' or 'fixed in stable'.

I think that if you are on -RELEASE branch and you change to -STABLE packages you have to modify the environment variable 'BRANCH' for uma. If I don't add BRANCH=8-stable to my /usr/local/etc/uma.conf then the script ports-update doesn't work.

Interesting, I havent heard before about UMA, must read more about this, but it seams that its not applicable here since UMA is for using RELASE packages on STABLE and CURRENT systems while this guide is for the opposite, using STABLE packages on RELEASE system ;)

mgp

September 20th, 2011, 06:23

You may want to try portrac (http://www.freshports.org/ports-mgmt/portrac/) to keep track of the port updates. It's a simple GUI tool that checks periodically if there are updates to the ports that are installed. I'd also love to get some feedback about the tool.

vermaden

September 20th, 2011, 07:46

@mgp

I have just tried it, not impressed, have that functionality already and even with port versions ;)

http://ompldr.org/vYWZsbA

nORKy

September 20th, 2011, 07:51

This how-to is very interesing but, I don't think I can track STABLE on my productions servers. I don't want to stop my sever too many times when there is a bug in the base system.
I think, the best for me, it's to have a RELEASE base system and packages compiled and up to date.

I'm building a local server ports for my servers and a configuration mangament server (with puppet). I think it's better for me. And, in the futur, I think about a PXE boot, and automatic installations.

vermaden

September 20th, 2011, 09:11

This how-to is very interesting but, I don't think I can track STABLE on my productions servers.
Currently I am using this 'way' on my laptop and home NAS, but I agree that for servers that have compiled ports it would require to rebuild the needed ports every time there was packages available, which may also be a little pain, on servers its also often 3-5 ports with little dependencies that need to be rebuild, so it also does not take ages like with the whole desktop/x11/multimedia and so.

I don't want to stop my sever too many times when there is a bug in the base system.
Who forces You to do so? ;)

As I written in the HOWTO, its done very rarely, for most of the time its not needed to rebuild the base system, but on STABLE You have that possibility, on RELEASE, You are 'dead' ;)

I'm building a local server ports for my servers and a configuration mangament server (with puppet). I think it's better for me. And, in the futur, I think about a PXE boot, and automatic installations.

That is the best attitude, even to have the own freebsd-update server, so You could fix these potential bugs from STABLE using the freebsd-update utility.

I check SA at the usual place, its the same for STABLE and RELEASE: http://www.freebsd.org/security/notices.html

... about BUGs, well, I browse from time to time this one: http://svnweb.freebsd.org/base/

I also forgot about that one: http://freshbsd.org/search?branch=RELENG_8&project=freebsd

graudeejs

September 20th, 2011, 09:58

You may wanna try portrac (http://www.freshports.org/ports-mgmt/portrac/) to keep track of the port updates. It's a simple GUI tool that checks periodically if there are updates to the ports that are installed. I'd also love to get some feedback about the tool.

Or you may try ports-mgmt/bxpkg for package updates (development is stalled, until FreeBSD-9)

alie

September 20th, 2011, 12:10

Anything can do these ?

1. Periodically refresh package databases and prompt for updates.
2. Install packages from tarballs.
3. Search for packages by name, description, category or file.
4. Show package dependencies, files and reverse dependencies.
5. System tray notifiers

vermaden

September 20th, 2011, 12:23

1. Periodically refresh package databases and prompt for updates.
ports-check + ports-update ;)

2. Install packages from tarballs.
The Ports tree automates that.

3. Search for packages by name, description, category or file.
# cd /usr/ports
# make search name=NAME

... or use http://freshports.org

4. Show package dependencies, files and reverse dependencies.
# pkg_info -L package
# pkg_info -r package
# pkg_info -R package

also BXPKG is able to do that graphically.

5. System tray notifiers
Doesn't PC-BSD have that? (haven't tried PC-BSD for quite long time)

alie

September 20th, 2011, 13:33

@Vermaden: Thanks for the reply. What I mean is a GUI Frontends coded with KDE/Qt/Gtk/X11 :e

Yes PC-BSD does that but not sure if the GUI available for FreeBSD?

nORKy

September 20th, 2011, 19:38

That is the best attitude, even to have the own freebsd-update server, so You could fix these potential bugs from STABLE using the freebsd-update utility.

hum.. freebsd-update can work on STABLE?

vermaden

September 20th, 2011, 22:06

No, I mean HAVE YOUR OWN update server, check this out ;)
http://freebsd.org/doc/en/articles/freebsd-update-server/index.html

That way You will be building ONCE and then using freebsd-update directly to your update server to binary updates for STABLE.

alie

September 21st, 2011, 02:37

We should donate to build freebsd-update server for STABLE :D

alie

September 21st, 2011, 15:00

PART I. Keeping the FreeBSD base system up-to-date

Do we need to run all these points for every updates?

vermaden

September 21st, 2011, 19:12

Do we need to run all these points for every updates ?

After the first time only steps from the # csup /root/stable-supfile are needed (including the csup step of course).

alie

October 6th, 2011, 05:52

What happens if I change the tag to RELENG_9? Will it compiles with llvm/clang instead of gcc?

vermaden

October 6th, 2011, 07:38

@alie

You probably mean building the world/kernel, dunno mate, I haven't tried to buildworld on 9.x yet.

alie

October 6th, 2011, 07:40

Yes, building the world. Then how to make sure it builds with llvm/clang?

vermaden

October 6th, 2011, 07:41

Yes, building the world. Then how to make sure it builds with llvm/clang ?

If the commands to build/install world/kernel are the same, then what is the difference? :)

alie

October 6th, 2011, 08:10

hmmm...

Another question. How to turn off debug mode for FreeBSD9-STABLE? Is it on by default?

vermaden

October 6th, 2011, 10:20

hmmm...

Another question. How to turn off debug mode for FreeBSD9-STABLE ? Is it on by default ?

When You download ISO images that are BETA, then Yes, debug is enabled in the kernel (WITNESS for example), when 9.0 will reach RC state, the additional debug is removed from the kernel default config (GENERIC), so its all about editing the kernel config.

alie

October 6th, 2011, 11:00

I am new with kernel config. Is there any tutorial how to edit the kernel config? What is the config file location and name anyway?

vermaden

October 6th, 2011, 11:24

I am new with kernel config. Is there any tutorial how to edit the kernel config ?

Sure, whole chapter at FreeBSD Handbook: http://freebsd.org/handbook/kernelconfig.html

What is the config file location and name anyway ?
Location is /usr/src/sys/${ARCH}/conf/GENERIC, for example /usr/src/sys/amd64/conf/GENERIC.

Every time You update Your local source tree with csup/svn it gets overwritten, so its better to:

# cd /usr/src/sys/${ARCH}/conf
# cp GENERIC /root/MINE
# ln -s MINE

And then use KERNCONF=MINE at the make buildkernel/installkernel/kernel option.

Carpetsmoker

October 6th, 2011, 14:21

What happens if I change the tag to RELENG_9? Will it compiles with llvm/clang instead of gcc?

llvm/clang will be installed, but GCC is still the default compiler. Progress is being made, but it will probably remain the default for quite some time. There are 23,000+ ports that are typically *only* tested with GCC by the developers.

alie

October 7th, 2011, 02:58

Means 9.0-RELEASE will be compiled with gcc instead of llvm/clang ? :(

Carpetsmoker

October 7th, 2011, 03:28

I believe that is what I said...

You can use clang if you want, but it's not the default.

vermaden

October 7th, 2011, 06:16

I believe that is what I said...

You can use clang if you want, but it's not the default.

Isn't that the base system will be compiled by llvm/clang and gcc is kept for building ports?

alie

October 7th, 2011, 06:42

This is something really not clear... I couldnt find any information about this anywhere on the FreeBSD wiki, forum or handbook... we should put this information somewhere that visible and easy to find. And there's no document about how to compile with llvm instead of gcc or otherwise...

Carpetsmoker

October 7th, 2011, 12:17

Isn't that the base system will be compiled by llvm/clang and gcc is kept for building ports?

IIRC I picked up something like that too once.

I don't know what future plans are. All I know is that FreeBSD 9 BETA3 ships with Both llvm/Clang and GCC4.2 and uses FreeBSD 4.2 as the default.

If you want to use Clang, simply set CC to point to Clang instead of GCC ... I didn't try it, but I don't think it's more complicated then that :)

kpa

October 7th, 2011, 12:44

Sure, whole chapter at FreeBSD Handbook: http://freebsd.org/handbook/kernelconfig.html

Location is /usr/src/sys/${ARCH}/conf/GENERIC, for example /usr/src/sys/amd64/conf/GENERIC.

Every time You update Your local source tree with csup/svn it gets overwritten, so its better to:

# cd /usr/src/sys/${ARCH}/conf
# cp GENERIC /root/MINE
# ln -s MINE

And then use KERNCONF=MINE at the make buildkernel/installkernel/kernel option.

Instead of making a full copy of GENERIC you can write your kernel config like this:

ident MINE

include GENERIC

nooption KDB
nooption DDB
nooption GDB
nooption DEADLKRES
nooption INVARIANTS
nooption INVARIANT_SUPPORT
nooption WITNESS
nooption WITNESS_SKIPSPIN
nooption MALLOC_DEBUG_MAXZONES

Big timesaver when GENERIC eventually changes. You don't have to manually track the changes between the new GENERIC and your own configuration.

Caliante

October 8th, 2011, 15:28

Thank you for this post, Vermaden :D

vermaden

October 9th, 2011, 07:28

@Caliante

Welcome mate ;)

alie

October 10th, 2011, 05:53

Hi Guys, I have 1 query regarding kernel config.

What do i need to set for cpu option on kernel config if i have this from dmesg for 64 bit profile (amd64):

CPU: Intel(R) Core(TM) i5 CPU 750 @ 2.67GHz (2673.35-MHz K8-class CPU)
Origin = "GenuineIntel" Id = 0x106e5 Family = 6 Model = 1e Stepping = 5
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,P GE,MCA,CMOV,PAT,
Features2=0x98e3fd<SSE3,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16, xTPR,PDCM,SSE4.1,
AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
AMD Features2=0x1<LAHF>
TSC: P-state invariant, performance statistics

Is it cpu I686_CPU ?

grigorovl

October 10th, 2011, 06:26

I686 for 32bit, HAMMER for 64bit.

More here:
http://forums.freebsd.org/showthread.php?t=2403

alie

October 10th, 2011, 06:40

From the handbook:
for a custom kernel it is best to specify only the CPU you have

So is it the best already to use HAMMER for my CPU ?

wblock@

October 10th, 2011, 15:30

I experimented with this earlier this summer: http://lists.freebsd.org/pipermail/freebsd-questions/2011-June/230491.html
For me, turning off i486 but leaving i586 and i686 resulted in a slight (6%) speed increase in buildworld. That was on a C2D E8400, FreeBSD 8-STABLE, i386.

Yampress

October 10th, 2011, 17:58

I up-to-date base system with a newer version of the cd freebsd stable. In sysinstall screen choose upgrade....

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/install/main1.png

YZMSQ

October 11th, 2011, 05:58

I up-to-date base system with a newer version of the cd freebsd stable. In sysinstall screen choose upgrade....

http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/install/main1.png
Does bsdinstall, the latest installer in FreeBSD9, offer this "upgrade" option?

alie

October 11th, 2011, 10:27

Does bsdinstall, the latest installer in FreeBSD9, offer this "upgrade" option?

Nope, AFAIK

Yampress

October 11th, 2011, 12:26

but now it is sysinstall still in stable 8 ...

vermaden

October 11th, 2011, 12:44

but now it is sysinstall still in stable 8 ...

But NOT in STABLE/9 ...

tankist02

October 19th, 2011, 01:36

On 9 RC1 amd64 I got:

pkg_upgrade -a -C
fetch: ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-stable/INDEX: File unavailable (e.g., file not found, no access)

The correct URL is:

ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-current/

How can I fix the URL in the script?

alie

October 19th, 2011, 05:10

setenv PACKAGESITE ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9-current//Latest/

tankist02

October 19th, 2011, 21:40

Thanks a lot, works great!

bbzz

October 22nd, 2011, 00:11

Vermaden, great guide. I used your zfs one in the past as well. It's always nice to have condensed information you can quickly check.

I upgraded to FreeBSD 9-RC1. I had to add a line in /usr/local/etc/uma.conf

BRANCH=9-current

otherwise ports-update fails; it checks 9-stable for packages, which doesn't exist.

Now couple of questions. When portmaster is run in ports-build, it checks for packages on 9.0-release depository, for example:

===>>> Checking package repository for latest available version
===>>> Package and/or archive not found at:
http://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-9.0-release/audio/

This happens even after I export PACKAGESITE to "ftp://ftp.freebsd.org/pub/FreeBSD/ports/i386/packages-9-current/Latest/".

More importantly, if I'm already using portmaster to build from ports, does it really have to check for packages (I thought pkg_update will do that)?

Thanks again for your help.

jrm

October 22nd, 2011, 18:24

The only reasons to rebuild the base system are that there has been found and fixed a bug in STABLE that affects You or that You need new features that has been merged into the STABLE branch (from CURRENT for example) like newer ZFS version or whatever.

This is an important point. From the handbook:
...this means that at any given time, the sources for FreeBSD-STABLE may or may not be suitable for any particular purpose. It is simply another engineering development track, not a resource for end-users.

oliverh

October 22nd, 2011, 22:10

The handbook is not the bible, it's just a bunch of advices. There is even a lot of 'nonsense' in it at least nowadays. I'm using stable since 5.0 without any problems, on the other hand you could have severe problems while using releases and your hardware is somewhat picky. It solely depends on your environment and your experience. Don't make the mistake and compare terms like beta, RC, stable etc. pp. with those used in the Linux world. 15+ RCs for the Linux kernel are hardly comparable to the modest use of those in BSD world. The same is true for terms like stable, try to compare it to Debians testing or sid and you'll fail.

Vermadens advices are experience-driven, so are mine. 'It just works' is a completely different world, there is a RC dangerous and betas are almost deadly ;-)

Stick to Vermadens advices and remember: FreeBSD is about quality, not quantity. This is even true for the things in development.

jrm

October 23rd, 2011, 03:41

Stick to Vermadens advices... Sure, like when he says, The only reasons to rebuild the base system are that there has been found and fixed a bug in STABLE that affects You or that You need new features that has been merged into the STABLE branch. There have been times when STABLE is, well, stable and there have been times when I really wished I had spent more time reading the stable/current lists before upgrading. To each his own. I just posted to share my experience in case it helps others. In '97 I needed to share a dsl connection with my roommates, so I tried IP masquerading with linux. A friend suggested NAT on FreeBSD might work better. We had a 286 running 2.something (I think) and it ran without problems for the entire year. I don't think I've installed a linux distribution since. :) So, my only advice is to do your research and don't blindly upgrade if there isn't a good reason.

vermaden

October 29th, 2011, 03:29

Sorry for late response, I have been pretty offline lately ...

Vermaden, great guide. I used your zfs one in the past as well. It's always nice to have condensed information you can quickly check.
Welcome mate.

I upgraded to FreeBSD 9-RC1. I had to add a line in /usr/local/etc/uma.conf

BRANCH=9-current

otherwise ports-update fails; it checks 9-stable for packages, which doesn't exist.

I havent tried that 'guide' on 9.x yet as I use 8.2-* series, havent got time to play more with 9.x series.

More importantly, if I'm already using portmaster to build from ports, does it really have to check for packages (I thought pkg_update will do that)?
Good point, if pkg_upgrade hasn't found anything, then why would portmaster? ;)

Remove the --packages-if-newer from the ports-build function, I will modify the first post some time when I have more time (about a week from now).

vermaden

December 13th, 2011, 06:40

Added updating FreeBSD source by SVN.

purgatori

December 28th, 2011, 21:25

Keeping FreeBSD up to date is a nightmare; such a nightmare, that I have abandoned the idea of attempting it anymore. The last time I attempted a binary upgrade from 8.1 to 8.2, it borked my system so bad that I couldn't even perform a clean install without using DBAN to completely erase the hard drive.

I appreciate the effort you've gone to here, but the multitude of steps involved don't really fill me with confidence. I was actually going to abandon FreeBSD altogether after what happened, but my attempts to go back to Linux were bug-filled experiences -- ahh, good ole Linux. Instead, what I'm going to do is not upgrade at all, until a major release comes along; at which point I will backup my home directory to an external HDD, and perform a clean install.

Even now, I'm looking at having to remove and reinstall all the packages I'm using, because I've been installing them from the STABLE branch, when I should have gone RELEASE. Since my clean install, I've been quickly trying to install every package I need, because if I install anything later, chances are it's going to want different versions of packages I already have installed -- such is the incredibly annoying nature of STABLE.

In every other way, I think FreeBSD beats the 'competition,' but upgrades/package management is definitely not one of them.

vand777

December 28th, 2011, 21:50

Keeping FreeBSD up to date is a nightmare; such a nightmare, that I have abandoned the idea of attempting it anymore.

I would disagree with you. My servers are always up-to-date. Both FreeBSD and ports. I've been upgrading between major releases many times without having any problems.

There are few simple rules. If you follow them then all upgrades will be very smooth and nothing will be broken.

vermaden

December 28th, 2011, 22:36

Keeping FreeBSD up to date is a nightmare; such a nightmare, that I have abandoned the idea of attempting it anymore.
This will be solved by PKGNG project, take a look here: http://youtu.be/IRa6wFBLU28

The last time I attempted a binary upgrade from 8.1 to 8.2, it borked my system so bad that I couldn't even perform a clean install without using DBAN to completely erase the hard drive.
I do not remember any 'mine' problems with binary FreeBSD update, but I 'walked' that road: 5.4 -> 6.0 -> 6.1 -> 6.2 -> 7.0 -> 7.1 without any problems by doing source upgrades (make buildworld ...)

I appreciate the effort you've gone to here, but the multitude of steps involved don't really fill me with confidence.
Its probably the longest HOWTO I have ever written, but maybe because I wanted to explain all the steps and the 'why that way' behind them, seems pretty simple after several tries, try it under clean machine in VirtualBox first to catch some confidence.

Instead, what I'm going to do is not upgrade at all, until a major release comes along; at which point I will backup my home directory to an external HDD, and perform a clean install.
Its definitely not my problem, but I would at least suggest keeping open eye on FreeBSD Security Advisories and patching when needed, along with portaudit from Ports.

Even now, I'm looking at having to remove and reinstall all the packages I'm using, because I've been installing them from the STABLE branch, when I should have gone RELEASE.
Thats very easy to accomplish, make a list of Your installed packages, then remove all installed packages, then install them again from the list You have just made, like that:

# pkg_info -qoa > LIST
# pkg_delete -a
# rm -r -f /var/db/pkg /usr/local
# while read I; do pkg_add -r $( basename ${I} ); done < LIST

In case You are using a CSH shell, type sh before while loop, like that:

# pkg_info -qoa > LIST
# pkg_delete -a
# rm -r -f /var/db/pkg /usr/local
# sh
# while read I; do pkg_add -r $( basename ${I} ); done < LIST

Since my clean install, I've been quickly trying to install every package I need, because if I install anything later, chances are it's going to want different versions of packages I already have installed -- such is the incredibly annoying nature of STABLE.
It may sound up like an advertising, but using methods described in this HOWTO solves that problem, packages are built every 2 weeks, and if newer version appears, its installed.

In every other way, I think FreeBSD beats the 'competition,' but upgrades/package management is definitely not one of them.
FreeBSD lacks a lot in serious clustering, like 3+ nodes sharing many resource groups with their services, a shared clustering filesystem, drivers for various Fibre Channel controllers etc, but that is a topic for long discussion on another thread ;)

Regards,
vermaden

gkontos

December 29th, 2011, 01:25

This will be solved by PKGNG project, take a look here: http://youtu.be/IRa6wFBLU28

I saw the video and read the wiki. I wasn't aware of the project. Looks very promising and it is certainly something that would give a competitive advantage to FreeBSD.

Do you have more info regarding this project?

vermaden

December 29th, 2011, 07:46

@gkontos

You can follow its development here:
http://freshbsd.org/search?project=pkgngc

purgatori

December 30th, 2011, 23:18

First off, thanks for your thorough reply vermaden :)

This will be solved by PKGNG project, take a look here: http://youtu.be/IRa6wFBLU28

Oh wow, it's great to know that something is in the pipeline. I had no knowledge of this project, and was despairing that the only solution anyone had even conceived of was the 'PBI' + jails system that PCBSD uses -- in my opinion, this is a solution that ends up being worst than the problem ;P

I do not remember any 'mine' problems with binary FreeBSD update, but I 'walked' that road: 5.4 -> 6.0 -> 6.1 -> 6.2 -> 7.0 -> 7.1 without any problems by doing source upgrades (make buildworld ...)

Its probably a mistake I made along the way somewhere. I'm not a computer, let-alone FreeBSD, expert. FreeBSD is a tool I use in aid of my profession (writing/editing), and although I have done my best to learn its ins-and-outs, I don't have the time it would take to achieve your level of knowledge.

Despite its reputation, I don't think FreeBSD is terribly difficult to learn and use, even if you're not a computing boffin; that is, with the exception of package management. It would be nice if *that* was a little more idiot-proof, so to speak ;)

Its probably the longest HOWTO I have ever written, but maybe because I wanted to explain all the steps and the 'why that way' behind them, seems pretty simple after several tries, try it under clean machine in VirtualBox first to catch some confidence.

I was in no way taking issue with your HOWTO. On the contrary, I think your HOWTO was very thorough, and helfpful. It's just the amount of steps that need to be taken in order to 'do it right' are kinda silly compared to:

apt-get update
apt-get upgrade

That's a fault with FreeBSD's package management system, not your HOWTO. I'm not saying Debian/Ubuntu's system is perfect, of course, but it is a lot more straightforward, and worked... the majority of the time, at least.

Thats very easy to accomplish, make a list of Your installed packages, then remove all installed packages, then install them again from the list You have just made, like that:

Ahhh, very handy, thankyou :)

FreeBSD lacks a lot in serious clustering, like 3+ nodes sharing many resource groups with their services, a shared clustering filesystem, drivers for various Fibre Channel controllers etc, but that is a topic for long discussion on another thread

Waaaaaay outta my depth, I'm afraid :p I suppose I'm evaluating things more from a desktop-users perspective. My activities on FreeBSD basically consist of: editing/proofreading documents in Emacs (which is as much my operating system as FreeBSD is :p ), writing LaTeX documents, running statistics & generating graphics in R, browsing the web/gopher, and watching videos/listening to music. For all of these activities, I find FreeBSD to perform admirably and with rock-solid stability -- and all on rather ancient hardware. But, as you rightly say, that's the topic of another discussion.

vand777

December 31st, 2011, 11:42

Waaaaaay outta my depth, I'm afraid :p I suppose I'm evaluating things more from a desktop-users perspective. My activities on FreeBSD basically consist of: editing/proofreading documents in Emacs (which is as much my operating system as FreeBSD is :p ), writing LaTeX documents, running statistics & generating graphics in R, browsing the web/gopher, and watching videos/listening to music. For all of these activities, I find FreeBSD to perform admirably and with rock-solid stability -- and all on rather ancient hardware. But, as you rightly say, that's the topic of another discussion.

In this case, PCBSD might be a more convenient choice for you: http://www.pcbsd.org/.

SNK

December 31st, 2011, 15:42

Or try GhostBSD (http://ghostbsd.org/)

purgatori, FreeBSD is indeed an excellent choice for the activities you mention (math/R, LaTeX, etc., I use these on a daily basis as well.) If you have a day or two available then just experiment with source upgrades (tracking -STABLE) and port maintenance. The ports that get both (i) regular updates and (ii) need an hour or so to compile (because you may not want to wait for the package) will probably only be your web browser and only if you use e.g. www/chromium or www/firefox. The investment (including headaches) will pay off.

vermaden

January 1st, 2012, 21:59

First off, thanks for your thorough reply vermaden :)
Welcome.

Despite its reputation, I don't think FreeBSD is terribly difficult to learn and use, even if you're not a computing boffin; that is, with the exception of package management. It would be nice if *that* was a little more idiot-proof, so to speak ;)
The good thing about FreeBSD, is when You learn something, that knowledge/experience very rarely outdates, with Linux You need to 'update Your knowledge' every release and as someone stated here, its often a lot different 'procedure' to achieve the same thing in, for example, RHEL 4.x, 5.x and 6.x, so You need to learn 3 different procedures, in FreeBSD, its just one.

I was in no way taking issue with your HOWTO. On the contrary, I think your HOWTO was very thorough, and helfpful. It's just the amount of steps that need to be taken in order to 'do it right' are kinda silly compared to:

apt-get update
apt-get upgrade

Its the difference between Linux and FreeBSD world, at Linux, everything is a regular package, even the kernel, or init, or crontab, so the command apt just updates them all.

In FreeBSD lan You have 'most important core', the FreeBSD's base system and optional packages, because these two 'worlds' can be tracked and updated without 'touching' the other one there is separate procedure for updating each of them.

Also, its not typical to update the base system as long as everything is working ok and there are no new security vulnerabilities, for the most part You just update the installed optional packages, which means using only these commands:

# ports-check
# ports-update

Waaaaaay outta my depth, I'm afraid :p I suppose I'm evaluating things more from a desktop-users perspective. My activities on FreeBSD basically consist of: editing/proofreading documents in Emacs (which is as much my operating system as FreeBSD is :p ), writing LaTeX documents, running statistics & generating graphics in R, browsing the web/gopher, and watching videos/listening to music. For all of these activities, I find FreeBSD to perform admirably and with rock-solid stability -- and all on rather ancient hardware. But, as you rightly say, that's the topic of another discussion.

In 'that part' we are very similar, I also use FreeBSD as my NAS and as my laptop/workstation: http://browse.deviantart.com/?q=by:vermaden+freebsd

purgatori

January 4th, 2012, 02:21

In this case, PCBSD might be a more convenient choice for you: http://www.pcbsd.org/.

I actually used PCBSD this time to install FreeBSD, because it sets up networking and a few other little things for me (for some reason, I couldn't get auto dhcp to discover my settings through Sysintall), but I don't like the PBI system at all.

It's kinda ok if the stuff you want is in the PBI library, but if it's not, then you have to install it through packages or ports anyway. When you do so, you run the risk of borking the system due to version conflicts or whatever else -- in one instance, I lost audio because PCBSD uses some sort of tweaked/driver system to initialize your sound hardware, and an updated package I installed overwrote it -- unless you use jails. Jails are, in my opinion, too inconvenient to bother with.

@SNK: I'm not a Gnome user (I favor Ratpoison, Stumpwm and the like),but GhostBSD looks very cool. I was kinda wondering why someone hadn't made a 'desktop edition' of FreeBSD, other than PCBSD and its problematic PBIs.

As for ports, I have definitely 'experimented' with them. The first two times I installed and operated FreeBSD, I used ports rather than packages, but I ran into similar problems with version conflicts, and no easy way to upgrade installed ports/packages. At least with packages, things don't take too long to install if something goes awry.

The good thing about FreeBSD, is when You learn something, that knowledge/experience very rarely outdates, with Linux You need to 'update Your knowledge' every release and as someone stated here, its often a lot different 'procedure' to achieve the same thing in, for example, RHEL 4.x, 5.x and 6.x, so You need to learn 3 different procedures, in FreeBSD, its just one.

Tragically true. I remember moving between releases back when I was using Ubuntu, and how there was always some new back-end or protocol succeeding another. Not only would you have to learn how to use it, but it often provided no benefit, and more often than not just broke everything: PulseAudio being a great example. System configuration also seemed to be increasingly opaque with each release; there were less human readable configuration files, and more arcane, semi-functional GUI-mediated rituals :p

That said, I have CentOS on my laptop. It pretty much 'just works,' but I think that's only because I don't mess with it much.

In 'that part' we are very similar, I also use FreeBSD as my NAS and as my laptop/workstation: http://browse.deviantart.com/?q=by:vermaden+freebsd

I notice we're also both fans of Fallout/Bethesda games, and have similar views on religion :p

vermaden

January 4th, 2012, 07:22

Tragically true. I remember moving between releases back when I was using Ubuntu, and how there was always some new back-end or protocol succeeding another. Not only would you have to learn how to use it, but it often provided no benefit, and more often than not just broke everything: PulseAudio being a great example. System configuration also seemed to be increasingly opaque with each release; there were less human readable configuration files, and more arcane, semi-functional GUI-mediated rituals :p
I forgot about 'Ubuntu problems' as I havent used that distribution since quite long time, 2008 maybe, but Ubuntu changes seem to be much frequent then UpstreamVendor/CentOS ones, it must be a horror to keep up with all thos changes ;)

That said, I have CentOS on my laptop. It pretty much 'just works,' but I think that's only because I don't mess with it much.
Besides the need to disable SELinux and that its still Linux and the need to always add all these various additional repositories that provide me the packages that I have at FreeBSD from the start, its not that bad, at least comparing to other Linux distributions ;)

I notice we're also both fans of Fallout/Bethesda games, and have similar views on religion :p
Good to know that there are more people like me, I am a big fan of Fallout series, but definitely pre-Bethesda ones, I would even say pre-Tactics ones ;)

Regards,
ver

hadrons123

January 7th, 2012, 19:25

PART II. Keeping the FreeBSD packages up-to-date

I have FreeBSD 9.0-RC1. I can't use the ports-check after following the above guide. I tried both bash and sh and it doesn't make any difference. Ports-check doesn't work if it's a root or a user account. What am I doing wrong? Do I need to edit anything else considering I am using 9.0-RC1?

[root@BSD ~]# ports-check
bash: ports-check: command not found
[root@BSD ~]# ports-audit
bash: ports-audit: command not found

vermaden

January 7th, 2012, 21:59

You need to have these functions KNOWN by Your shell, for example they need to be at ~/.zshrc for the ZSH shell and You can always 'add' them manually by typing source ~/.zshrc in terminal.

I also do not know where You got the ports-audit, its not part of my HOWTO ;)

vermaden

January 17th, 2012, 09:03

Added PDF and HTML formats for download (first post).

purgatori

January 18th, 2012, 04:09

Added PDF and HTML formats for download (first post).

That's awesome. Definitely going to keep that PDF on hand as a reference =D

vermaden

February 2nd, 2012, 11:57

Welcome.

ryu

March 24th, 2012, 22:57

I've just installed FreeBSD 9.0-RELEASE and would like to update to FreeBSD 9.0-STABLE and have my system up-to-date. But it looks like this guide is for FreeBSD 8.2 or some older version?

cat > /root/stable-supfile << EOF
*default host=cvsup.pl.freebsd.org
*default base=/var/db
*default prefix=/usr
*default release=cvs tag=RELENG_8
*default delete use-rel-suffix
*default compress
src-all
EOF

Should this not be RELENG_9 ? Is it even possible to update with this guide from RELEASE to STABLE? Sorry for the noobish questions, but i don't want to destroy my fresh installed system.

bbzz

March 24th, 2012, 23:29

First of all, why do you want to go from -RELEASE to -STABLE? Are you sure you know what you are doing?

Remember, you can use all the packages from -STABLE without needing to move base system from -RELEASE.

ryu

March 24th, 2012, 23:50

First of all, why do you want to go from -RELEASE to -STABLE? Are you sure you know what you are doing?

Well, I guess FreeBSD-STABLE is something like Debian Testing or Debian Sid. If I'm right in this case, I think I know what I'm doing. Would just like to have a fully up-to-date system.

Remember, you can use all the packages from -STABLE without needing to move base system from -RELEASE.

All right. Didn't know that. I've read in the handbook about freebsd-update, but it seems like it is not the best choice. For example, I want to update Opera and Pidgin to the newest STABLE version, how would I do this without also updating the RELEASE base system?

wblock@

March 25th, 2012, 01:25

The operating system and applications aren't tied together. There is no -stable version of ports, the ports tree always has the latest version that's been ported. The reason to run FreeBSD-STABLE is to get the latest features and bug fixes in the operating system.

Yampress

March 25th, 2012, 10:33

STABLE -> Probably because of the binary package, he can update to newer version ...

fonz

March 25th, 2012, 18:55

The reason to run FreeBSD-STABLE is to get the latest features and bug fixes in the operating system.
FWIW: As well as the new bugs. -STABLE doesn't break often, but it does happen occasionally.

Fonz

srivo

May 29th, 2012, 21:48

I was using that script in my /etc/profile and when I installed gnome2 with GDM, GDM doesn't want to start with this script! I put it in .profile and the same thing happens when I try to log with my user.

Is there any other place I can put it?

stefanlasiewski

July 3rd, 2012, 01:00

Thats very easy to accomplish, make a list of Your installed packages, then remove all installed packages, then install them again from the list You have just made, like that:

# pkg_info -qoa > LIST
# pkg_delete -a
# rm -r -f /var/db/pkg /usr/local
# while read I; do pkg_add -r $( basename ${I} ); done < LIST

In case You are using a CSH shell, type sh before while loop, like that:

I have a FreeBSD 8.3-RELEASE system which I just upgraded to FreeBSD 9.0-RELEASE. When I try the above command, most packages install cleanly. But I get a variety of errors, like this:

# while read I; do pkg_add -r $( basename ${I} ); done < LIST
Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.0-release/Latest/ruby-bdb.tbz: File unavailable (e.g., file not found, no access)
Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.0-release/Latest/ncftp3.tbz: File unavailable (e.g., file not found, no access)
Error: Unable to get ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.0-release/Latest/perl5.12.tbz: File unavailable (e.g., file not found, no access)
[/CODE]

From what I can tell, according to `pkg_info -qoa`, my system had a package named databases/ruby-bdb . However, there is no package named ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-9.0-release/Latest/ruby-bdb.tbz .

What is happening here? Is does ruby-bdb have a different name in ports then as a package?

I can go ahead and install the packages by hand on this one system, but is there a way to cleanly do this automatically with some shell code?

jb_fvwm2

July 3rd, 2012, 02:59

I can go ahead and install the packages by hand on this one system, but is there a way to cleanly do this automatically with some shell code?
For next time maybe, you may find the small howto I wrote yesterday useful... Unsure how
it would work for all ports, but I just completed a reinstall seamlessly.
Every so often I'd have to stop/restart it, thereupon I'd check the latest packages saved
to /usr/ports/packages/All and delete the subset (ports already
reinstalled) from the temporary file (package)
list (see the writeup in the How_To section...) with "edit" or "nano -w", to restart the reinstall again.

stefanlasiewski

July 3rd, 2012, 19:18

For next time maybe, you may find the small howto I wrote yesterday useful...

Thank you.

I assume you mean this tutorial: Unexpectedly easy reinstall with portmaster of some ports (mostly Perl) (http://forums.freebsd.org/showthread.php?t=33119)

I will check it out!

vermaden

July 9th, 2012, 22:40

What is happening here? Is does ruby-bdb have a different name in ports then as a package?

Yes, its sometimes that way, that package has different name then a port.

I can go ahead and install the packages by hand on this one system, but is there a way to cleanly do this automatically with some shell code?
Probably yes, it will require scanning all 23 000 Makefiles in the Ports tree to get create the mapping of port name and a package name, and according to that mapping translate port name to package name and vive versa.

JackOps

September 29th, 2012, 15:02

Awesome, great, this is what I was looking for. I'm new with FreeBSD but on my environtment at least there are three machines, and I update it manually. Thanks.

emmex

March 29th, 2013, 10:30

I have successfully upgraded my laptop O.S. to 9-STABLE from PC-BSD (not FreeBSD).

But I noticed that the binary packages in are, at least, 5 months old. Why? Can I use an alternative package site repository?