Hi gang!
Editorial: I've been using ports-mgmt/portmaster for a long time already and I figured it could use a little bit of extra love and attention
Also because there is a lot which Portmaster can do, so I'm hoping that briefly showcasing some of those features might be helpful for some of you out there.
The ports collection
As you probably know FreeBSD provides tons of 3rd party software through the so called ports collection. The ports collection is basically what I like to describe as a collection of blue prints which can tell the system how to obtain a certain piece of software (usually the source code, but binaries are also possible), then in case of a source tree it'll showcase how the application should be build (compiled) and finally a package is made which then gets installed using pkg.
Managing your ports collection is usually done using portsnap(8) but you can also manually keep track of things usingSubversion (Update: as per January 2021 Subversion got replaced by Git(1) and I'm not sure if a manual checkout still applies). But that's also beyond the scope of this guide, we're going to look at the process of building a port, and Portmaster in particular.
The vanilla way of installing a port
Most people who build ports manually will probably know about the
Now, the vanilla routine definitely gets the job done but it's not perfect. For example: if you're compiling a larger port such as, for example, www/apache24 then you're looking at a pretty large amount of dependencies, some will be required to actually use the port, whereas others will be required for the building process:
.... And some of the problems.
Now, this is all fine and well but it creates a bit of a hassle: plenty of those dependencies will have their own configuration to deal with, which then could lead up to the system presenting you with a configuration screen 35 minutes after having started the build process. While you were already busy getting some tea and perhaps do the dishes Not quite useful, and basically a waste of time.
Of course there are ways to overcome this, for example by using the
Also: have you ever thought about using pkg-create(8) to make a backup of whatever it is you're installing before you replace it with a newer version?
And what about all those dependencies? As you might know a port has 2 types of dependencies: build and run dependencies. You can check for these yourself using the run-depends-list and build-depends-list build targets respectfully. What if you wanted to get rid of those build dependencies to save disk space?
Well, for all that and a whole lot more there's Portmaster. An rather small utility which can automate several of these tasks for you.
Portmaster
Portmaster is a small (well..., sort off) shell script which uses all the default standard commands to automate the building of a port. You can get it by running
An example... If we once again look at the previously mentioned www/apache24 port then all it takes to install that would be:
After issuing this command Portmaster will check for all the required dependencies and go over all of them. If some of them haven't been configured yet then you can just sit back and relax because Portmaster will show you all the required configuration for all those ports which are being processed, long before the actual building starts.
If Portmaster needs to do more tasks other than building the port which you specified (for example: installing build or run dependencies) then you will first be presented with a list of everything Portmaster is about to do, otherwise the building will be done straight away.
Of course Portmaster can do more than that...
However, always keep in mind that Portmaster basically uses the default set of tools. For example, one of its features is the --list-origins option, this will produce a list of all your installed ports, and that list can be directly fed to Portmaster again so that it can (re)install all the ports when needed. It's a useful option for sure, but... nothing which
Configuration & logging
Now, Portmaster has several options and I'll go over some of them in a moment, but before I do I'd like to specifically point your attention to Portmasters configuration file, by default this can be found in /usr/local/etc/portmaster.rc. Do note that this doesn't get automatically installed, look for portmaster.rc.sample when in doubt.
The best thing to do is to go over all the available options yourself, but there are a few which I'd like to specifically mention...
This option can be a life safer. What it does is ensure that Portmaster will create a backup copy of the currently installed port before applying an upgrade. This ensures that you'll be able to go back to the previous version no matter what.
So if something goes wrong simply go to /usr/ports/packages/portmaster-backup and use pkg-add(8) to install it (after you removed the previous version of course).
(2021 addition)
And in the same line as above you can also chose to save copies of old shared libraries. This is a highly recommended option (as you can see) which is why I added it. While it is true that this could create some overlap I'd still recommend to use both backup options. For example... say you're rebuilding a port and its dependencies, One of which is a library which other ports also depend on. So what would happen if you changed a build option for the library which caused other ports to stop functioning? Sure, if you also backup old ports you'd have 2 backup copies, true. But I'd rather have 2 than none
(/2021 addition)
When you're building (new) software then the system will start by downloading the original (vanilla) source code package, extract that and then optionally apply some FreeBSD specific patches to it. But what about all those distribution files? Eventually they'll slowly fill up your system.
Portmaster counters for this and it will always ask you if it's ok to remove the unused (existing) version. However, that can sometimes become quite annoying because this basically means that after every successful package installation you'd have to tell Portmaster if it can remove the old distifles. So basically: instead of getting configuration options presented to you somewhere during the build process, you now get Yes/No questions
Fortunately this option can help you counter that.
This will allow you to always keep track of what has been done on your system. I personally consider this very useful because I can always check back at what has happened on my system in the past. If you are going to use this approach then I also recommend that you look into newsyslog.conf(5) in order to ensure that the sizes of your logfile(s) doesn't grow out of hand (see also /usr/local/etc/newsyslog.conf.d).
Some (maybe) interesting command options
(fun fact: did you know that /usr/bin/more and /usr/bin/less are more or less the same program?)
Portmaster divides the installed ports into 4 main categories: Root, Trunk, Branch and Leaf ports.
Root ports are ports which are purely stand-alone. They don't have any dependencies, and other ports don't depend on them.
Trunk ports are depended on by other ports but have no further dependencies themselves.
Branch ports are what their name implies: branches. They are depended on by other ports and also have dependencies themselves.
Leaf ports finally have dependencies on other ports, but are not being depended on themselves.
Well, you may have guessed as much: this command will show you all the installed ports as well as the category they belong to.
Normally you can simply install a port by mentioning it on the Portmaster command line. But sometimes that might not be enough. By default Portmaster will check the dependencies and if they're already present (so: a needed package is already installed) then that's that.
But what if you need for the dependencies to get rebuild as well? Well, in that case the -f parameter may come in handy. It will force Portmaster to check the dependencies of the port you're (re)installing and it will rebuild all those other dependency ports as well.
Keep in mind though that this command recurses. So it will also "chain-check" all the dependencies, and apply the required rebuilding on those as well. Thorough for sure, but depending on the situation not always needed.
If you're trying to fix problems (for example building problems) and you think the cause could be with a mis-installed dependency then a "1 tier" rebuild (as I tend to call it) might be more useful (quicker too). So, in the ports directory run something like this:
(2021 update: Although sed does a good job it's also a lot of typing. These days I prefer using cut instead, much easier:
Of course, your mileage may vary as is the saying
In some way you could argue that -r is a little bit of the opposite of -f. Instead of rebuilding all the dependencies of the target port it tells Portmaster to rebuild all the ports which depend ('rely') on the port you're processing. This option can be especially useful if you're upgrading a library port and you want to ensure that all programs which use that get rebuild.
For example devel/apr1. If you look at all its requirements (= ports which depend on it) then you'll see yourself:
So using the above command would ensure that other than apr all these mentioned ports get rebuild also, even if they're up to date:
As mentioned a few times now the ports collection depends on official source packages. This is in my opinion also the beauty of the system: all it does is grab the source code from a 3rd party (official) repository, then process it.
But as you keep installing and upgrading ports your collection of distribution files ('distfiles') will slowly but steadily grow. That's where this command can come in handy: it will check all the downloaded distfiles and if it finds one which isn't associated with a currently installed port Portmaster will ask you if you want to remove it.
Of course you can also chose to simply remove all the distfiles (see /usr/ports/distfiles), but I'd advice against that. It won't break anything, no worries, but if you need to reinstall a port it also means you'll have to download the source again.
What gobbles up more disk space: one file of 10kb or four files of 10b? Sounds easy right? 10.000b > 40b, and thus the four files require less disk space. This may sound bizarre to you but you're probably wrong. However, it heavily depends on how your file system was set up. A file system is usually divided into data blocks, meaning that even a small 1kb file will take up at least a minimum amount of space:
Here you can see that the block size on this filesystem is 32768, divided over several groups this would boil down to an 8kb minimum size. So one file will at least gobble up approx. 8kb. So going back to my question above; four files of 8kb account for 32kb and will therefor take up more space.
Why all this dry theory you ask?
Every time a port gets configured the system will store the build options in /var/db/ports. These options are stored in a matching directory using the file options. As you can imagine these files aren't very large. For example, these are the stored options of my favorite archiver (rar):
This file is approx. 184 bytes in size. But effectively it gobbles up 8kb no matter what. Its regular size may be 184 bytes but its effective size is 8kb.
Now, for one single file this is obviously peanuts. And well, what is an extra 8kb exactly? But, uhm, what if you had 75 unused option files in your ports directory? 75 * 8 = 600kb, almost 1Mb.
Now this may not seem like much to you, but keep well in mind that block sizes can vary. On this system a file has a minimum size of 8kb. But larger file systems usually use larger block sizes, and that will result in larger minimum file sizes.
Fortunately Portmaster has you covered. By running the --check-port-dbdir command it will check the /var/db/ports directory and if it finds any unused option files it will offer to remove those.
So this command can save you some precious disk space!
Remember my comments about how some ports have build dependencies, so ports which they need to have installed before the port itself can be build? Although it's usually a good idea to leave these ports in place they do take up precious disk space. So... this option can help you remove those ports which were only installed to help build other ports.
(update 2021)
If you want to know the dependencies of a port you could use build targets such as run-depends-list and/or build-depends-list. While this does give you a list of ports which the main port will require, it doesn't tell you if these ports are already installed. That's what --show-work can do for you; it shows all the required ports and also if they're already installed or not.
(/update 2021)
And finally...
When in doubt: use -n!
If you're not sure about using a specific command then always keep the -n option in mind. This tells Portmaster to do a dry run; so it will simulate whatever you told it to do but without actually doing it. This can help you to see for yourself what would actually happen so that you can be sure that it is what you wanted.
And there you have it, a brief look at some of the Portmaster options.
Editorial: I've been using ports-mgmt/portmaster for a long time already and I figured it could use a little bit of extra love and attention
Also because there is a lot which Portmaster can do, so I'm hoping that briefly showcasing some of those features might be helpful for some of you out there.The ports collection
As you probably know FreeBSD provides tons of 3rd party software through the so called ports collection. The ports collection is basically what I like to describe as a collection of blue prints which can tell the system how to obtain a certain piece of software (usually the source code, but binaries are also possible), then in case of a source tree it'll showcase how the application should be build (compiled) and finally a package is made which then gets installed using pkg.
Managing your ports collection is usually done using portsnap(8) but you can also manually keep track of things using
The vanilla way of installing a port
Most people who build ports manually will probably know about the
# make config and # make install clean commands (also referred to as build targets). Basically you configure a ports options using the first command, and then use the latter to actually build and install the port after which the source tree gets cleaned up (provided the build was a success). But there's much more to it than that, in case you haven't already I definitely advice you to take a glimpse at the ports(7) manual page sometime, it will list most commonly available build targets, and some can be very useful.Now, the vanilla routine definitely gets the job done but it's not perfect. For example: if you're compiling a larger port such as, for example, www/apache24 then you're looking at a pretty large amount of dependencies, some will be required to actually use the port, whereas others will be required for the building process:
Code:
peter@macron:/usr/ports/www/apache24# make all-depends-list | wc -l
143Now, this is all fine and well but it creates a bit of a hassle: plenty of those dependencies will have their own configuration to deal with, which then could lead up to the system presenting you with a configuration screen 35 minutes after having started the build process. While you were already busy getting some tea and perhaps do the dishes
Not quite useful, and basically a waste of time.Of course there are ways to overcome this, for example by using the
# make config-recursive target, this will present you with all required options at the same time. But even so, it is a little bit prone to issues sometimes.Also: have you ever thought about using pkg-create(8) to make a backup of whatever it is you're installing before you replace it with a newer version?
And what about all those dependencies? As you might know a port has 2 types of dependencies: build and run dependencies. You can check for these yourself using the run-depends-list and build-depends-list build targets respectfully. What if you wanted to get rid of those build dependencies to save disk space?
Well, for all that and a whole lot more there's Portmaster. An rather small utility which can automate several of these tasks for you.
Portmaster
Portmaster is a small (well..., sort off
) shell script which uses all the default standard commands to automate the building of a port. You can get it by running # make install clean in the /usr/ports/ports-mgmt/portmaster directory.An example... If we once again look at the previously mentioned www/apache24 port then all it takes to install that would be:
# portmaster www/apache24. This can be executed from any random directory.After issuing this command Portmaster will check for all the required dependencies and go over all of them. If some of them haven't been configured yet then you can just sit back and relax because Portmaster will show you all the required configuration for all those ports which are being processed, long before the actual building starts.
If Portmaster needs to do more tasks other than building the port which you specified (for example: installing build or run dependencies) then you will first be presented with a list of everything Portmaster is about to do, otherwise the building will be done straight away.
Of course Portmaster can do more than that...
However, always keep in mind that Portmaster basically uses the default set of tools. For example, one of its features is the --list-origins option, this will produce a list of all your installed ports, and that list can be directly fed to Portmaster again so that it can (re)install all the ports when needed. It's a useful option for sure, but... nothing which
pkg info -qao couldn't handle.Configuration & logging
Now, Portmaster has several options and I'll go over some of them in a moment, but before I do I'd like to specifically point your attention to Portmasters configuration file, by default this can be found in /usr/local/etc/portmaster.rc. Do note that this doesn't get automatically installed, look for portmaster.rc.sample when in doubt.
The best thing to do is to go over all the available options yourself, but there are a few which I'd like to specifically mention...
Code:
# Always save the backup packages of the old port (-b)
BACKUP=boptSo if something goes wrong simply go to /usr/ports/packages/portmaster-backup and use pkg-add(8) to install it (after you removed the previous version of course).
(2021 addition)
Code:
# Save copies of old shared libraries (recommended) (-w)
SAVE_SHARED=wopt(/2021 addition)
Code:
# Always delete stale distfiles without prompting (-d)
ALWAYS_SCRUB_DISTFILES=doptPortmaster counters for this and it will always ask you if it's ok to remove the unused (existing) version. However, that can sometimes become quite annoying because this basically means that after every successful package installation you'd have to tell Portmaster if it can remove the old distifles. So basically: instead of getting configuration options presented to you somewhere during the build process, you now get Yes/No questions
Fortunately this option can help you counter that.
Code:
# Log actions taken by portmaster with a date/time stamp
PM_LOG=/var/log/portmaster.logSome (maybe) interesting command options
portmaster -l | less (of course feel free to use more if preferred).(fun fact: did you know that /usr/bin/more and /usr/bin/less are more or less the same program?
)Portmaster divides the installed ports into 4 main categories: Root, Trunk, Branch and Leaf ports.
Root ports are ports which are purely stand-alone. They don't have any dependencies, and other ports don't depend on them.
Trunk ports are depended on by other ports but have no further dependencies themselves.
Branch ports are what their name implies: branches. They are depended on by other ports and also have dependencies themselves.
Leaf ports finally have dependencies on other ports, but are not being depended on themselves.
Well, you may have guessed as much: this command will show you all the installed ports as well as the category they belong to.
# portmaster -f security/gnupgNormally you can simply install a port by mentioning it on the Portmaster command line. But sometimes that might not be enough. By default Portmaster will check the dependencies and if they're already present (so: a needed package is already installed) then that's that.
But what if you need for the dependencies to get rebuild as well? Well, in that case the -f parameter may come in handy. It will force Portmaster to check the dependencies of the port you're (re)installing and it will rebuild all those other dependency ports as well.
Keep in mind though that this command recurses. So it will also "chain-check" all the dependencies, and apply the required rebuilding on those as well. Thorough for sure, but depending on the situation not always needed.
If you're trying to fix problems (for example building problems) and you think the cause could be with a mis-installed dependency then a "1 tier" rebuild (as I tend to call it) might be more useful (quicker too). So, in the ports directory run something like this:
# portmaster `make build-depends-list | sed -e 's/\/usr\/ports\///g'`. This would tell Portmaster to only reinstall those ports which are directly required for the rebuild process, but without looking at any of their dependencies.(2021 update: Although sed does a good job it's also a lot of typing. These days I prefer using cut instead, much easier:
# portmaster `make build-depends-list | cut -d '/' -f 4-`).Of course, your mileage may vary as is the saying
# portmaster -r apr-1In some way you could argue that -r is a little bit of the opposite of -f. Instead of rebuilding all the dependencies of the target port it tells Portmaster to rebuild all the ports which depend ('rely') on the port you're processing. This option can be especially useful if you're upgrading a library port and you want to ensure that all programs which use that get rebuild.
For example devel/apr1. If you look at all its requirements (= ports which depend on it) then you'll see yourself:
Code:
$ pkg info -rx apr
apr-1.6.2.1.6.0:
serf-1.3.9_1
subversion-1.9.5
p5-subversion-1.9.5_2
apache24-2.4.27
Code:
===>>> The following actions will be taken if you choose to proceed:
Re-install apr-1.6.2.1.6.0
Upgrade mysql56-client-5.6.36 to mysql56-client-5.6.37
Upgrade postgresql95-client-9.5.7_1 to postgresql95-client-9.5.7_2
Re-install serf-1.3.9_1
Upgrade subversion-1.9.5 to subversion-1.9.6
Upgrade p5-subversion-1.9.5_2 to p5-subversion-1.9.6_2
Re-install apache24-2.4.27 # portmaster --clean-distfilesAs mentioned a few times now the ports collection depends on official source packages. This is in my opinion also the beauty of the system: all it does is grab the source code from a 3rd party (official) repository, then process it.
But as you keep installing and upgrading ports your collection of distribution files ('distfiles') will slowly but steadily grow. That's where this command can come in handy: it will check all the downloaded distfiles and if it finds one which isn't associated with a currently installed port Portmaster will ask you if you want to remove it.
Of course you can also chose to simply remove all the distfiles (see /usr/ports/distfiles), but I'd advice against that. It won't break anything, no worries, but if you need to reinstall a port it also means you'll have to download the source again.
# portmaster --check-port-dbdirWhat gobbles up more disk space: one file of 10kb or four files of 10b? Sounds easy right? 10.000b > 40b, and thus the four files require less disk space. This may sound bizarre to you but you're probably wrong. However, it heavily depends on how your file system was set up. A file system is usually divided into data blocks, meaning that even a small 1kb file will take up at least a minimum amount of space:
Code:
peter@macron:/var/db/ports/archivers_rar# file -s /dev/ada0p2
/dev/ada0p2: Unix Fast File system [v2] (little-endian) last mounted on /, last written at Mon Jul 31 15:21:12 2017, clean flag 0, readonly flag 0, number of blocks 262144, number of data blocks 253831, number of cylinder groups 4, block size 32768, fragment size 4096, average file size 16384, average number of files in dir 64, pending blocks to free 0, pending inodes to free 0, system-wide uuid 0, minimum percentage of free blocks 8, TIME optimizationWhy all this dry theory you ask?
Every time a port gets configured the system will store the build options in /var/db/ports. These options are stored in a matching directory using the file options. As you can imagine these files aren't very large. For example, these are the stored options of my favorite archiver (rar):
Code:
# This file is auto-generated by 'make config'.
# Options for rar-5.4.0,3
_OPTIONS_READ=rar-5.4.0,3
_FILE_COMPLETE_OPTIONS_LIST=DOCS SFX
OPTIONS_FILE_UNSET+=DOCS
OPTIONS_FILE_SET+=SFXNow, for one single file this is obviously peanuts. And well, what is an extra 8kb exactly? But, uhm, what if you had 75 unused option files in your ports directory? 75 * 8 = 600kb, almost 1Mb.
Now this may not seem like much to you, but keep well in mind that block sizes can vary. On this system a file has a minimum size of 8kb. But larger file systems usually use larger block sizes, and that will result in larger minimum file sizes.
Fortunately Portmaster has you covered. By running the --check-port-dbdir command it will check the /var/db/ports directory and if it finds any unused option files it will offer to remove those.
So this command can save you some precious disk space!
# portmaster --delete-build-only.Remember my comments about how some ports have build dependencies, so ports which they need to have installed before the port itself can be build? Although it's usually a good idea to leave these ports in place they do take up precious disk space. So... this option can help you remove those ports which were only installed to help build other ports.
(update 2021)
# portmaster --show-work www/apache24If you want to know the dependencies of a port you could use build targets such as run-depends-list and/or build-depends-list. While this does give you a list of ports which the main port will require, it doesn't tell you if these ports are already installed. That's what --show-work can do for you; it shows all the required ports and also if they're already installed or not.
(/update 2021)
And finally...
When in doubt: use -n!
If you're not sure about using a specific command then always keep the -n option in mind. This tells Portmaster to do a dry run; so it will simulate whatever you told it to do but without actually doing it. This can help you to see for yourself what would actually happen so that you can be sure that it is what you wanted.
And there you have it, a brief look at some of the Portmaster options.
Last edited:
