Questions from the "About ports and (binary) packages" howto

S

sackadoo

A couple of questions....
1) If I use all default settings while installing from ports, mixing ports and packages isn't a problem?
2) If I install (for instance) bash from the ports collection, installing an unrelated package (x-windows?) shouldn't be a problem?
 
sackadoo said:
1) If I use all default settings while installing from ports, mixing ports and packages isn't a problem?
Click to expand...
Not necessarily, a better description here would be: it may not have to be a problem. But it also remains to be seem how long that status quo is going to last.

See, another issue here is that updates will first appear in the ports collection, so before they appear in the binary repositories. Simple reasoning: those binaries get build from the same ports collection, and building takes time. So if you then start working with an updated ports collection while also having out of date binary packages installed you're in for a nasty dependency mess.

Seriously: if you're just going to use default settings then it's best not to bother with ports at all, rely on binaries. FreeBSD isn't like Linux where several developers seem clueless about optimization. Normally a binary package is already well optimized (as a side note: the same applies to the kernel, FreeBSD also doesn't have this needless demand for kernel building either, it's actually a waste of time if you're not going to apply specific changes. And to make matters worse: even applying specific changes can sometimes be a bad idea ;)).

sackadoo said:
2) If I install (for instance) bash from the ports collection, installing an unrelated package (x-windows?) shouldn't be a problem?
Click to expand...
Can't say for sure. Maybe not, maybe so. See...

Code: 
root@vps:/usr/ports/x11/xorg # make all-depends-list | wc -l
     412
While x11/xorg may not directly rely on bash, what about any of those 412 dependencies? Or any of the ports which those 412 ports possibly depend on?

And of course: what about ports that are depended on by both Bash and Xorg (or one of its many required ports)?

In the end I wouldn't bother with this, either build the whole thing or rely on binary packages. It's bound to spare you a lot of dependency issues in the upcoming future.
 
sackadoo said:
1) If I use all default settings while installing from ports, mixing ports and packages isn't a problem?
Click to expand...
Not sure whether this is already somewhere in this howto, didn't find it quickly:

I have the impression that the biggest source of "mixing problems" for people is mixing quarterly and latest. On a fresh FreeBSD installation, pkg is configured to use repositories built from the "quarterly" branch, while tools like portsnap will by default fetch a "latest" ports tree. Mixing this is a recipe for getting into trouble very quickly, so either reconfigure pkg to use "latest" packages or check out a quarterly branch of the ports tree for building your own.

I assume other problems are already addressed ;) In general, once you start mixing, you will of course have more manual maintenance work to do.

edit: quick conclusion: it's not so much about build options; for them, follow the simple rule that if you change options for a port, make sure to build all dependent ports yourself as well.
 
I like ports and have installed graphics/gimp from pkg as last resort but only because it's a program I use daily.

pkg didn't make a machineid like it should have to function and pkg installed a version of graphics/openEXR that had an active vulnerability according to pkg audit -F. Which is where ports-mgmt/portmaster would throw an error and stop the build, just as I would want it to.

I had to look but the file wasn't hard to create. I deinstalled openEXR manually due to reason stated and Gimp worked fine without it. Relatively easy for someone with experience to work through. Most likely not as easily before I had enough struggle time in figuring things out behind me.

And that's just an example of something bound to come up now and then using ports as part of the learning process. pkg is no doubt quicker than ports and no sense in making it hard on yourself mixing them when you don't have to. That opportunity will show up when you least expect it.
 
I have successfully mixed ports and packages, but with some caveats.

Most importantly, as mentioned above, make sure pkg is pulling from latest, not quarterly and your portstree is up to date. Secondly, install from pkg not ports and only then, for the hopefully very few ports you want custom settings for, update rather than install the relevant ports.

I use portinstall which will almost always either build and install the relevant port or stop and list the ports it's going to build. If it does stop and display a list of ports, hit N for No and check carefully what it's about to do. Any missing ports, install from pkg. Any other ports, think carefully if you want portinstall to rebuild/upgrade them. Personally, I just leave the port alone until after the next pkg update and see if the other out-of-date ports are now in sync. I will normally ONLY upgrade a port when it builds cleanly with the existing pkgs in place. Sometimes that means waiting a week or two, but it seems a lot safer to me.

In my case, it's a couple of ports which don't support mp3 OOTB for whatever reason.

Note that if this is a first time use of ports after exclusively using packages, you will likely find lots of build dependancies that you need to add, again preferably from pkg. You probably don't want GCC, CMake LLVM and/or others all being built from ports just to install a custom build of audio/sox!
 
sackadoo said:
A couple of questions....
1) If I use all default settings while installing from ports, mixing ports and packages isn't a problem?
Click to expand...
The problem is that it likely won't be a problem, perhaps even for years. When you do stumble into an incompatibility deep in your dependency tree, however, it will be very difficult to find. You may conclude Freebsd is just broken and abandon it. This is why we don't recommend you even try this until you know what you're doing.

For the record, I don't do this. It's not worth the trouble. I'll burn the extra time building Rust in Poudriere just to save myself from wandering around in WTF land.
 
Now, first of all a well meant comment: please be more careful with your quoting. You don't always have to quote something to answer a question and well... quoting an entire guide (or message) for a single question (or answer) is quite counter productive and only makes things harder to read. Depending on the size of the original message of course...

Barney said:
Where is the source code used to build packages, for when I need to compile something from source but the source doesnt compile and there is no port?
Click to expand...
I'm not sure I understand your question.

Packages get build from ports, so if you have a package you'll also have access to a port *. The port will grab the source code (if available **) and build it. If you only want to get access to the source code you'd use the make extract command within the port directory, see also the ports(7) manualpage.


* Provided that the software was part of the ports collection in the first place. People can distribute packages without the source code, so then you obviously wouldn't get access to it.

** There is also closed-source software within the ports collection, obviously you don't get access to any source code; in these cases the port grabs the software and sets it up on your system.
 
Mixing is unavoidable if you use binary packages but want something that cannot be packaged for licensing or other reasons, such as libdvdcss. Until recently, some very common tools like faac, lame, and exfat were also not available as binary packages. Fortunately, restrictions tend to become more relaxed over the years, but there will always be some things we have to install from source.

The important thing is to keep your installed packages and ports tree in sync. That means on the same branch (quarterly or latest/head) and up-to-date. The default FreeBSD install uses quarterly packages by default. If you stay with that, be sure to clone the matching quarterly ports tree and upgrade it at the start of each quarter.
auto-update-system from sysutils/auto-admin will ensure that your ports tree and packages are always in sync. It updates packages, ports, and base all at once and makes sure your porta and packages are on the same branch.
I frequently run auto-update-system --defaults, then review the output and decide if I should reboot.

sysutils/desktop-installer uses these tools under the hood to ensure that your setup is consistent from the start.
 
outpaddling said:
Mixing is unavoidable if you use binary packages but want something that cannot be packaged for licensing or other reasons, such as libdvdcss.
Click to expand...
No it's not. You can easily switch from binary to building your own ports for example.

Or... install the software outside the packaging system, so manually.

And, uhm...

Code: 
libdvdcss-1.4.2
Name           : libdvdcss
Version        : 1.4.2
Origin         : multimedia/libdvdcss
Architecture   : FreeBSD:12:amd64
Prefix         : /usr/local
Repository     : FreeBSD [pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/quarterly]
Categories     : multimedia
Licenses       : GPLv2
Maintainer     : jpaetzel@FreeBSD.org
WWW            : http://www.videolan.org/developers/libdvdcss.html
Comment        : Portable abstraction library for DVD decryption
Options        :
What made you so sure that libdvdcss wasn't available as a package while it clearly is? :rolleyes:
 
ShelLuser said:
What made you so sure that libdvdcss wasn't available as a package while it clearly is? :rolleyes:
Click to expand...
It is about some options, which are verboten someplace of the world. Like decss for DVDs or TrueType fonts for libreoffice. That was a patent issue, no idea if it still is.
 
outpaddling said:
I frequently run auto-update-system --defaults, then review the output and decide if I should reboot.
Click to expand...
I don't do auto Admin anything or run portmaster -a as SOP and never update anything unless ports-mgmt/portmaster recommends it as a dependency update. Or if it's a new version of something like www/youtube_dl and I want the latest version.

I've only built a Systenm using pkg once in all the time I've used FreeBSD and that was just a few months ago. I downloaded a snapshot of the ports tree and if there is a vulnerability that pkg hasn't addressed will use portmaster if a fix has hit the ports tree.

I haven't had any problems with it and am using that IBM T43 as my recliner-based text editor at 23 days uptime, located next to my W520 .mp3 player at 56 days uptime, so I can musically multitask with multiple machines.
 
ShelLuser said:
No it's not. You can easily switch from binary to building your own ports for example.

Or... install the software outside the packaging system, so manually.

And, uhm...

Code: 
libdvdcss-1.4.2
Name           : libdvdcss
Version        : 1.4.2
Origin         : multimedia/libdvdcss
Architecture   : FreeBSD:12:amd64
Prefix         : /usr/local
Repository     : FreeBSD [pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/quarterly]
Categories     : multimedia
Licenses       : GPLv2
Maintainer     : jpaetzel@FreeBSD.org
WWW            : http://www.videolan.org/developers/libdvdcss.html
Comment        : Portable abstraction library for DVD decryption
Options        :
What made you so sure that libdvdcss wasn't available as a package while it clearly is?
Click to expand...
I said "If you use binary packages ... mixing is inevitable". How does the fact that you don't have to use binary packages invalidate this statement?

Code: 
<<<ROOT@barracuda.uits>>> /home/bacon 1035 # pkg install libdvdcss
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
pkg: No packages available to install matching 'libdvdcss' have been found in the repositories

PORTNAME=       libdvdcss
DISTVERSION=    1.4.3
CATEGORIES=     multimedia
MASTER_SITES=   https://download.videolan.org/pub/${PORTNAME}/${DISTVERSION}/

MAINTAINER=     ports@FreeBSD.org
COMMENT=        Portable abstraction library for DVD decryption

LICENSE=        GPLv2 DMCA
LICENSE_COMB=   multi
LICENSE_NAME_DMCA=      DMCA
LICENSE_TEXT_DMCA=      CSS code may violate the DMCA
LICENSE_FILE_GPLv2=     ${WRKSRC}/COPYING
LICENSE_PERMS_DMCA=     auto-accept

libdvdcss used to be packaged, but it isn't anymore. Please read the post and gather the facts before you respond.
 
First of all, this is a guide, you should discuss your opinions elsewhere.

Secondly, you're dispensing bad advice. It is indeed possible to run a system with just binary packages, and mixing ports and packages is a really bad idea.

outpaddling said:
I said "If you use binary packages ... mixing is inevitable". How does the fact that you don't have to use binary packages invalidate this statement?
Click to expand...
Lastly, do you not see that that's a directly contradictory statement? You state it is "inevitable", yet somehow Shelluser and I have managed to do it. Do you know what the word "inevitable" means?
 
Jose said:
Secondly, you're dispensing bad advice. It is indeed possible to run a system with just binary packages, and mixing ports and packages is a really bad idea.
Click to expand...
Right you are.

I only do it as a last resort, and only because I feel confident I can work out any problems mixing ports and pkg is likely to bring with it down the line.

Something a new user would be unable to do, become needlessly frustrated by and might cause them to lose interest in FreeBSD
 
Jose said:
First of all, this is a guide, you should discuss your opinions elsewhere.

Secondly, you're dispensing bad advice. It is indeed possible to run a system with just binary packages, and mixing ports and packages is a really bad idea.
Click to expand...
Opinion? Those cheap sentences are the same as a "shut up & go away"… You know f.e. the option "NO_PACKAGE" in the Makefiles? Here's a list of ports with it:

audio/madfufw
devel/papi
lang/petite-chez
mail/sendmail-devel
mail/sendmail
misc/estic
misc/posixtestsuite
multimedia/sms1xxx-kmod
net/bwi-firmware-kmod
net/bwn-firmware-kmod
net/dimes
net/malo-firmware-kmod
net/opennx
net/vmware-vsphere-cli
security/stunnel
security/tripwire-131
security/truecrypt
security/vpnc

So, let's tell me how to get one of them as binary package - otherwise you should write an apology ;) outpaddling wrote some really good hints if it comes to mixing ports and packages. And not an opinion.
 
Reiterating the original statement that's inexplicably under attack here:

"Mixing is unavoidable if you use binary packages but want something that cannot be packaged for licensing or other reasons..."

This is obvious, not controversial. If there is no binary package, you have to build from source. So your choices are either mix or don't use binary packages at all.

Users will run into problems with compatibility *IF* their ports tree and installed packages are not in sync. That's why I wrote auto-update-system and put it in the auto-admin menu, so new users won't get themselves into trouble. It's a very robust script developed and tested over 10+ years. Trying to manage a FreeBSD system without tools like this is what scares new users away.

There is nothing inherently bad about mixing ports and packages. The only problem is people not knowing how to do it safely. I've been doing it for many years without any problems. Not only for licensing reasons, but also for optimizing scientific software. A small number of ports will benefit greatly from building with -march=native on a CPU with AVX extensions, for example. Building all of the 900+ ports on my scientific workstation would be an absurd waste of time and electricity, though, given that only a handful would benefit from AVX in a meaningful way. One can just add CFLAGS+=-march=native to /etc/make.conf, then cd /usr/ports/biology/mmseqs2 && make install to get a binary that's ~30% faster than the binary package.

There are also some ports that will fail to build in the presences of certain installed software, but build fine under poudriere, which always starts with a clean slate. Hence, using binary packages avoids some problems and falling back on building from source only when necessary is a good strategy.

Lastly, if building something from source with many large dependencies, one can expedite the process greatly by first installing the dependencies as binary packages. This will often reduce hours of build time to minutes.

Again, if you do use both binary packages and ports, be sure that both your installed packages and ports tree are on the same branch and fully updated. If you do that, you'll likely never encounter a problem.
 
jmos said:
Opinion? Those cheap sentences are the same as a "shut up & go away"… You know f.e. the option "NO_PACKAGE" in the Makefiles? Here's a list of ports with it:
...
net/bwi-firmware-kmod
...
So, let's tell me how to get one of them as binary package - otherwise you should write an apology ;) outpaddling wrote some really good hints if it comes to mixing ports and packages. And not an opinion.
Click to expand...
Well, I just had to do
doas poudriere bulk -j 122amd64 -O overlay -p HEAD net/bwi-firmware-kmod
Push it to my remote pkg repository
And then I can found it without issue:
pkg search bwi-firmware-kmod
Which output
Code: 
bwi-firmware-kmod-3.130.20     Broadcom AirForce IEEE 802.11 Firmware Kernel Module
 
So 17 out of 44 thousand means that mixing ports is inevitable? I'd never even heard of most of those, and I've certainly never needed them. If I did, I'd either run a system with just ports or make my own packages.

Saying that mixing ports and packages is "inevitable" is so wrong, it's just plain stupid. It's also bad advice to give.
 
Jose said:
So 17 out of 44 thousand means that mixing ports is inevitable? I'd never even heard of most of those, and I've certainly never needed them. If I did, I'd either run a system with just ports or make my own packages.

Saying that mixing ports and packages is "inevitable" is so wrong, it's just plain stupid. It's also bad advice to give.
Click to expand...
One of a million will turn true into false.

Of course you can use FreeBSD without mixing ports and packages (you can even use it without any port or package). And of course you can set up your own package server and compile all by yourself. But what if you want to go with the official packages, but f.e. need MariaDB as well as LibreOffice and Kdenlive? You need to compile some packages by yourself. And that's not a problem. Done that many years.

Simple example: You don't want the subshell support of Midnight Commander. Where's the problem if you're compiling only that port? None other port depends on it, so nothing can beak. The day MC stops to work will come (but: never came to it), but then you just need to recompile this port. "Quarterly package update" means "quarterly ports update of a few own packages". Much more less complicated, far less tripping hazards, and much more time- and resource-saving than your solution with poudriere. Using poudriere therefore would be "plain stupid" and a "bad advice".

You found your way, and it fits perfect. And you're sharing your experience here with others - and that's really great! But it's not the holy grail for everyone; It's just your recommendation.
 
jmos said:
One of a million will turn true into false.
Click to expand...
en.wikipedia.org

False dilemma - Wikipedia

en.wikipedia.org en.wikipedia.org
jmos said:
Of course you can use FreeBSD without mixing ports and packages (you can even use it without any port or package). And of course you can set up your own package server and compile all by yourself. But what if you want to go with the official packages, but f.e. need MariaDB as well as LibreOffice and Kdenlive?
Click to expand...
Use ports and not packages. This is not complicated.
jmos said:
(Snip dead horse flogging)
You found your way, and it fits perfect. And you're sharing your experience here with others - and that's really great! But it's not the holy grail for everyone; It's just your recommendation.
Click to expand...
No. Not mixing ports and packages is not just "my recommendation." It's the recommendation of most knowledgeable Freebsd users.
 
I compile ports in the background. I've set the number of parallel build so I don't even notice (lag) . Then its even ok if compilation would take weeks.
 
You must log in or register to reply here.
Back
Top