Do not use ports-mgmt/portmaster and other tools who build in the main system

Is this right?

  • Yes

    Votes: 18 27.7%
  • No

    Votes: 39 60.0%
  • Don't know

    Votes: 8 12.3%

  • Total voters
    65
What a load of nonsense... I can't be bothered to read the whole thread but unfortunately I voted 'No' because of my opinion, and then I realized that this post is so poor that it doesn't even manage to clearly explain what I've been voting on. Am I saying "No, this post is not right" or is the OP thinking that I'm voting in agreement with whatever he preaches to be not right?

So to avoid any confusion: my "No" vote is directed against the main post itself, that's the part which I think is not right here.

There's nothing wrong with ports-mgmt/portmaster and I dare say so because I've been using it for even longer than the main post exists. I started using it ever since I discovered it somewhere around FreeBSD 10 and I've been using it right up to this point. Even during the time when it became unmaintained for a while and it has never failed me.

And I've heard plenty of people share their frustrations with Portmaster over the years and all fair enough because they shared their opinion. But to start boasting how people should not be using it because it's bad is just ridiculous and something I pick up as utter slander, which I think it's really a poor display. Like I always say: only tools blame their tools.


In fact... I dare argue * that it's actually a lot worse to rely on tools like Synth and/or Poudriere. Not because I think they're bad, nonsense, but because of what they do.. See, both environments obfuscate the actual building process. Meaning? If you know how the ports collection works, if you're familiar with things like /etc/make.conf and the several build targets as explained in ports(5) then all that experience will do you little good when working with either environment.

Because you're no longer working with the ports collection anymore, nah, you're now working with the building environment which will be working with the ports collection for you. You don't have to know anymore how to use build targets, nah, you need to know how to use your build environment instead.

Eventually resulting in some users who will become so alienated from the actual process that they'll start to believe that whatever those building environments do is right and everything else to be wrong. How unfortunately however that "everything else" involves the actual FreeBSD build process for the Ports collection, a process that has been in use long before those build environments even existed.

Which is actually one of the reasons I personally dislike both Synth and Poudriere and I don't think they really add much useful to the whole process. But that's only my opinion, you will never see me start a rant against their usage. Simply because while I may dislike both programs I still respect the projects themselves.

In fact.. the following argument applies to all three, so including Portmaster: all build environments get the job done. So who am I to start whining about how using Poudriere or Synth could be bad?

TL;DR?

I think the OP is ridiculous, been using Portmaster for almost as long as I've been using FreeBSD itself and I never had issues with it. From setting up servers to KDE powered workstations, things always worked out for me. IMO only tools blame their tools.

</vent>

* (edit): Not really, I definitely can't be bothered to waste more time on this thread, let along argue about a pointless topic as this. Just wanted to clear up my vote which ended up sharing my dislike for the thread.
 
Well, I didn't bother to read this whole posting. I agree there's quite something wrong with this thread, mixing in unrelated stuff. I disagree there's "nothing wrong" with portmaster: Not building in a clean and defined environment always bears the (small) risk of strange effects. Other tools solve this problem. But that's probably all about it.

edit: Of course, you could approach it the other way around. Testing your ports ONLY in e.g. poudriere, you easily miss even the most obvious CONFLICTS_BUILD settings. But then, there's no sane way to identify ALL these conflicts. Building in a clean environment at least makes sure you never forget any dependencies. Given how easily poudriere can provide clean build environments with ZFS and jails, I see no reason not to use it, although of course, it should always be possible to build ports on your running system. Just saying it's NOT possible to ensure that for every weird edge case.
 
Sorry, I obviously failed to make understandable what I want.
Please apologize my difficulties in explaining clearly.

What I want is to build on the respective local computer.
No external build server and all that.

I do not care which build system to be used.
I just need to be able to inject patches where I want.
(possibly using an unionfs for keeping the patches alive through updates, like Mjölnir suggested).

Ideally I would like some tool that can work the way people are used from pkg.

a)
You want to try out some stuff.
You run pkg install <somestuff>.
It installs what is needed.
If version conflicts arise, it updates the conflicting stuff and all other packages that depend on this updated stuff (and only these).

b)
When you decide you do not want that particular <somestuff> anymore, you can run pkg delete <somestuff>.
bb)
After that, you can run pkg autoremove to remove dependencies that are not no longer needed. This is a must to avoid garbage piling up over time.

So, I need only three functionalities:
"portinstall" : works like cd /usr/ports/<somestuff>; make fetch; make patch; make config_recursive + make install
(case a)
"portremove" : works like pkg delete <somestuff> + pkg autoremove (case b+bb)
"portupdate" : updates what is installed, like pkg update + pkg upgrade

But, I do not know a solution for bb) when using ports' make system.
How can I keep the system clean of no longer needed dependencies?


Having to read and actually understand all the documentation for portmaster, poudriere and synth, only to figure out whether and how I could achieve that task with that particular tool, would take a lot of time.
This is why I am asking.
 
But, I do not know a solution for bb) when using ports' make system.
How can I keep the system clean of no longer needed dependencies?
I don't have an immediate answer, but using poudriere WILL solve this, cause you will actually install with pkg, so pkg autoremove will work as expected on your own package repository.
 
It was bad enough that the very first paragraph in the fine Tutorial you linked to was something someone with experience using it can work out from the Errors shown and information given what happened to stop the build as a normal matter of course, work though it manually then continue the build with portmaster:

"Building of ports as outlined in ports(7) is performed on the live system. Many build failures stem from this when an already installed port or leftover files from an old version causes conflicts with the to be built new version of the port."

I didn't bother with "the well-known Recursive Make Considered Harmful" link provided when I looked at this the other day. But I followed the link to the article written in 1997 today and it directs to real-linux.org.uk.

"Please consult the well-known Recursive Make Considered Harmful for reasons why such tools do not handle the build process theoretically sound -- they do not use make(1) correctly to compute the graph of dependencies."

Are you kidding me? You're sending FreeBSD people to get advise from an article posted on a Linux site on how to compile ports? The same link shows "the well-known Recursive Make Considered Harmful" article appears on microsoft.com and gnu.org as well.

EOL
 
I didn't bother with "the well-known Recursive Make Considered Harmful" link provided when I looked at this the other day. But I followed the link to the article written in 1997 today and it directs to real-linux.org.uk.
I think this article does have its merits, and I still tend to structure my own projects to avoid recursive make. But that's what I meant with unrelated stuff. It has nothing to do with building ports. Recursive make is an issue inside a single project, and recursive make can work just fine, but it has its pitfalls. So much about that.
 
I don't have an immediate answer, but using poudriere WILL solve this, cause you will actually install with pkg, so pkg autoremove will work as expected on your own package repository.
Yes, but wouldn't mean using poudriere just moving the garbage pile?
I would prefer a solution without garbage pile.
 
Trihexagonal, cool down... :) that paper is very popular, you'll find it on numerous sites all over the net and AFAIK the author has had no connection to Linux, M$, GNU or whatever, but IIRC he was working at a university? You find it at Linux/GNU/M$/etc. because they all use make(1) or variants thereof to build software. That's what make(1) is for: solve dependencies. That's all about that so far. Use your intellect, man, instead of jumping on Linux/M$/GNU like a bull on a red flag. Read the paper so you can judge yourself...
 
Mjölnir cool down as well please 😈 😁
Yes, considerations about recursive make and the pitfalls (there be dragons!) are not Linux-specific. But they're not related to the ports tree either, or to a specific tool for building ports. The "naked" ports tree of course uses recursive make, cause there is simply no other option: It's a – let's call it that – "meta buildsystem" meant to build many individual upstream software packages. There is just no way (using make) NOT to make that recursive.

edit: no objections about the paper in general, as I said before, I think it's indeed a good idea to structure your build system for your own project without using recursion. Immediate benefit is dependencies DO work reliably without special considerations. Recursive approaches always need extra work for that.
 
Trihexagonal, can you confirm that I linked a DuckDuckGo query? Yes or No? "Please press here to change the weather". Do you think I moved up M$ & Linux & GNU in thre search results intentionally to cheat you?
 
For the packages on your system, pkg has you covered with subcommands autoremove and clean. poudriere itself also offers housekeeping, have a look at subcommands distclean, pkgclean and logclean.
Great!
Does there already exist any ready-to-use script package which takes care of calling poudriere first, and when it finished, calls pkg?
(This script package would then have to keep track what "master" packages poudriere actually needs to make, by editing the package list file which is given to poudriere using the -f flag)

What I want is sort of seamless integration of poudriere.
So the user (and the scripts he uses) can just use pkg as he is used to, without having to deal with poudriere, after he got poudriere set up?
(I guess this script would have to be put into the path before pkg, so it gets called in place of pkg, and forward/pass to actual
/usr/sbin/pkg after it did the respective poudriere actions)
 
Sorry Snurg never looked into that direction cause I have a "builder" machine building packages for a whole park of other machines ;) But seriously, something like this should be pretty simple to script yourself, although triggering a build from issuing a pkg command might not exactly follow the principle of least astonishment :D
 
Great!
Does there already exist any ready-to-use script package which takes care of calling poudriere first, and when it finished, calls pkg?
I don't think so. Here's what I do at a high level. I can get into details if you like.

* I keep a branch off the quarterly ports branch in Git that I use to build packages from.
* Single beefy build machine build two separate port sets, one for itself and one for a significantly less beefy server.
* I rsync the ports to the server. I have my private repository on both machines set to a higher priority than the Freebsd quarterly repo. This allows me to install a mix of locally-built and standard packages.

You can build a single port using poudriere-testport(8) but this won't package it, IIRC. The poudriere-bulk(8) command should not clean any ports if you don't give it the -c flag.

Edit: Thanks for the correction Mefizto!
 
Last edited:
There is poudriere-status(1) & you can RTSL. I'm sure you can hook in when a port & all it's dependencies are built. If you don't find the right script/routine to hook into, you can write the author a kind e-mail after streamlining your Q to the bare minimum. Another method: you can poll for a package to appear in the repository & check it's checksum (don't try to install until the pkg is fully written...). We have filemon(4) (manpage incomplete: it needs to be loaded from loader.conf(5), not via kld_list), so you can detect that the contents of some directory has changed? This is all brainstorming, but IMHO it shows your issue is solvable with the existing toolboxes.
 
Trihexagonal, can you confirm that I linked a DuckDuckGo query? Yes or No?
Are you questioning my Honesty or ability to click on a link and copy and paste a URL?




"Please press here to change the weather". Do you think I moved up M$ & Linux & GNU in thre search results intentionally to cheat you?

Cheat me out of what? Are you watching the Weather Channel and multitasking at your age? 🥛🍪🛌

My site isn't even online anymore and my name appeared on 3 of the 5 top ranked listings in a Google search for FreeBSD Desktop Tutorial. It still ranks the #1 spot posted here and the article about it in freebsdnews.com is #9.

Microsoft, Linux and GNU aren't even in competition with me for ranking.
 
RTSL. I'm sure you can hook in when a port & all it's dependencies are built.
Yes, there is the question how to find out when poudriere has finished, and whether that was successful.
I guess that would be difficult and awkward but manageable. But I would like to avoid such kind of hackery.

But, reading the synth intro thoroughly, I get the impression that using synth might have some advantages for desktop users:
  • a) there seems no need to figure out how to check when it is safe to run pkg,
  • b) synth can combine package build and install by just synth install <somestuff>
  • c) when installing stuff via a web interface (postinstaller), the user could watch the build process via Synth's web interface in an iframe and actually see what is happening. This would be MUCH better than any kind of "progress bar".
  • d) synth is much faster than poudriere
  • e) and would be usable on Dragonfly BSD, too.

But the best is that synth seems to be designed to make possible to only build the user-modified packages, and to pass through/use the normal FreeBSD repos for all other packages.
This would be an extremely lean solution - storage space and compilation overhead reduced to an absolute minimum, just what actually needed.

Thank you very much for the discussion on the various approaches and their pros and cons.
I get more and more the feeling that for me (as a complete retard when it comes to system administration), it might be best to try Synth first, and then see whether I need poudriere at all.
 
Yes, there is the question how to find out when poudriere has finished, and whether that was successful.
I guess that would be difficult and awkward but manageable. But I would like to avoid such kind of hackery.

But, reading the synth intro thoroughly, I get the impression that using synth might have some advantages for desktop users:
  • a) there seems no need to figure out how to check when it is safe to run pkg,
  • b) synth can combine package build and install by just synth install <somestuff>
  • c) when installing stuff via a web interface (postinstaller), the user could watch the build process via Synth's web interface in an iframe and actually see what is happening. This would be MUCH better than any kind of "progress bar".
  • d) synth is much faster than poudriere
  • e) and would be usable on Dragonfly BSD, too.

But the best is that synth seems to be designed to make possible to only build the user-modified packages, and to pass through/use the normal FreeBSD repos for all other packages.
This would be an extremely lean solution - storage space and compilation overhead reduced to an absolute minimum, just what actually needed.

Thank you very much for the discussion on the various approaches and their pros and cons.
I get more and more the feeling that for me (as a complete retard when it comes to system administration), it might be best to try Synth first, and then see whether I need poudriere at all.
I want to make an amendment to your very useful comment. DragonflyBSD still uses Poudriere to build DPorts, though. The information below is controversial, as I can't recall the source where I read it. Don't take it for granted!

The DragonflyBSD people seemed to want Synth but don't want the Ada requirements, so they seemed to rewrite Synth in plain C. I don't know the current status of this effort, or it was even started at all.
 
Snurg, having read minimally on the subject of synth I'll address what you see as its possible advantages from the vantage of a ports user.

a) there seems no need to figure out how to check when it is safe to run pkg
It's always safe to compile ports and I don't regularly update all my programs as maintenance, only when the need arises from a vulnerability or requirement for another port.

b) synth can combine package build and install by just synth install <somestuff>
That's the job of Portmaster and we do it from source.

c) when installing stuff via a web interface (postinstaller), the user could watch the build process via Synth's web interface in an iframe and actually see what is happening. This would be MUCH better than any kind of "progress bar".
So is the login terminal, at a glace from my recliner to see if text is still scrolling or stopped, and if that means done or I have to step in abd fix things before starting the build with the same command and it will pick up from there to finish the job.

d) synth is much faster than poudriere
You have me beat there beyond a doubt. It takes a long time to compile some ports. Nap time.

e) is a moot point.

I would say you're a long way from "retard" status when it comes to FreeBSD Admin, Snurg.
 
  • a) there seems no need to figure out how to check when it is safe to run pkg,
  • b) synth can combine package build and install by just synth install <somestuff>
  • c) when installing stuff via a web interface (postinstaller), the user could watch the build process via Synth's web interface in an iframe and actually see what is happening. This would be MUCH better than any kind of "progress bar".
  • d) synth is much faster than poudriere
  • e) and would be usable on Dragonfly BSD, too.
Uhm, I don't see anything really convincing here.
  • a) there's no need to ever figure that out. poudriere doesn't touch the old state of the repository until it is completely finished.
  • b) So, poudriere bulk [...] && pkg install [...] will achieve this as well.
  • c) Not sure what the idea behind the iframe is, but poudriere has a web interface as well (you only have to host it, e.g. with nginx).
  • d) Do you have figures? Nothing can magically speed up the compilation itself, so differences could only be in preparation/cleanup steps, globally and per package. poudriere can be flexibly configured to use your available CPU cores as good as possible.
  • e) as would poudriere ... uhm ...
Doesn't mean you can't just like synth better for some personal preference of course ;)
 
Zirias, I am not sure you understand the problem.

Imagine a desktop with only a few packages locally modified.

The user wants to save time and storage.
So he wants to use the normal FreeBSD packages.
Technically it is only necessary to build those packages that are different, e.g. patched or non-default build options.

So the user needs a build manager which only builds the packages that actually need to be built, and builds them just-in-time on-demand, and fetches all other packages directly from the FreeBSD repo.

What the user (who has no personal preference yet) has read and been told, makes the impression that this is difficult-to-impossible to achieve with poudriere, but very easy with synth.
Is the users' impression correct or not?
 
Well, if THAT's what you mean with faster, then yes. But this is a bit misleading cause this is about fetching packages built somewhere else. Still, poudriere only builds what is necessary (and reuses packages once they are built). Yes, if options is all you want to change, this can of course speed things up.
 
I want to make an amendment to your very useful comment. DragonflyBSD still uses Poudriere to build DPorts, though. The information below is controversial, as I can't recall the source where I read it. Don't take it for granted!

The DragonflyBSD people seemed to want Synth but don't want the Ada requirements, so they seemed to rewrite Synth in plain C. I don't know the current status of this effort, or it was even started at all.
They did. It’s in the DragonFly base system since 5.7, see the dsynth(1) manual page. Apparently they have made an effort to make the software as portable as possible, so it should work on FreeBSD, too, but I haven’t tried this.

Personally I like Synth much better than Poudriere, for various reasons. But that might be purely a matter of taste. YMMV. If you like Poudriere, then use Poudriere. If you like Synth, then use Synth.
 
Back
Top