Do not use ports-mgmt/portmaster and other tools who build in the main system

Is this right?

  • Yes

    Votes: 12 26.7%
  • No

    Votes: 27 60.0%
  • Don't know

    Votes: 6 13.3%

  • Total voters
    45

Sevendogsbsd

Aspiring Daemon

Reaction score: 531
Messages: 982

I have never had an issue with synth. The times I used it, it worked very well. My use case was ports on a desktop so perhaps not everyone’s situation.
 

Lamia

Aspiring Daemon

Reaction score: 163
Messages: 631

Poudriere seamlessly integrates with Buildbot. That's a big plus.
 

SKull

Member

Reaction score: 57
Messages: 96

Take a look at the source code for synth -- I've seldom seen such clean and beautiful code. Even if developers do prefer to build with the poudriere script (and use jails instead of null mounts to allow building malicious/bad-actor ports) every programmer could benefit from learning a little bit from synth's Ada code. It is really sweet. And I have to add that I've seldom seen any package receive such good support as synth does here. Mention any problem with synth and the author will be show up as if by magic to help...

(Synth is the most straight-forward replacement for portmaster:
Code:
synth configure
synth upgrade-system
synth install <port>
Development here.
That ist exactly why I pressure my teammates to write clean code.
Because bugfixing and new featured are a piece of cake when the codebase is nice and clean.
 

wisdown

Active Member

Reaction score: 13
Messages: 131

For Years, I have used portmaster to compile all my programs on FreeBSD.

I think the point is:

Do you need custom options?

If yes, then you will need compile the stuff
, and like handbook advices, do not mix the things (with some exceptions) for compiling stuff way do it for all.

If not, then use pkg.

On this exactly moment I am running an test Virtual Machine with poudriere for see if it will save some time for me.

But what's I mean for save some time?

Lets start from beginning...

Usually I use one Virtual Machine per service, so in a minimal network I will have:

1 DNS Server Forwarder
1 DNS Server Authoritative
1 NTP Server
1 Database Server
1 Web Server (inside running Apache on root, jailed node, jailed php, etc...)
1 Mail Server

So from this basic setup I have 6 Virtual Machines...

Working only on DNS servers the compiler time is not a real issue, bind compile fast because there no many dependencies...

But since it is 2 Servers, then I will spend the double of time for the same job (compiling one by one to save resources on server)...

What make me compile dns/bind916?
Usually enterprises looking for performance improvements only will have that with same hardware through tunings... dns/bind916 have this nice option:

Code:
LARGE_FILE
It's off by default, and the requirements to get an real performance boost is only use it when the system have at least 8 GB of RAM or more...

So, it's mean, you need know what you doing when enabling extra options or disabling something because security issues.

On this scenario, if poudriere can make an local customized repository to me instead use the default pkg from FreeBSD main server, will be a time save. (Read will compile only one time with my custom options, and the update will be deployed to both servers)

Maybe the real issues only occurs with more complex setups, like on the webserver...

Enabling graphic extra options on graphics/php74-gd will require you compile graphics/gd and others, and those will probably ends in the recursive problem (aka chicken egg) breaking the process.

Using portmaster on the webserver scenario, I have the workaround:

I can install the pkg version and compile one of the biggest dependencies...

Then I can remove one of pkg version downloaded before to fill the requirement, and now compile it from source...

And redo it till get as close of possible from entire compiled system...

For as close possible I mean there situations where only an outdate pkg will fix the issue...

From last time I have compiled graphics stuff, if I remember right something related to H264 or H265 was broken, the only way to get this dependency is using an outdate pkg version instead the ports versions...

How poudriere will work on previous scenario?

No idea, this is why I am testing now...

But the previous scenario using portmaster I was able to get the things up and running.

If poudriere can solve the chicken egg problem, so it's the awesome solution...
One big time only recompiling everything when need, and then all your servers can downloading it in the sequence locally.

If poudriere fails on my test, there no way instead back to portmaster and schedule the recompilations because updates...

How much time we spend doing the things per machine?
Have you tried compile devel/llvm80? Whats the time for do it in 6 machines one by one for example? (Remembering you can have more servers for example an dedicated redis server)

For last, I think there an extra reason to use compiled versions, like the option CPU extra stuff, on default packages that will be always off, compiling from source you will get extra boost from this option, one time your compiled version of the program will have extra functions to use features from your specic processor (this is why is off on generic pkg - read the program does not know what's the processor will enduser have)...

By the way, like many tests around the internet already have reported, llvm versions mostly of time run slowly if compared to gcc version, but this is not the discuss here (read any extra boost to better perfomance is an plus).
 

chrcol

Well-Known Member

Reaction score: 41
Messages: 436

Poudriere to me would make sense if you want one build machine to provide packages for many machines which are duplicates of each other, so by duplicates I mean same packages installed, same OS version, same port options and same cputype.

The argument you dont need to clutter your system with build dependencies doesnt stick with me, because they will still be on the system, they will just be in the jail directory instead of the base directories, and you actually have more clutter because there will be copies of the binary ports in the jail on top of on the base system as well.

The only other potential advantage is you avoid the potential problem of a half updated system breaking the live configuration, this one I recognise, but I am against the idea of making jails or building on a different machine to my system, so I will stick with what I use now which is primarily portupgrade. I will probably keep on using it until it stops working.
 

leebrown66

Well-Known Member

Reaction score: 152
Messages: 429

Poudriere to me would make sense if you want one build machine to provide packages for many machines which are duplicates of each other, so by duplicates I mean same packages installed, same OS version, same port options and same cputype.
I disagree.

I used to use poudriere to build repos for both 386 and amd64, but thankfully that old hardware is no longer alive.

I now use poudriere to build 3 different repos, each with different options, one optimised for my workstation (cpu tweaks, no docs, no examples), one for general puropose servers that have varying CPU's (no cpu tweaks, specific options set, docs, examples) and a third I use for experimental purposes. That's part of the point with poudriere, different repos can have different option sets and the repo is consistent within itself. You do get into trouble if trying to mix different repos with different options though.
 

Oclair

Member

Reaction score: 7
Messages: 32

I use Portmaster primarily, and very limited use of pkg. People who do not use Portmaster should refrain from giving advice regarding the use of Portmaster. How you update your ports is none of my business, portmaster works great for me, I am grateful dougb contributed it to the community! I am also grateful there are soo many useful tools to manage ports which facilitate automating updating several installs. I've used FreeBSD back in 2000 installing 4.2 from a windriver cd. I just appreciate all the massive talent and expertise who have all helped make our OS and community maintain such a great OS.
 

olli@

Aspiring Daemon
Developer

Reaction score: 845
Messages: 798

I’m also one of those who prefer to not use poudriere (nor synth). It has no advantages for my workflow and the way I use the ports.

If poudriere works fine for you, that’s great. But don’t force it on others.
 

Francisco Reyes

New Member

Reaction score: 4
Messages: 16

People who do not use Portmaster should refrain from giving advice regarding the use of Portmaster.
People who use these forums should actually try to be helpful and polite. Take the other thread for example.

Should I be using a different build tool to make this more bulletproof?
There was more noise from the portmaster purists that could not be bothered to actually properly read the post try to understand the actual use case or needs. It is possible, and we may never know due to all the noise, that some other tool may be what the user actually needed. Ultimately the goal should be "how do we help people seeking help to achieve what they are looking to do in FreeBSD". They may be using a tool and workflow that won't achieve what they need.

I am also grateful there are soo many useful tools to manage ports which facilitate automating updating several installs.
And yet mention of other tools besides portmaster when the requester actually asked if there were other tools seemed to be a problem.

I do not like the black&white attitude “poudriere is the only correct way to build ports,
Last I checked we are here to try and help each other in a constructive way. What someone "likes" is up to them to post on their posts. To hijack someone else's question with noise about what someone likes is far less helpful or constructive than offering an option to the original poster based on his own question of what other options are available.

I’m also one of those who prefer to not use poudriere (nor synth). It has no advantages for my workflow and the way I use the ports.
And that is how it should be. People should use what works for them for their needs. However, they should not assume that their preferred tool is what others need. So when a user asks "are there other options" the tool you use will not always be the right answer, but you can present it as information that then the user can look into and consider.
 

olli@

Aspiring Daemon
Developer

Reaction score: 845
Messages: 798

Last I checked we are here to try and help each other in a constructive way. What someone "likes" is up to them to post on their posts. To hijack someone else's question with noise about what someone likes is far less helpful or constructive than offering an option to the original poster based on his own question of what other options are available.[
Please stop creating confusion by quoting partial sentences from different threads. My remark that you quoted (partially) was in response to someone who hijacked that other thread in a non-constructive way, and my response was meant to actually support the original poster. You are misrepresenting the facts.

On the other hand, let me remind you of the topic of this thread: “Do not use ports-mgmt/portmaster and other tools who build in the main system”. This is a quite confrontational thesis. It’s no surprise that it provokes a debate like this.
 
OP
mjollnir

mjollnir

Daemon

Reaction score: 830
Messages: 1,267

On the other hand, let me remind you of the topic of this thread: “Do not use ports-mgmt/portmaster and other tools who build in the main system”. This is a quite confrontational thesis. It’s no surprise that it provokes a debate like this.
Yes, that was in fact my intention. My observation is that portmaster causes more bad than good, espc. for newbies. That's why I wanted a thread where the interested newbie can read about the pros & cons and take an informed decision.
 

richardtoohey2

Active Member

Reaction score: 125
Messages: 246

I'm getting dizzy. Think we are almost all agreeing:

1) For new users and maybe most users, pkg is the way to go. Fast, convenient, less risk of getting lost in dependancy hell or having issues.
2) Sometimes, though, ports are the only way to go. But there may be dragons here.
3) Mixing ports and packages is not generally speaking a good idea, but some people have success doing so.
4) We have differing views on how best to use ports and what tools to use. poudriere seems to be the one currently recommended (is it safe to say that?) by the FreeBSD developers. Others prefer (for all its flaws) portmaster and there are yet more options - portupgrade, synth, roll-your-own scripts, etc.
5) Every post about "how do I use portmaster to do this" seems to get replies of (a) "don't do that, it drinks kitten blood", (b) "don't, use pkg" (c) "don't, use synth, it's lovely", (d) "don't, use poudriere, it's the one true way" and the OP's original question gets lost in the expertise/opinions of the port-tool-x fans. Who then fire up the port-tool-y fans. And round we go.
 

Francisco Reyes

New Member

Reaction score: 4
Messages: 16

How about we work together and come up with pros and cons of tools and then put it all together in a post in the Howtos and moderated forum?
Ultimately, in my opinion, the best we can do is present facts and let people decide what works best for them.
 

mickey

Aspiring Daemon

Reaction score: 195
Messages: 591

Yes, that was in fact my intention. My observation is that portmaster causes more bad than good, espc. for newbies. That's why I wanted a thread where the interested newbie can read about the pros & cons and take an informed decision.
I have been using portmaster on multiple machines on a daily basis for years now and I can say with confidence that it has done me a lot of good, especially considering the nightmares I was having using portupgrade before that. The use of ports over packages is probably something for more advanced users, but that is not something caused by whatever tool you use to update your system from ports, this is by nature of ports. Anyone who decides to use ports instead of packages probably has a reason to do so, otherwise he/she would not even consider it worth the additional time and effort to do so.

But that aside.... don't you think a thread with such particular title leaves little room for any pros or informed decisions?
 
OP
mjollnir

mjollnir

Daemon

Reaction score: 830
Messages: 1,267

I have been using portmaster on multiple machines on a daily basis for years now and I can say with confidence that it has done me a lot of good, especially considering the nightmares I was having using portupgrade before that. The use of ports over packages is probably something for more advanced users, but that is not something caused by whatever tool you use to update your system from ports, this is by nature of ports. Anyone who decides to use ports instead of packages probably has a reason to do so, otherwise he/she would not even consider it worth the additional time and effort to do so.
To build (a non-trivial large number of) ports on the host requires that the admin is able to cope with issues that will eventually arise, i.e. mainly the so-called dependency hell:
[...] I have been using the ports tree (and portmaster/portupgrade) almost exclusively from the very beginnings and never had any major headaches I wasn't able to somehow solve by myself. When stuff breaks it is often due to software X version B using a header file or library that software X version A installed on the system, instead of using the correct version that comes bundled with the source. Of course a clean room environment would circumvent such problems, but that is hardly a solution to the problem in itself.
While I do not deny that the scripts & tools to assist building ports on the host work sometimes & save a few keystrokes, your post is one of a few pro-portmaster(8) posts that honestly tell about the fact that this method has inherent issues. Most others do not come up with arguments other than "Don't tell me how I have to administer my system" & "portmaster served me well for decades", while at the same time embezzling the issues they had to solve manually. Please note that I'm not talking about building, say, a dozen of ports, but e.g. a GUI and/or the version mismatch (latest ports vs. quarterly packages) you'll run into when building a few selected ports yourself on a system otherwise installed with packages from the official repository.
But that aside.... don't you think a thread with such particular title leaves little room for any pros or informed decisions?
No, because there are obviously very few reasonable pro-arguments to build (a non-trivial large number of) ports on the host unjailed.
 

olli@

Aspiring Daemon
Developer

Reaction score: 845
Messages: 798

To build (a non-trivial large number of) ports on the host requires that the admin is able to cope with issues that will eventually arise, i.e. mainly the so-called dependency hell:

While I do not deny that the scripts & tools to assist building ports on the host work sometimes
The word “sometimes” is a misrepresentation. Do you have any numbers to back that claim? I dare to say it works fine most of the time. I can back that with numbers from my workstation (see below), which is not representative, but not an uncommon kind of setup. Problems caused by ports accidentally using files installed by unrelated ports are a rare exception, not the rule.

Regarding your mention of “a non-trivial number” of ports: I have about 800 ports on my workstation; among them there are large applications with many dependencies (e.g. ffmpeg, mplayer, TeX, Gimp, Scribus, InkScape). I assume this counts as non-trivial. Of those 800 ports there are exactly 2 that have caused dependency problems in the past (one of them is InkScape). This is easy to workaround once you’re aware of it, for example by simply adjusting the order of ports while (re-)building. (My own update script supports the option to build in a jail, only for those few ports that need it. It would probably be a good idea if the portmaster author added such a feature, too. It’s actually not very difficult at all.)
 
OP
mjollnir

mjollnir

Daemon

Reaction score: 830
Messages: 1,267

The word “sometimes” is a misrepresentation. Do you have any numbers to back that claim? I dare to say it works fine most of the time.
When I was new to FreeBSD, I tried to build a KDE desktop plus some utilities (avg. workstation use-case) and failed, and I ran into the version mismatch between latest ports tree vs. quarterly packages when I tried to build selected ports with options different from the defaults. Both issues were not clearly covered by the handbook.
[...] This is easy to workaround once you’re aware of it, for example by simply adjusting the order of ports while (re-)building.
The correct build order should be figured out by make(1). While I've been critizising recursive make often, I realize that the ports(7) tree as a collection of 3rd party source trees is inherently restive to declare all inter-port dependencies correctly. Even if it's theoretically possible, there will always be human flaws since naturally the ports tree is a moving target.
(My own update script supports the option to build in a jail, only for those few ports that need it. It would probably be a good idea if the portmaster author added such a feature, too. It’s actually not very difficult at all.)
File in a bug report and flag it wishlist or enhancement or such? For shure an option to build chroot(8)'ed or jail(8)'ed would make perfect sense.
 

olli@

Aspiring Daemon
Developer

Reaction score: 845
Messages: 798

When I was new to FreeBSD, I tried to build a KDE desktop plus some utilities (avg. workstation use-case) and failed, and I ran into the version mismatch between latest ports tree vs. quarterly packages when I tried to build selected ports with options different from the defaults. Both issues were not clearly covered by the handbook.
I see. You are right, mixing latest ports with quarterly packages is a mistake that often leads to pain and suffering. I agree that the Handbook should contain a big fat warning not to do that. (Actually I thought it did, but apparently it doesn’t, unfortunately.)

The correct build order should be figured out by make(1).
Yes, when building a single port (plus its dependencies), that works very well. Otherwise tools like poudriere would have the same problem, because it also relies on the Makefiles of the ports collection.

But when you build two unrelated ports that don’t depend on each other, there is no order of them stored in the Makefiles. In theory the order should not matter. But if one of these ports has a bug causing it to use the wrong include file, for example, then the order of building does matter. In this case poudriere clearly wins because it builds the ports separate from each other, while portmaster doesn’t do that, because it builds everything on the host. Thankfully this happens rather rarely, but still it is annoying, and you have to be aware of it, so you know how to work around it.

File in a bug report and flag it wishlist or enhancement or such? For shure an option to build chroot(8)'ed or jail(8)'ed would make perfect sense.
It’s a good idea, but I’m not using portmaster myself, so I have little incentive to spend time with that. (I have to admit this is a selfish attitude, but the day has only 24 hours, so I have to set priorities.)
 

Zvoni

Active Member

Reaction score: 48
Messages: 113

chipping in with my unprofessional opinion ;)
Let's be honest with each other:
How many of the "new" FreeBSD-users, who come here to ask for help (and to stay in context: about this particular subject), are your "off-the-shelf"-user, who doesn't have an IT-background and/or longtime-experience with computers?
As in: My Grandma bought a new Laptop to finally get online, and now want's to use FreeBSD?
For that matter: How many of the existing Users here do have such a background?

Point for Honesty: Pretty much none (or a very small percentage).
Meaning: Users asking such questions, do have experience with computers and/or different operating systems.
Meaning: Your off-the-shelf-Winbloze-User is not your regular here, because he wants to try FreeBSD for the heck of it.

Speaking for myself: I came here because i did have my longtime-experience with Windows and Linux, and i had an opportunity to check out a "new"/"different" OS for the Servers of my Skydiving-club.

In that vein, i would expect experienced FreeBSD-users to advise me, "don't do that, because the script has bugs, that might launch a nuclear rocket in north-korea, if not handled properly".

Yes, i started out with pkg/binaries from quaterly, until i found out, that to have a feature i needed, i had to compile myself.
First contact with portstree, make config, make, make install
until i run afoul (as it's usual with no experience) with something, and had to research, until i found "make deinstall install".
Additionally, it annoyed the hell out of me, how to update/upgrade the ports.
Nevermind the amount of ports you would have to handle by hand.
So i found portsmaster (since it's mentioned in the handbook).
Yeah, went well, until i stumbled across the pkg/binary vs. "built-from-ports"-trap
Now, some months ago, i found that HowTo-Thread for poudriere (Note: There is no such HowTo for synth, or at least i couldn't find it).

Bottom-Line: I use poudriere, and never looked back (only forward, or until a "better" way presents itself)

As i said: I expect experienced users to tell me the "do's and don'ts", and if a program/script/whatever is flawed (even if it's just flawed in their opinion), i'd expect a warning and/or alternative.
Just because there are users, who don't have problems with it (and i'd bet, those users are at the upper end of the "expert"-scale), doesn't mean the others warning of it are wrong.
 

mickey

Aspiring Daemon

Reaction score: 195
Messages: 591

To build (a non-trivial large number of) ports on the host requires that the admin is able to cope with issues that will eventually arise, i.e. mainly the so-called dependency hell:
My workstation has 828 packages installed as of now, all built and updated (yes, using portmaster) exclusively from latest ports, usually on a daily basis. Yet I would not want to go as far as to call it a dependeny hell.

While I do not deny that the scripts & tools to assist building ports on the host work sometimes & save a few keystrokes, your post is one of a few pro-portmaster(8) posts that honestly tell about the fact that this method has inherent issues. Most others do not come up with arguments other than "Don't tell me how I have to administer my system" & "portmaster served me well for decades", while at the same time embezzling the issues they had to solve manually.
Updating my systems using portmaster has worked for me more than just sometimes and also saved more than just a few keystrokes. Back in the day when I started using 386BSD 0.1 I believe there wasn't even anything remotely like ports. The usual procedure to install some piece of software was to download the source .tar.Z file, extract it, read and follow the instructions in some README file (if there was one) and hope for the best. Failing that either fix the source manually or complain to the authors. So yes, having ports was kind of a quantum leap. Are there problems? Most likely, but none of the problems I encountered recently had anything to do with portmaster in particular, and fell into one of two categories:
  1. Ports that received an update that turns out to not be quite ready yet. Usually fixed by updating the ports tree again some time later and retrying. A more recent example I can think of was an update to libuv that broke name resolution in my entire network. Was fixed by another update couple hours later.
  2. Ports with a broken build system. One such port that immediately comes to my mind is www/qt5-webengine, which I have seen shoot itself in the foot more than just once, probably because it uses include files installed on the host by a previous version of itself, when in fact it should be using the include files from the source/working directory. Fact is, building qt5-webengine often fails if qt5-webengine is already installed, and again this is not related to portmaster as it fails in exactly the same way when you try building it manually.
Please note that I'm not talking about building, say, a dozen of ports, but e.g. a GUI and/or the version mismatch (latest ports vs. quarterly packages) you'll run into when building a few selected ports yourself on a system otherwise installed with packages from the official repository.
That mixing quarterly and latest is a bad idea is nothing new, but a proven fact, whether you build from ports or use packages. Again that has no relation to portmaster whatsoever. Maybe the handbook and/or other documentation could put more emphasis on this matter if it bothers you that much. Personally I never ran into this issue. As I said, when I started there was hardly any other option but build it yourself, and quite frankly, I wouldn't want it any other way.
 

chrbr

Aspiring Daemon

Reaction score: 306
Messages: 771

(My own update script supports the option to build in a jail, only for those few ports that need it. It would probably be a good idea if the portmaster author added such a feature, too. It’s actually not very difficult at all.)
Some weeks ago there have been issues with the speed on pkg and/or long unfixed vulnerabilities or so. I do not remember the details. I have looked for a work around. At that time I have tried to create a jail and letting portmaster make packages. Fortunately portmaster has the option to use packages for the build dependencies which saves time. Then I could install the packages from the new source if everything went well. After a few days the pkg servers have been back to normal and I did not follow up the method. I switched back to pkg because I can live with the default settings. But with building in a jail the host is not spoiled and nothing can be overwritten or deleted unintentionally (I think so).
 

getopt

Aspiring Daemon

Reaction score: 582
Messages: 840

this is not a question of agreement, but right and wrong ;)
There is no binary answer to this. Trying to force "Is this right?" patterns is prompting for reactance. You will be shown that there is a "depends on".

Polls are attracting attention. In return you get opinions.

A missing item among others in the poll is: "I do not care."
 

msplsh

Well-Known Member

Reaction score: 85
Messages: 283

mixing latest ports with quarterly packages is a mistake that often leads to pain and suffering. I agree that the Handbook should contain a big fat warning not to do that.
It would be nice to know how to not do this. Packages seem to default to quarterly (Which sometimes backports security updates?) and ports using portsnap (as the handbook suggests) defaults to... something very recent?

The hacked up way I've found to keep these in sync is to go to https://svnweb.freebsd.org/ports/branches/ and then figure out what the current quarterly SVN directory revision is, and then feed that into svnup ports -r instead of using portsnap.
 
Top