Teach me package management

[drhowarddrfine]

if I modify package settings I would probably want package to stay that way, not to be overwritten with newer/same version stock package.

You can't modify packages. You can only change settings if you use ports. Now I'm confused by what you're asking. You don't use pkg upgrade to upgrade ports.

 

[SirDice]

When you customize 'make config' and change options that port is definitely not the same as stock pre-compiled port.

Yes, but it's the same package you end up with. Although the package that was created from that port will have different options set. This is the main reason why mixing ports and packages is generally a bad idea. To take the example of nvidia-driver, it's still the nvidia-driver package that going to be installed, regardless of the options you enabled/disabled.

if I modify package settings I would probably want package to stay that way, not to be overwritten with newer/same version stock package. Because settings definitely differ, right?

You can't change package settings, once a package is built its options and dependencies are 'set in stone'. So for the nvidia-driver all it sees is a nvidia-driver package that's installed and will therefor try to "upgrade" that package regardless if you built it from ports or not.

 

[dpx]

It said 'package settings'. Not 'packages.' Meaning 'make config' then nice dialog pops up, then I pick options I need. Then 'make install'. Settings.

Package config modified this way definitely differs from stock binary package, right?

Please read the thread. I don't use pkg upgrade to upgrade ports but that's what pkg upgrade insists on.

 

[SirDice]

It said 'package settings'. Not 'packages.' Meaning 'make config' then nice dialog pops up, then I pick options I need. Then 'make install'. Settings.

Those are port settings. Package settings/options cannot be changed.

I don't use pkg upgrade to upgrade ports but that's what pkg upgrade insists on.

Yes, and I'm trying to tell you there is no difference between a port and a package once they are installed, from the system's point of view they are all packages.

 

[dpx]

SirDice, are you a programmer in any form? It seems this is semantics discussion while it should be very obvious that my vim built from ports (with python and without gtk2/gtk3) differs as a result from stock package. They only share name and version but pkg query very well knows it is installed from 'unknown-repository'. So there are no logical/code obstacles to add switch to pkg that handles both variants well.

Reason I keep asking the same question is that fixing it would take five minutes, it offers more flexibility and it doesn't break anything. I am bit puzzled that simple logical stream of deduction can't get trough but I don't mind asking again if we can have better OS as a result.

 

[dpx]

Those are port settings. Package settings/options cannot be changed.

Ah semantics. Ok. So customizing port setting makes resulting package (since that's how pkg sees it, right?) different than what stock package offers.

Yes, and I'm trying to tell you there is no difference between a port and a package once they are installed, from the system's point of view they are all packages.

That's plain not true. Their functionality and their repository are different. Even pkg knows this. I wonder if pkg@ guys can make some sense into this. I have sent email.

 

[SirDice]

Reason I keep asking the same question is that fixing it would take five minutes, it offers more flexibility and it doesn't break anything.

In that case I suggest you submit patches for pkg(8): https://github.com/freebsd/pkg

I really do understand what you are trying to say but the functionality simply isn't there.

 

[tobik@]

Why does pkg touch something that's clearly not installed by
pkg?

Ok, let me first repeat what others have said in this thread already:
ports are installed by pkg(8). It's the underlying mechanism to
install them and installed ports are recorded into the package
database. There is no conceptual distinction between a package
installed from ports or a package installed from the FreeBSD package
repository.

Also unknown-repository should be a pretty good indication for pkg not to touch that package?

It would say that changing pkg's behavior on the basis of the
repository name (or lack thereof) of the currently installed package(!) would be
insane and a POLA violation.

So in order to pkg upgrade sudo, sqlite3, firefox and curl I
have to upgrade nvidia which was clearly not part of the pkg based
installation?

pkg(8) will do its best to upgrade your system with the
available information it has. It handles package upgrades on the
basis of the configured package repositories. Since you clearly have
only setup the FreeBSD repository (or are simply using the defaults)
it will only use it to do so. A package's options can change in the
repository and pkg(8) naturally wants to follow that change
and wants to reinstall the package.

I bet the solution to your problem would be to setup your own package
repository and point pkg(8) to it with a higher priority than the FreeBSD one.

make package will create a package of a port and put it
into /usr/ports/packages (only if the directory exists,
so create it first). pkg repo /usr/port/packages will then
create a repository from them. And to make pkg aware of it, create
/usr/local/etc/pkg/repos/local.conf with

local: {
	url: "file:///usr/ports/packages",
	priority: 10,
	enabled: yes
}

Now reinstall the desired packages from the local repository to make
them stick to it during upgrades (see pkg-repository(8)):
pkg install -f -r local nvidia-driver-340 vim

It's likely that you will end up in dependency hell with this at some
point though if you don't take care to keep the local package versions
in sync with the FreeBSD repository.

 

[jb_fvwm2]

I concur with dpx's questions, and am facing a 'do not upgrade ANYTHING' without triple workaround hassles, [ though my fault for using CURRENT and too many ports... ] and terse messages from pkg as to why operations on a subset of packages want an all-or-nothing approach.
From years of back and forth, I think it is a lack of coders, as well as not too many persons in the same boat as dpx and I.
Thankfully, this is FreeBSD where it should all work out in the end.
/edit/
As I typed this post, the well-thought out guide right above it appeared. If I had read it before posting this, I might be well on my way to such an implementation.
/end edit/

 

[dpx]

It would say that changing pkg's behavior on the basis of the
repository name of the currently installed package(!) would be
insane and a POLA violation.

Thanks tobik@. I was about to clone pkg and send small PR that would make pkg much more user friendly without sacrificing any security. Are you saying that's not acceptable in any form? I have no intention to keep local patched pkg version only for myself so if it can be accepted OS wide I won't do it.

Here is what I had in mind:

• Keep pkg and pkg upgrade working as is
• Add switch -d <repo name> which means "don't touch packages installed from this repository". Maybe allow multiple -d too.

So for everybody not interested they can keep using pkg upgrade as usual, nothing is changed. For those like me who want 602 binary packages and only two or so built from ports, I could use 'pkg upgrade -d unknown-repository' and have all the functionality I need.

If it is futile effort and can't be accepted please let me know.

...

That's helluva path to follow, even for me. Not in terms of if I know how to do it but in terms of time/effort needed. I understand the logic but man the overhead.

So if it is official policy that proposed change can't be accepted please let me know.

 

[tobik@]

If it is futile effort and can't be accepted please let me know.

No, by all means submit it. I do not speak for the pkg developers. The post is my own personal opinion only.

That's helluva path to follow, even for me. Not in terms of if I know how to do it but in terms of time/effort needed. I understand the logic but man the overhead.

Hmm, it takes like 10 minutes to setup

 

[dpx]

No, by all means submit it. I do not speak for the pkg developers. The post is my own personal opinion only.

Thanks will do. It should be small patch anyway so if I intend to stay here better test the waters with something tiny. I'll have PR in a couple of days.

Hmm, it takes like 10 minutes to setup

I know but I need to setup repo then keep it in sync for two small packages. Even if there is ten of them it is easier to 'make deinstall/reinstall' when something breaks then to keep repo in shape. I understand the need for bigger more-than-one-machine setups.

 

[SirDice]

I know but I need to setup repo then keep it in sync for two small packages. Even if there is ten of them it is easier to 'make deinstall/reinstall' when something breaks then to keep repo in shape.

If you have a good repository there would be no need to rebuild something from ports (because there's a good package in your repository). Build once, install many

 

[dpx]

I bet the solution to your problem would be to setup your own package
repository and point pkg(8) to it with a higher priority than the FreeBSD one.

Actually, more that I think about it, this is starting to make sense. I'll experiment with local repo first then see if there is any need to modify pkg.

 

[SirDice]

I would actually recommend disabling the FreeBSD repository completely and only use your own. If your repository is missing something you'll find out quickly enough and mixing custom packages (with custom options) with the official packages (which all use the default settings) is bound to cause conflicts. In my view it's better to have something crash and burn because your repository is missing a package than being completely oblivious and end up with mixed options/defaults and a bunch of weird conflicts.

 

[rufwoof]

Thanks tobik@. I was about to clone pkg and send small PR that would make pkg much more user friendly without sacrificing any security. Are you saying that's not acceptable in any form? I have no intention to keep local patched pkg version only for myself so if it can be accepted OS wide I won't do it.

https://www.freebsd.org/doc/en/articles/linux-users/startup.html

Since FreeBSD is developed as a complete operating system, user-installed applications are not considered to be part of the “base” system. User-installed applications are generally installed using Packages or Ports.

https://www.freebsd.org/doc/en_US.ISO8859-1/books/porters-handbook/why-port.html

In FreeBSD, anyone may submit a new port, or volunteer to maintain an existing unmaintained port. No special commit privilege is needed.

 

[dpx]

I would actually recommend disabling the FreeBSD repository completely and only use your own. If your repository is missing something you'll find out quickly enough and mixing custom packages (with custom options) with the official packages (which all use the default settings) is bound to cause conflicts. In my view it's better to have something crash and burn because your repository is missing a package than being completely oblivious and end up with mixed options/defaults and a bunch of weird conflicts.

I am trying to avoid that, especially since I have such disproportion between things I have to customize and those that work for me as stock builds. As long as I need up to ten-ish compiled ports I can live with occasional breakdowns. Everything is not process critical, including nvidia drivers.

I would avoid compiling _everything_ at all cost, that simply doesn't look appealing to me. I have run away from systemd mess on linux, even if there is Gentoo distro that offers not-systemd (OpenRC) init. It would work for me but Gentoo compiles everything from sources - big no for me. Otherwise, if we ignore compiling, it is one of the few systemd-free distros left.

FreeBSD (OS) has better performance than linux on my computer so I'd love to stay around. I have set local repo, for vim and nvidia only, and so far so good, we'll see how it behaves when new updates come.

Since I am using mutt-offlineimap-vdirsyncer-khal-rxvt it is all very lightweight and easy to set, if I ever need to change OS again. I hope I won't have to do that any time soon.

Please don't make me compile everything, I'll miss ZFS badly

 

[drhowarddrfine]

dpx The typical reason for compiling from ports is one needs some feature that isn't in the defaults or won't make the package for some time down the road. In my case, for example, I needed a feature in nginx that was only available in ports with different flags set (SPDY then HTTP2 if your familiar). Many, maybe most, don't need to do that.

 

[tankist02]

dpx Have you looked at Synth to create a local repo and its option to use precompiled packages to reduce recompilations? I read this thread with great interest - I'd also love to have a few custom ports and the rest as precompiled packages.

 

[SirDice]

I would avoid compiling _everything_ at all cost, that simply doesn't look appealing to me.

I have set up poudriere for a client. I'm not building everything (by everything I mean all 26000+ ports), I'm only passing a list of ports/packages I need. Sure, it needs to build 200-300 packages (due to all sorts of dependencies) but it only builds them all from scratch the first time. Most of the time the builds are incremental and really only rebuild things that require it. It does a fairly good job of it. Sometimes there's something odd going on and I just force it to rebuild the whole list from scratch.

In the end my repository only has the stuff we need/use. So there's no Gnome or KDE for example in my repository because we have no need for it on our servers.

 

[dpx]

dpx Have you looked at Synth to create a local repo and its option to use precompiled packages to reduce recompilations? I read this thread with great interest - I'd also love to have a few custom ports and the rest as precompiled packages.

Not yet but it looks interesting, thanks.

I am using local repo as tobik@ suggested and it works in terms of pkg not bugging me to install stock packages with each pkg update. I am expecting to have to recompile both packages when version bumps up but for only two packages so far that's not a big deal.