PkgBase

[grahamperrin@]

… how to safely update the system (regardless of how far out of date) reliably. …

Let's assume that PkgBase is the way forward.

<https://alpha.pkgbase.live/> and so on.

https://lists.freebsd.org/mailman/listinfo/freebsd-pkgbase

In addition to the list, there's sometimes discussion of PkgBase in IRC for FreeBSD.

Retrospective (2018–2019)

Solved - What is Pkgbase?

Could someone explain what is pkgbase, which problem it is going to solve, and (roughly) what will be its capabilities? I have read this page but unfortunately it is more like an implementation tracker than a wiki explaining the role of pkgbase. I feel like if you do not already know what...

forums.freebsd.org

Call for Testing (CFT) for FreeBSD + ZoL and Pkgbase

Kris Moore, FreeBSD contributor and spear-header of the ZFS on FreeBSD project, announced a Call for Testing of the latest development built on FreeBSD 12-STABLE. You can download the ISO from the link below. In other recent news, a TrueOS inspired package base for FreeBSD has also been uploaded...

forums.freebsd.org

pkgbase instructions?

These are the instructions for pkgbase? https://wiki.freebsd.org/PkgBase just relate it to 12.0 releng, instead of head. cd /usr/src make [make flags] buildworld make [make flags] buildkernel make packages /usr/local/etc/pkg/repos/FreeBSD-base.conf # FreeBSD base system repository...

forums.freebsd.org

 

[tux2bsd]

Can it tie in to a better transport method? (i.e. bittorrent (+extra validation))

freebsd-update is slow and then the mirrors (incl pkg) are also very slow in NZ.

When I downloaded the ~90G of all 13-RELEASE torrents to seed that came down at ~20MB/s (bottleneck being by my wifi).

Downloads freebsd-update/pkg generally maxes at ~300KB/s for download, but most often at ~120KB/s.

So a faster delivery and a better/faster update tool is win win. Keeping the current way for those that need that too, say with sysrc.

 

[grahamperrin@]

… freebsd-update is slow

This being the first weekend following announcement of 13.0-RELEASE, I should expect some slowness.

Why is freebsd-update so horrible slow?

and then the mirrors (incl pkg) are also very slow in NZ. …

pkg upgrade slow

Right now I'm in Kenya and having a really hard time getting a reliable download speed with pkg (only). I'm getting a pretty reliable 15Mbps right now and I can do a lot such as stream, download Ubuntu updates from the VM on my laptop, and everything else. I've tried manually setting geo...

forums.freebsd.org

 

[tux2bsd]

It's faster than Kenya so be happy? At any given time, not just major release, the mirrors are slow. Compared to available usable bandwidth (e.g. mirror: 120KBps vs usable: 20MBps), which is why thinking about the transport would be beneficial for NZ & Kenya & Everyone.

freebsd-update is slow, period. It is simply a very slow process regardless of when you perform your upgrade, or if it is a minor or major version.

 

[grahamperrin@]

You wrote "… mirrors (incl pkg) are also very slow …", I linked to a recent topic entitled "pkg upgrade slow".

It's faster than Kenya so be happy?

No, but slowness of pkg upgrade might be discussed in the pkg upgrade slow topic.

 

[grahamperrin@]

… (i.e. bittorrent (+extra validation)) …

Sorry, this got lost amongst my drafts:

<https://www.google.com/search?q=tor.../pipermail/freebsd-pkgbase/&tbs=li:1#unfucked> no mention of torrent.

I'll add the list address (and a little more) to the opening post.

 

[tux2bsd]

I didn't mean to overtake your conversation re pkgbase, my input was around "why not improve it holistically" and hit the other major factor, content delivery, as well.

I did watch this some months ago and thought it was "very soon" tm.
pkgbase: Are we there yet ???

 

[grahamperrin@]

Ah, thanks. I'm all for holistic. I might watch the 2019 video later today (I'm on holiday for a few days).

… thought it was "very soon" tm. …

It does work in that (for example) I used one of the base system package sets at <https://alpha.pkgbase.live/#package-sets> to perform an upgrade:

• from 13.0-RC5
• to 13.0-RELEASE

– in a disposable virtual machine. Upgrade succeeded.

Alpha

Re: <https://alpha.pkgbase.live/howto/bootstrap.html#pkgsave> if I understand correctly, it should be possible to (for example):

mv /etc/master.passwd.pkgsave /etc/master.passwd
mv /etc/group.pkgsave /etc/group
/usr/sbin/pwd_mkdb -p /etc/master.passwd

– however, this will be less than pleasing if your users are mysteriously missing from /etc/master.passwd.pkgsave. Keyword: clobber.

Part of my success – without the user-related clobber – probably involved straying from the <https://alpha.pkgbase.live/howto/bootstrap.html#setup> part of the routine. I should discuss the experiment in private before sharing details here.

Documentation

Omission of <https://wiki.freebsd.org/PkgBase> from the opening post was intentional. <https://wiki.freebsd.org/action/info/PkgBase?action=info> my comment "… This page will benefit for a non-trivial re-write to reflect ongoing work in and around the FreeBSD community." is a polite way of saying that the page is horribly outdated.

I'm an editor, but I'm not the person to pull the page into shape; I don't have the big picture …

 

[tux2bsd]

I'll give it a try at some point. I remember looking at this wiki page a while ago and thinking the project was stagnant but PkgBase is definitely an important optional improvement going forward and if proven over time could become default. It also makes sense that pkg users have OS updates be handled via packages but that doesn't mean the traditional way needs to go away either - both can be used to verify the same outcome occurred (else a bug exists somewhere).

Plus, to re-iterate better delivery could be added on too i.e. torrent / CDN etc

 

[grahamperrin@]

If I'm not mistaken, major upgrades to GhostBSD are succeeding with something like a PkgBase approach:

Upgrading to newer versions of GhostBSD base on 13-STABLE - GhostBSD

– a single repository with a set of packages that is broader than base alone …

 

[grahamperrin@]

In the quarterly status report: https://www.freebsd.org/status/report-2021-01-2021-03/#_pkgbase_live

 

[astyle]

Hey... I think the posters in this thread are not considering the fact that this very well may have to do with network bandwidth in Kenya and NZ (New Zealand, I assume?) you gotta pick a FreeBSD mirror that is closer by. I know that FreeBSD offers scripts to do IP resolving and do it for you, but they also provide a list of mirrors and what they host. How fast those mirrors are - that's really out of FreeBSD's hands.

FreeBSD does offer their .iso files over bittorrent.... Now, if pkgbase.live used scripts that roll an iso that is later distributed via bittorrent and then slurped in/processed by a client machine - now that would be something.

 

[tux2bsd]

> the posters in this thread are not considering

No, it is not that.

> Now, if pkgbase.live used scripts that roll an iso that is later distributed via bittorrent and then slurped in/processed by a client machine - now that would be something.

Yes, that was also described above... see: https://forums.FreeBSD.org/threads/pkgbase.79917/post-507425

 

[grahamperrin@]

… GhostBSD are succeeding with something like a PkgBase approach: …

For reference:

ghostbsd-ports/os at main · ghostbsd/ghostbsd-ports

FreeBSD ports with GhostBSD ports. Contribute to ghostbsd/ghostbsd-ports development by creating an account on GitHub.

github.com

– ports to build the base

 

[grahamperrin@]

… a separate utility that allows one to see what would be downloaded using freebsd-update fetch without downloading? …

When PkgBase becomes a norm, there'll be something like this:

pkg upgrade -n -r FreeBSD-base

For example:

root@mowa219-gjp4-8570p-freebsd:~ # pkg upgrade -n -r FreeBSD-base
Updating FreeBSD-base repository catalogue...
Fetching meta.conf: 100%    164 B   0.2kB/s    00:01   
pkg: https://alpha.pkgbase.live/current/FreeBSD:14:amd64/latest/packagesite.pkg: Not Found
Fetching packagesite.tzst: 100%   34 KiB  34.7kB/s    00:01   
Processing entries: 100%
The provides database is up-to-date.
FreeBSD-base repository update completed. 355 packages processed.
All repositories are up to date.
Checking for upgrades (1 candidates): 100%
Processing candidates (1 candidates): 100%
The following 1 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        FreeBSD-clibs: 14.snap20210513182414 -> 14.snap20211201183053 [FreeBSD-base]

Number of packages to be upgraded: 1

4 MiB to be downloaded.
root@mowa219-gjp4-8570p-freebsd:~ #

 

[tux2bsd]

That looks promising, very nice.

 

[tux2bsd]

note: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255967

 

[grahamperrin@]

Thanks, ▼

pkg upgrade # What is "(1 candidates)"? [reproducible]

root@freebsd-dns:~ # pkg update Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. root@freebsd-dns:~ # pkg upgrade Updating FreeBSD repository catalogue... FreeBSD repository is up to date. All repositories are up to date. Checking for...

forums.freebsd.org

 

[grahamperrin@]

Packaging and PkgBase, roadmapped:

▼

Technology Roadmap

https://freebsdfoundation.org/blog/technology-roadmap/ Enjoy.

forums.freebsd.org

 

[grahamperrin@]

Michael W Lucas has a few laughs about PkgBase and other things: <https://issue.freebsdfoundation.org/publication/?m=33057&i=739644&p=52&ver=html5> (FreeBSD Journal, January/February 2022). From the penultimate paragraph:

… Packaged base is the dread dragon of FreeBSD, devouring every developer who sets out to conquer it. The world has an endless supply of optimistic developers, however, …

I count myself as an optimistic tester …

 

[sidetone]

Package base seemed like a good idea then. If it's really that difficult with not proportional benefit, it may be better to put what they learned from that into something else. Here's the individual PDF of that article: https://freebsdfoundation.org/wp-content/uploads/2022/03/JanFeb_letters.pdf.

A vanguard subset of the ports tree, that's like an extension of base where each distribution set is optional, would be better.

 

[astyle]

Package base seemed like a good idea then. If it's really that difficult with not proportional benefit, it may be better to put what they learned from that into something else. Here's the individual PDF of that article: https://freebsdfoundation.org/wp-content/uploads/2022/03/JanFeb_letters.pdf.

A vanguard subset of the ports tree, that's like an extension of base where each distribution set is optional, would be better.

That PDF is mostly humorous stuff, but it does make good points about the never-resolved dependency hell. Along the same lines, I think that KDE should be upgradeable independently of the rest of the ports tree, and have been putting in some effort into that idea: Thread poudriere-bulk-not-finding-updates-to-the-tree.83134/. Post #51 in that thread has a possible solution that I still need to verify.

 

[grahamperrin@]

… If it's really that difficult with not proportional benefit, …

There's great benefit; see how easy it is to update GhostBSD.

 

[sidetone]

I think that KDE should be upgradeable independently of the rest of the ports tree, and have been putting in some effort into that idea: Thread poudriere-bulk-not-finding-updates-to-the-tree.83134/. Post #51 in that thread has a possible solution that I still need to verify.

KDE, Gnome, MATE and XFCE desktops each need their own separate repository on top of the main repository. These repositories would work together and the main repository would contains dependencies for the repository layered on top of it. Common applications and libraries that aren't specific to any of these desktops, but may be heavily associated or named along with these desktops, can be in the main repository. These desktops can be hosted on FreeBSD, but they really need their own portstree that works on top of the main portstree. These projects are too large, and their excessive dependencies mix up with other basic dependencies.

I didn't see my understanding of that suggestion about KDE in post #51, but more may have been on your mind along the lines of what you wrote there. A lot of port suites require looking into /usr/ports/Mk/Uses/.

 

[grahamperrin@]

KDE

Not related to PkgBase (base system); can the discussions of desktop enviroments be taken to the relevant topics? Thanks.

Window Managers

forums.freebsd.org