Solved Is a base system replacement for bootstrapping ports being created?

What do I do if I want a snapshot from June 10, 2021, 17:09:54 PST? I hunted the link down using a web browser, but can the base tools take a hash like that and get me the version I want?
Yeah. There is a snapshot download for each commit. One of the beauties of git is that commits aren't diffs they are complete snapshots.

The urls seem to follow the same format I outlined above, with a prefix of "ports-" followed by the hash then ".tar.gz" or ".zip" depending on your preference. You could use the base sytem "fetch" command or just use the download link on the page.
 
This basically boils down to having an SSH connection into the machine that has a barebones install, then you can copy-paste hashes (that you hunt down) into the SSH window. 😩

I was expecting some clever solution to the scenario of having only the barebones install, without the SSH connection (No cheating by using the old-school method of pen, paper, and carefully keying in the hash you actually wrote down)!
 
This basically boils down to having an SSH connection into the machine that has a barebones install, then you can copy-paste hashes (that you hunt down) into the SSH window. 😩

I was expecting some clever solution to the scenario of having only the barebones install, without the SSH connection (No cheating by using the old-school method of pen, paper, and carefully keying in the hash you actually wrote down)!

I’m not sure what else you could want here. The link you posted was to a commit in the port tree which implies you wanted to build something from source. It’s long been the standard in BSD to build non-base software locally from source. The command line you’d use for this exactly the same every time except for the hash which would be the exact same as the hash of the commit you want to build from. You could write a script that takes the hash as an argument and would download and build everything that you want.
 
fetch https://cgit.freebsd.org/src/snapshot/main.tar.gz tar -xf main.tar.gz --strip-components 2 -C /usr/src

Attempting to time things (for comparison purposes, in relation to <https://old.reddit.com/r/freebsd/comments/pfo6pc/-/hb80sg0/>)

Code:
% pwd
/tmp
% sudo mkdir /usr/timing
% time fetch https://cgit.freebsd.org/src/snapshot/main.tar.gz
fetch: https://cgit.freebsd.org/src/snapshot/main.tar.gz: size of remote file is not known
main.tar.gz                                            256 MB 2792 kBps 01m34s
1.439u 1.719s 1:34.99 3.3%      20+171k 14+0io 0pf+0w
% time sudo tar -xf main.tar.gz --strip-components 2 -C /usr/timing
ntp/scripts/ntptrace: Can't replace existing directory with non-directory
tar: Error exit delayed from previous errors.
4.260u 6.797s 0:21.34 51.7%     41+168k 475+107478io 0pf+0w
% cd /usr/timing/ntp/scripts/
% file ntptrace
ntptrace: directory
%
 
It's been one year since the portsnap deprecation announcement. I understand that the current non-deprecated method to bootstrapping ports is to install the net/gitup package and run gitup ports. I haven't see any mention of a user-friendly base system method to bootstrap the ports tree. Is anyone aware of work on a replacement in the base system for portsnap?
First of all, the deprecation wasn't announced but suggested. The technical arguments to do so are valid, so I'd expect this to happen some time in the future – but so far, it didn't happen. Portsnap is still there and supported in 13, we will see whether this will still be the case in 14.

But then: the easiest method for "bootstrapping" is to use some installation media that comes with a ports tree and select it. From there, you can build and install whatever you like, e.g. gitup. This is not an official method to keep your ports tree updated (maybe it will be in the future? I don't know), but one that's pretty easy to use.
 
First of all, the deprecation wasn't announced but suggested. The technical arguments to do so are valid, so I'd expect this to happen some time in the future – but so far, it didn't happen. Portsnap is still there and supported in 13, we will see whether this will still be the case in 14.

But then: the easiest method for "bootstrapping" is to use some installation media that comes with a ports tree and select it. From there, you can build and install whatever you like, e.g. gitup. This is not an official method to keep your ports tree updated (maybe it will be in the future? I don't know), but one that's pretty easy to use.
If you are downloading a release wouldn't the ports tree be frozen from the release date?

I'm just saying this is a simple way supported by the base system to download any branch, tag, or other "ref" of the ports or src using the base system without needing to build or install anything else so you can then build whatever you may want from source. To me that's the whole beauty of BSD, you can automagically build anything from source starting from just the base system.
 
tar -xf main.tar.gz --strip-components 2 -C /usr/src

ntp/scripts/ntptrace: Can't replace existing directory with non-directory

--strip-components 2 seems to be the problem.

linesteppr please, what's the intention?

Code:
% pwd
/tmp
% mkdir huh
% tar -xf main.tar.gz -C huh
% rm -r huh
% mkdir huh                  
% tar -xf main.tar.gz --strip-components 2 -C huh
ntp/scripts/ntptrace: Can't replace existing directory with non-directory
tar: Error exit delayed from previous errors.
% rm -r huh
%
 
If you are downloading a release wouldn't the ports tree be frozen from the release date?
You asked about "bootstrapping". So, you have an initial ports tree, it's bootstrapped. If you need any tool to keep it updated, you can build it from this initial ports tree.

That is, of course, assuming you want to avoid any binary package for whatever reason. And it is assuming portsnap will be removed indeed some day. And frankly, if that happens, it would be nice to have a tool to keep the ports tree updated (yep, that's different from "bootstrapping") in base. And we don't know yet where that may lead.

To me, this sounds a lot like "I don't like the move to git". This is unfounded for more than just one reason. Portsnap is still officially supported. And most reasons to deprecate it have nothing to do with git. The transition to git was just a convenient occassion to bring up the topic.
 
I frankly like portsnap's simple functionality. I can just roll up /var/db/portsnap/files into a gzipped tarball, and keep it around. FWIW, portsnap(8) can be configured to work with different directories, and if you pay attention, you can even do some simple version management with it.
 
You have to bootstrap pkg the first time you use pkg audit -F on a new base system install, after extracting a snashot of the ports tree.

I believe it's going to stay in the base system. Because I like it so much and they know it, so they will do it just for me. I've got a GitHub account, would rather have portsnap in the base system and only install programs from ports.

But Mindguard might be the exception, I first played that on PC-BSD and it still fascinates me. Simple as it is.
 

Also (primary archive):

<https://lists.freebsd.org/pipermail/freebsd-ports/2020-August/thread.html#119098> via [HEADS UP] Planned deprecation of portsnap

– and for anyone who has not yet seen these:





Beginning to use Git for FreeBSD ports ▶ <https://gist.github.com/grahamperrin/65f53251d819e70839e7aec7378f08a8>
 
Portsnap is actually in base FreeBSD....
Yes, the question was for how much longer. I said for ever and ever just like always and stay the same because Beasite is my Bestie and don't fix what isn't broken applies. I don't mind if they delete every text reference in print or how much it costs, as long as command not found what I find returned when I invoke the portsnap command. If it does I will adapt and survive.

And I actually asked the PC-BSD forums where it could be found in the ports tree, but nobody said a word. I stopped asking questions, started figuring things out on my own and never have to ask questions. But I told everybody here about it, and pointed it out in forum archives. it was so embarrassingly stupid I couldn't pass up the change to make fun of myself.

Then somebody thought that meant I really was stupid and everything PC-BSD vanished, including Weixiong and seven years of my existence within a ten day period I nailed down, only two people had the Admin Rights to do it and it wasn't the short one this time.

Well, not quite everything. I was still here, so was my post at Wilders and they had underestimated me twice and overestimated themselves by doing so. Now who has vanished and who never left to begin with, only got out another name with an x in it.
 
At a glance, no problem with stable/13:

1630826057539.png

The problem occurs with the main example:

tar -xf main.tar.gz --strip-components 2 -C /usr/src

1630826184037.png

--strip-components 2 seems to be the problem.

@linesteppr please, what's the intention?
 
don't fix what isn't broken
In fact, portsnap is arguably broken. Ever since introduction of quarterly branches, portsnap not supporting them was considered a missing feature, and you could rightfully say for a "canonical" tool for users to maintain their local ports tree, this qualifies as "broken".

Sure, you could finally try to add this feature. But then, there are the other shortcomings of portsnap (the mailing list posts summarize them). So, IMHO, it makes sense to think into the direction of deprecating portsnap (and maybe come up with some other solution for inclusion in base…)
 
In fact, portsnap is arguably broken. Ever since introduction of quarterly branches, portsnap not supporting them was considered a missing feature, and you could rightfully say for a "canonical" tool for users to maintain their local ports tree, this qualifies as "broken".

Sure, you could finally try to add this feature. But then, there are the other shortcomings of portsnap (the mailing list posts summarize them). So, IMHO, it makes sense to think into the direction of deprecating portsnap (and maybe come up with some other solution for inclusion in base…)
I disagree... I like portsnap's simplicity. Just pay attention to the dates, capture a timely snapshot, save it elsewhere. If you want to get fancy, start grepping for quarterly announcements, and then daisy-chain that to portsnap, and automate the whole capture process using just the base tools. :p I would think that this is the UNIX way of doing things :p
 
Zirias, I don't doubt your word but I don't follow mailing lists and it doesn't look broke to me.
Code:
root@bakemono:/ # portsnap fetch update
Looking up portsnap.FreeBSD.org mirrors... 4 mirrors found.
Fetching snapshot tag from ipv4.aws.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Wed Sep  1 11:12:42 CDT 2021 to Sun Sep  5 15:43:22 CDT 2021.
Fetching 5 metadata patches... done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
Fetching 594 patches. 
(594/594) 100.00%  done. .                                     
done.
Applying patches... 
done.
Fetching 8 new ports or files... done.
Removing old files and directories... done.
Extracting new files:
.......
Building new INDEX files... done.
root@bakemono:/ # pkg audit -F
vulnxml file up-to-date
0 problem(s) in 0 installed package(s) found.
root@bakemono:/ # freebsd-update fetch
Looking up update.FreeBSD.org mirrors... 2 mirrors found.
Fetching metadata signature for 12.2-RELEASE from update1.freebsd.org... done.
Fetching metadata index... done.
Inspecting system... done.
Preparing to download files... done.

No updates needed to update system to 12.2-RELEASE-p10.
root@bakemono:/ #

That's the way I like to see things. Working like they should.
But a lot of things work for me differently than what seems to be SOP, like never bothering to open /etc/fstab.
 
You asked about "bootstrapping". So, you have an initial ports tree, it's bootstrapped. If you need any tool to keep it updated, you can build it from this initial ports tree.

That is, of course, assuming you want to avoid any binary package for whatever reason. And it is assuming portsnap will be removed indeed some day. And frankly, if that happens, it would be nice to have a tool to keep the ports tree updated (yep, that's different from "bootstrapping") in base. And we don't know yet where that may lead.

To me, this sounds a lot like "I don't like the move to git". This is unfounded for more than just one reason. Portsnap is still officially supported. And most reasons to deprecate it have nothing to do with git. The transition to git was just a convenient occassion to bring up the topic.
ports-mgmt/portsnap only bootstraps the main branch which, afaik, you're then stuck with. Fetching and untaring a quartely branch snapshot seems to be the only way to bootstrap quarterly ports from the base system.

It's not about the switch to git at all other than git isn't in the base system.
 
Code:
07 Jan 2009 21:34:27 -  Remove portsnap port, it's now integrated into the base system

only bootstraps the main branch which, afaik, you're then stuck with.
Then what's going on here?
Code:
root@bakemono:/ # portsnap fetch update
Looking up portsnap.FreeBSD.org mirrors... 4 mirrors found.
Fetching snapshot tag from ipv4.aws.portsnap.freebsd.org... done.
Fetching snapshot metadata... done.
Updating from Wed Sep  1 11:12:42 CDT 2021 to Sun Sep  5 15:43:22 CDT 2021.
Fetching 5 metadata patches... done.
Applying metadata patches... done.
Fetching 0 metadata files... done.
Fetching 594 patches. 
(594/594) 100.00%  done. .                                     
done.
Applying patches... 
done.
Fetching 8 new ports or files... done.
Removing old files and directories... done.
Extracting new files:

Fetching and untaring a quartely branch snapshot seems to be the only way to bootstrap quarterly ports from the base system.
That's the way it's worked for me since 2005.
I don't get a say in what goes in the Base System, but said it should stay anyway.

It's not about the switch to git at all other than git isn't in the base system.
It is for me.
 
It's not about the switch to git at all other than git isn't in the base system.
Well, kind of my words, and it's nice to see you do not follow this somewhat common misconception. There's one way this is slightly (and indirectly!) related to git: The move to git was a good opportunity to bring up the topic ;)

But I also told you portsnap does not just "bootstrap", it's meant for ongoing maintenance of an end-user's local tree. Extracting an archive from an installation media would technically qualify as bootstrapping – from there, you can build and install whatever software you want/need to do further maintenance ;)
 
Then what's going on here?
This fetches the latest ports (from the main branch). Again: portsnap never supported quarterly. And yes, that's a serious issue.

It's even worse because pkg defaults to quarterly packages. And yes, people will attempt to mix ports and packages. A general warning about that is appropriate although it can be "done right", BUT: Mixing two different branches is definitely recipe for instant disaster.
it doesn't look broke to me.
Not sure what you're trying to show with that? Yes, fetching latest(!) ports with portsnap (still) works. It still works because obviously, some people opposed to getting rid of it more quickly, and as a result, the team had to do all the work to adapt portsnap's backend after switching to git – only to get a tool that does a little part of what git (or svn) can do in a less efficient (more disk space incl metadata, slower speed) way…

And btw, freebsd-update(8) is completely unrelated.
 
Again: portsnap never supported quarterly. And yes, that's a serious issue.
I don't see why exactly... If you pay attention to the quarterly announcements, and do timely snapshots, it shouldn't be too much of an issue. I say, keep it simple. You want fancy, use git. To bootstrap, just use pkg to install it as the VERY FIRST THING, then install absolutely nothing else using pkg, and then stick to using git for dear life.
 
Back
Top