Solved poudriere/git - moving to new quarterly branch

S

sko

It's that time of the year again to move the quarterly ports repos on my build hosts to the new branch. And again I fail to find any sane way in the poudriere manpages to achieve this (supposedly) simple task without just destroying & re-downloading the whole quarterly portstree.

Code: 
root@srv1:/usr/local/poudriere/ports/quarterly # git status
On branch 2023Q2
Your branch is up to date with 'origin/2023Q2'.

nothing to commit, working tree clean
root@srv1:~ # poudriere ports -p quarterly -u -B 2023Q3
[00:00:01] Updating portstree "quarterly" with git+https... done
root@srv1:/usr/local/poudriere/ports/quarterly # git status
On branch 2023Q2
Your branch is up to date with 'origin/2023Q2'.

nothing to commit, working tree clean

also it seems poudriere isn't using a full git clone to create the tree, because there is no information about remote branches except the one specified when creating the tree, and hence I can't switch to the new branch:
Code: 
root@srv1:/usr/local/poudriere/ports/quarterly # git branch -r
  origin/2023Q2
root@srv1:/usr/local/poudriere/ports/quarterly # git switch origin/2023Q3
fatal: invalid reference: origin/2023Q3


So in short: how can I switch to a new quarterly branch with poudriere without completely deleteing & re-creating the ports tree every 3 months?
 
sko said:
So in short: how can I switch to a new quarterly branch with poudriere without completely deleteing & re-creating the ports tree every 3 months?
Click to expand...
Simplest way,
Code: 
cd /usr/local/poudriere/ports/quarterly
git pull -p
git checkout 2023Q3
 
It's painless when you use git directly instead of using poudriere to manage it. Just clone the ports repository yourself and add it to poudriere with the -m null method. Then all you need to to is git pull and git checkout 2023Q3.
 
SirDice said:
Simplest way,
Code: 
cd /usr/local/poudriere/ports/quarterly
git pull -p
git checkout 2023Q3
Click to expand...
sadly this won't work with a ports tree created by poudriere:
Code: 
root@stor2:/usr/local/poudriere/ports/quarterly # git pull -p
Already up to date.
root@stor2:/usr/local/poudriere/ports/quarterly # git branch -r
  origin/2023Q2

zirias@ said:
It's painless when you use git directly instead of using poudriere to manage it. Just clone the ports repository yourself and add it to poudriere with the -m null method. Then all you need to to is git pull and git checkout 2023Q3.
Click to expand...

That seems to be the better way, yes.
I now came up with a rather 'hacky' solution by just nuking (for the last time I hope...) the contents of the poudriere-cloned 'quarterly' repository, running a manual git clone https://git.freebsd.org/ports.git /usr/local/poudriere/ports/quarterly and then checking out the 2023Q3 branch.
This way poudriere still knows the method to update the tree, so it is still possible to run poudriere -u -p quarterly.
Checking out the next quarterly branch can then be done via git.


I still think it's odd that poudriere doesn't have a simple way to update ports to a new quarterly branch.
 
sko said:
sadly this won't work with a ports tree created by poudriere:
Click to expand...
git checkout 2023Q3

But maybe checkout main first, then the pull, then the switch to 2023Q3. Could probably be done with a clever git fetch ... too. But I never tried those, I know a bit of git but haven't used it as much as CVS and SVN yet.

My typical workflow is just to git checkout main; git pull; git checkout <branchname>.
 
SirDice said:
git checkout 2023Q3

But maybe checkout main first, then the pull, then the switch to 2023Q3. Could probably be done with a clever git fetch ... too.
Click to expand...
I tried the --refetch and --unshallow options with git fetch to no avail, I couldn't see (via git branch -r) or checkout any other branch. However, I only tried to checkout 2023Q3, not main...
I bet somewhere in the deep and unholy swamp of git options and arguments there would have been the correct command to fix the crippled repo, but if this was the last time I had to clone it, I'm fine with the quick&dirty solution of rm -rf and git clone.


For newly created ports trees, I think the -F switch for poudriere ports and running a manual git clone would be the easiest solution.
 
After a while I gave up of using poudriere's git repo. Now all my poudriere ports are made with -m null -M /usr/ports, way easier to maintain.
 
Oh, it's due to poudriere using a shallow (--depth 1) clone.

Code: 
root@molly:~ # poudriere ports -c -p test
[00:00:01] Creating test fs at /usr/local/poudriere/ports/test... done
[00:00:02] Cloning the ports tree... done
root@molly:~ # cd /usr/local/poudriere/ports/test/
root@molly:/usr/local/poudriere/ports/test # git rev-parse --is-shallow-repository
true
Converting a shallow clone back to a full one is a bit of a struggle though.
TL;DR:
Code: 
git fetch --unshallow
git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
git fetch origin

Code: 
root@molly:/usr/local/poudriere/ports/test # git fetch --unshallow
load: 10.75  cmd: git-remote-https 14878 [select] 158.71r 0.05u 0.04s 0% 10844k
remote: Enumerating objects: 5675370, done.
remote: Counting objects: 100% (5675338/5675338), done.
remote: Compressing objects: 100% (2183676/2183676), done.
remote: Total 5620086 (delta 3421625), reused 5570267 (delta 3387590), pack-reused 0
Receiving objects: 100% (5620086/5620086), 1.00 GiB | 3.25 MiB/s, done.
Resolving deltas: 100% (3421625/3421625), completed with 23724 local objects.
remote: Enumerating objects: 13, done.
remote: Total 13 (delta 0), reused 0 (delta 0), pack-reused 13
Unpacking objects: 100% (13/13), 2.09 KiB | 2.09 MiB/s, done.
From https://git.FreeBSD.org/ports
 * [new tag]                   10-eol     -> 10-eol
 * [new tag]                   11-eol     -> 11-eol
 * [new tag]                   12.3-eol   -> 12.3-eol
 * [new tag]                   12.3-last  -> 12.3-last
 * [new tag]                   7-eol      -> 7-eol
 * [new tag]                   8-eol      -> 8-eol
 * [new tag]                   9-eol      -> 9-eol
 * [new tag]                   pkg-install-eol -> pkg-install-eol
 * [new tag]                   release/2.2.5 -> release/2.2.5
 * [new tag]                   release/2.2.6 -> release/2.2.6
 * [new tag]                   release/2.2.7 -> release/2.2.7
 * [new tag]                   release/2.2.8 -> release/2.2.8
 * [new tag]                   release/3.0.0 -> release/3.0.0
root@molly:/usr/local/poudriere/ports/test # git checkout 2023Q3
error: pathspec '2023Q3' did not match any file(s) known to git
root@molly:/usr/local/poudriere/ports/test # git branch -a
* main
  remotes/origin/HEAD -> origin/main
  remotes/origin/main
root@molly:/usr/local/poudriere/ports/test # git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
root@molly:/usr/local/poudriere/ports/test # git fetch origin
remote: Enumerating objects: 62472, done.                                                                                   [25/311]
remote: Counting objects: 100% (24319/24319), done.
remote: Compressing objects: 100% (23/23), done.
remote: Total 62472 (delta 24299), reused 24313 (delta 24296), pack-reused 38153
Receiving objects: 100% (62472/62472), 31.27 MiB | 15.78 MiB/s, done.
Resolving deltas: 100% (41824/41824), completed with 7977 local objects.
From https://git.FreeBSD.org/ports
 * [new branch]                2014Q1     -> origin/2014Q1
 * [new branch]                2014Q2     -> origin/2014Q2
 * [new branch]                2014Q3     -> origin/2014Q3
 * [new branch]                2014Q4     -> origin/2014Q4
 * [new branch]                2015Q1     -> origin/2015Q1
 * [new branch]                2015Q2     -> origin/2015Q2
 * [new branch]                2015Q3     -> origin/2015Q3
 * [new branch]                2015Q4     -> origin/2015Q4
 * [new branch]                2016Q1     -> origin/2016Q1
 * [new branch]                2016Q2     -> origin/2016Q2
 * [new branch]                2016Q3     -> origin/2016Q3
 * [new branch]                2016Q4     -> origin/2016Q4
 * [new branch]                2017Q1     -> origin/2017Q1
 * [new branch]                2017Q2     -> origin/2017Q2
 * [new branch]                2017Q3     -> origin/2017Q3
 * [new branch]                2017Q4     -> origin/2017Q4
 * [new branch]                2018Q1     -> origin/2018Q1
 * [new branch]                2018Q2     -> origin/2018Q2
 * [new branch]                2018Q3     -> origin/2018Q3
 * [new branch]                2018Q4     -> origin/2018Q4
 * [new branch]                2019Q1     -> origin/2019Q1
 * [new branch]                2019Q2     -> origin/2019Q2
 * [new branch]                2019Q3     -> origin/2019Q3
 * [new branch]                2019Q4     -> origin/2019Q4
 * [new branch]                2020Q1     -> origin/2020Q1
 * [new branch]                2020Q2     -> origin/2020Q2
 * [new branch]                2020Q3     -> origin/2020Q3
 * [new branch]                2020Q4     -> origin/2020Q4
 * [new branch]                2021Q1     -> origin/2021Q1
 * [new branch]                2021Q2     -> origin/2021Q2
 * [new branch]                2021Q3     -> origin/2021Q3
 * [new branch]                2021Q4     -> origin/2021Q4
 * [new branch]                2022Q1     -> origin/2022Q1
 * [new branch]                2022Q2     -> origin/2022Q2
 * [new branch]                2022Q3     -> origin/2022Q3
 * [new tag]                   release/10.0.0 -> release/10.0.0
 * [new tag]                   release/10.1.0 -> release/10.1.0
 * [new tag]                   release/10.2.0 -> release/10.2.0
 * [new tag]                   release/10.3.0 -> release/10.3.0
 * [new tag]                   release/10.4.0 -> release/10.4.0
 * [new tag]                   release/11.0.0 -> release/11.0.0
 * [new tag]                   release/11.1.0 -> release/11.1.0
 * [new tag]                   release/11.3.0 -> release/11.3.0
 * [new tag]                   release/11.4.0 -> release/11.4.0
 * [new tag]                   release/12.0.0 -> release/12.0.0
 * [new tag]                   release/12.1.0 -> release/12.1.0
 * [new tag]                   release/12.2.0 -> release/12.2.0
 * [new tag]                   release/12.3.0 -> release/12.3.0
 * [new tag]                   release/12.4.0 -> release/12.4.0
 * [new tag]                   release/13.1.0 -> release/13.1.0
 * [new tag]                   release/13.2.0 -> release/13.2.0
 * [new tag]                   release/9.3.0 -> release/9.3.0
root@molly:/usr/local/poudriere/ports/test # git checkout 2023Q3
branch '2023Q3' set up to track 'origin/2023Q3'.
Switched to a new branch '2023Q3'
 
Looking through the code, I noticed a -D option that isn't mentioned in the poudriere-ports(8) man page. It negates the --depth 1 at checkout and does a full clone.

Code: 
root@molly:~ # poudriere ports -c -p test -D
[00:00:01] Creating test fs at /usr/local/poudriere/ports/test... done
[00:00:02] Cloning the ports tree... done
root@molly:~ # cd /usr/local/poudriere/ports/test
root@molly:/usr/local/poudriere/ports/test # git checkout 2023Q3
branch '2023Q3' set up to track 'origin/2023Q3'.
Switched to a new branch '2023Q3'

Option may only be available in ports-mgmt/poudriere-devel through, I didn't check the regular version.
 
SirDice said:
Looking through the code, I noticed a -D option that isn't mentioned in the poudriere-ports(8) man page. It negates the --depth 1 at checkout and does a full clone.
[...]
Option may only be available in ports-mgmt/poudriere-devel through, I didn't check the regular version.
Click to expand...
Unless I'm mistaken, isn't this the -D option in poudriere-ports(8) you are referring to?
Code: 
OPTIONS
[...]
     -m	method	    Specify which method to use	to create the ports tree.  The
		    default is git+https.
[..]
		    git[+protocol]  Use	Git to download	the sources.

				    Sources will be cloned shallowly unless -D
				    is specified.
This -D option is equally available in poudriere ports.8 as in poudriere ports.8 devel
 
Ah, you're right. I simply had a quick look through the list under options, and it's not mentioned there. But it's explained in a sub section of a sub section.
 
Coming back to this old thread, I encountered this today when trying to switch my quarterly ports tree to the recently released 2024Q4 branch via poudriere ports -v -p quarterly -u -B 2024Q4, and kept noticing that nothing happened no matter what I did.

I tried "unshallowing" the tree first via git -C /usr/local/poudriere/ports/quarterly fetch -v --unshallow origin and that did succeed in fetching the history of the branch I already had checked out and in removing the .git/shallow file, but still did not fetch other branches.

Then I noticed my clone was hardcoded to fetch only a single branch, i.e.:
sh: 
-> cat .git/config
(…)
[remote "origin"]
        url = https://github.com/freebsd/freebsd-ports.git
        fetch = +refs/heads/2024Q3:refs/remotes/origin/2024Q3
(…)
Manually setting the references to +refs/heads/*:refs/remotes/origin/* successfully got a new poudriere ports -v -p quarterly -u -B 2024Q4 to fetch all remote branches, including my target origin/2024Q4, but still without switching to a local 2024Q4 counterpart.

So at this point, running out of ideas, I simply did what I should have done from the start, i.e. turning to the source:
sh: 
-> poudriere version
poudriere-git-3.4.2
-> less -N /usr/local/share/poudriere/ports.sh
(…)
385                 ${GIT_CMD} -C ${PORTSMNT:-${PTMNT}} pull --rebase ${quiet} || \
(…)
And that right there pretty much tells you that the ports command will never switch you from any branch to another when passed the -B flag, because all that's happening under-the-hood is a simple Git pull --rebase action, which is not equipped for that task.

So, all that needs to be done for this to work is submitting a merge request for poudriere's ports command to recognize a -B flag when requested to update an existing tree ;) Other than that, manually adjusting your tree as described here, and then switching to the target branch, will accomplish the task, and enable poudriere ports -u to continue working into the future with your new branch, at least until you decide to switch again.
 
You must log in or register to reply here.
Back
Top