Upgrading versions via source tree

Phishfry

Phishfry

I have a simple question. I have never upgraded an installation from source tree but I am ready to try.

All of my recent freebsd-update upgrade problems revolve around password files. I don't understand why it needs to happen.

But my question is: Does upgrading from source look at the password files and scrutinize master password files?
I guess it would be the mergemaster step.

Is that just an unavoidable part of upgrading? Mucking with the accounts?

Because it has gone horribly wrong for me recently after years of reliable use.
 
My bad not mergmaster but etcupdate seems to be the checker..

It will show the conflicted file. master.passwd

etcupdate(8)

man.freebsd.org man.freebsd.org

I like the sound of this one:
(p) postpone Ignore this conflict for now.
 
So to be proper when source upgrading across multiple versions I need to check /UPDATING on each version?

IE. going from 13.2-RELEASE to 14.1-RELEASE.
Check 13.3, 13.4, 14.0, 14.1 /UPDATING
 
I might as well move to STABLE on this build. Its the oldest Rock Pi 4 in my farm.

rm -f -d -R /usr/src
git clone --branch stable/14 https://git.FreeBSD.org/src.git /usr/src
make -j6 buildworld buildkernel

Code: 
--- AArch64GenMCPseudoLowering.inc ---
llvm-tblgen -gen-pseudo-lowering  -I /usr/src/contrib/llvm-project/llvm/include -I /usr/src/contrib/llvm-project/llvm/lib/Target/AArch64  -d AArch64GenMCPseudoLowering.inc.d -o AArch64GenMCPseudoLowering.inc  /usr/src/contrib/llvm-project/llvm/lib/Target/AArch64/AArch64.td
--- AArch64GenFastISel.inc ---
Killed
*** [AArch64GenFastISel.inc] Error code 137

make[4]: stopped in /usr/src/lib/clang/libllvm
1 error

make[4]: stopped in /usr/src/lib/clang/libllvm

make[3]: stopped in /usr/src/lib/clang

make[2]: stopped in /usr/src
      142.93 real       313.33 user        21.64 sys

make[1]: stopped in /usr/src

make: stopped in /usr/src

OK will regroup and try 14.1-RELEASE
 
500 megs of tempfs.
tmpfs /tmp tmpfs rw,mode=1777,size=500m 0 0
Click to expand...

No swap partiton.
real memory = 2112208896 (2014 MB)

256GB NVMe running at PCIe 2.0 x4

RK3399 uses big+little cores. So I am going to try and scale back to -j4 first to see where it goes.
 
You can't go -j 6 with 2 GB RAM.

Even if you add lots of swapspace you are probably overall faster with -j4 and -j3.
 
Phishfry said:
All of my recent freebsd-update upgrade problems revolve around password files. I don't understand why it needs to happen.
Click to expand...
I never used freebsd-update(8) myself (outside of poudriere jails which are "unconfigured" anyways, so no conflicts ever happen), but I'm pretty sure it does the classic 3-way merge as well. If an upstream change and a local change affect the same line of a file, this can't be merged automatically. This had to happen when the default shell for root was changed, as soon as you set a password for root, both changed the same line in the file.

Phishfry said:
But my question is: Does upgrading from source look at the password files and scrutinize master password files?
I guess it would be the mergemaster step.
Click to expand...
Phishfry said:
My bad not mergmaster but etcupdate seems to be the checker..
Click to expand...
Yes, indeed. A make installworld doesn't touch /etc at all. The config files are (initially) installed with make distribution (which also requires make distrib-dirs if you're installing into something completely empty). You won't do that on upgrades, it would just overwrite your custom configuration.

etcupdate(8) will do the 3-way merge. It has a "resolve" mode, I'd suggest to just always run it (etcupdate; etcupdate resolve), which will give you the typical resolve options for each conflicted file, including opening it in your $EDITOR for manual resolving.

Warning: etcupdate(8) needs the old "vanilla" /etc tree for the 3-way merge, and /usr/src will only contain the new versions after upgrading e.g. with git. On a fresh install from source, don't forget to initially create it with etcupdate extract, otherwise the tool will be useless when it comes to upgrading.
 
Just for recap build failed at -j6 in 40 min; -j4 in 3 hours; and -j3 in 5 hours.
I am now building with -j2 and its +13 hours and still cranking out LLVM object files.

From top -P
Mem: 507M Active, 210M Inact, 15M Laundry, 462M Wired, 199M Buf, 742M Free
Click to expand...

gstat shows not much disk activity at all.
 
Phishfry said:
Just for recap build failed at -j6 in 40 min; -j4 -n 3 hours; and -j3 in 5 hours.
I am now building with -j2 and its +13 hours and still cranking out LLVM object files.

From top -P
Click to expand...

Ouch.

I had a request to recommend a OS for a 256 MB RAM machine, so I spun up a FreeBSD-current VM and started `make -j 1 world`. Not a chance, way too slow. Couldn't keep 1 core busy without excessive disk wait.
 
Finally I have something to work with.
Code: 
    12379.68 real     23397.50 user      1440.32 sys
--- buildworld_epilogue ---
--------------------------------------------------------------
>>> World build completed on Tue Oct 22 03:05:27 UTC 2024
>>> World built in 173738 seconds, ncpu: 6, make -j2
--------------------------------------------------------------

Kernel build went much quicker.
Code: 
created kernel.bin from kernel.full
     4837.38 real     18048.76 user      1175.58 sys
--------------------------------------------------------------
>>> Kernel build for GENERIC completed on Tue Oct 22 07:59:11 UTC 2024
--------------------------------------------------------------
>>> Kernel(s)  GENERIC built in 4868 seconds, ncpu: 6, make -j4
--------------------------------------------------------------
 
  • Like
Reactions: bjs
I have to ask what part my "No Swap on NVMe" policy has to do with this.

I have a small group of these RockPi4 so I am going to add a swap partition to see how much faster it compiles buildworld.

MarkMillard seemed to suggest 11GB was good from 2021 for arm64. So what would be a good amount now? 16GB or more?

I don't compile software much so "NoSwap on SSD" has worked out no problem but I know it is there for a reason for it....
 
I also wonder about my fstab setting of tempfs size. 500 megs is quarter of the RAM on RockPi4.

Maybe I will try one run with no size set. Is it tempfs set dynamically if not specified?
I assumed it was fixed and small amount.

Nothing in here about amount:

tmpfs

man.freebsd.org man.freebsd.org

Here neither:


My bad here it is:
If zero (the default) the available amount of memory (in-cluding main memory and swap space) will be used.
 
You must log in or register to reply here.
Back
Top