What's the best way to update all ports after upgrading from FreeBSD 11 to 12?

I have 3 web servers running FreeBSD 11.4 and I want to upgrade them to 12.2. I upgraded one last night and ran into some issues I've never seen before in past FreeBSD updates. I'm always trying to learn new things and I figured maybe I haven't been doing things properly in the past, even though they seemed to work. What's the best way?

I updated the server to FreeBSD 12.2 using the normal procedure described in the docs. No issues.

Maybe I did something wrong (it was 3am of course) but my usual method of updating all of my third party apps is to:
# portsnap fetch && portsnap update
# portupgrade -urRa

This has worked fine on all 3 servers, for the past few years. Sometimes I have to make some adjustments noted in /usr/ports/UPDATING but everything usually works well.

But when portupgrade was trying to update Apache, it stopped. So, I went to do the update manually and got this...
# cd /usr/ports/www/apache24
# make install clean
unixd.c:245:25: error: variable has incomplete type 'union semun'
            union semun ick;
unixd.c:245:19: note: forward declaration of 'union semun'
            union semun ick;
1 error generated.
*** [unixd.lo] Error code 1

make[5]: stopped in /usr/ports/www/apache24/work/httpd-2.4.48/os/unix
1 error

make[5]: stopped in /usr/ports/www/apache24/work/httpd-2.4.48/os/unix
*** [all-recursive] Error code 1

make[4]: stopped in /usr/ports/www/apache24/work/httpd-2.4.48/os/unix
1 error

make[4]: stopped in /usr/ports/www/apache24/work/httpd-2.4.48/os/unix
*** [all-recursive] Error code 1

make[3]: stopped in /usr/ports/www/apache24/work/httpd-2.4.48/os
1 error

make[3]: stopped in /usr/ports/www/apache24/work/httpd-2.4.48/os
*** [all-recursive] Error code 1

make[2]: stopped in /usr/ports/www/apache24/work/httpd-2.4.48
1 error

make[2]: stopped in /usr/ports/www/apache24/work/httpd-2.4.48
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1


Next, I tried Googling and found a few people experiencing the same thing, but from 1 or 2 years ago, and no clear solutions. I did notice one person said he used pkg to install apache24 and it worked, so I tried that and found some interesting (to me, lol) output:
# pkg install apache24
Updating FreeBSD repository catalogue...
Fetching meta.conf: 100%    163 B   0.2kB/s    00:01   
pkg: http://pkgmir.geo.freebsd.org/FreeBSD:12:amd64/quarterly/packagesite.pkg: Not Found
Fetching packagesite.txz: 100%    6 MiB   6.6MB/s    00:01   
Processing entries: 100%
FreeBSD repository update completed. 30844 packages processed.
All repositories are up to date.
Updating database digests format: 100%
The following 19 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
    apache24: 2.4.48
    mozjpeg: 4.0.3

Installed packages to be REINSTALLED:
    apr- (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')
    giflib-5.2.1 (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')
    jansson-2.13.1 (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')
    jbigkit-2.1_1 (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')
    json-c-0.15_1 (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')
    libgd-2.3.1,1 (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')
    libunwind-20201110 (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')
    libyaml-0.2.5 (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')
    tiff-4.3.0 (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')
    webp-1.2.0 (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')
    wget-1.21 (ABI changed: 'freebsd:11:x86:64' -> 'freebsd:12:x86:64')

Number of packages to be installed: 2
Number of packages to be reinstalled: 17

WOAH!! Why are all of those FreeBSD 11 packages still on my system??? Did I not use the right options with portupgrade ??

Did I do something else wrong??

After I completed the above pkg command, I did an experiment to see if I could reinstall apache24 from ports... AND IT WORKED!! No errors.

So, I must have done something wrong that caused a bunch of FreeBSD 11 stuff to remain on my system and cause issues. But what? And how can I be sure there still aren't packages from 11 lingering around? I'm trying to figure out the best approach to take when I upgrade the other 2 servers. This one is a backup server but the other 2 have live, client web sites on them.

Thanks!! P.S. I love this forum <3
Note that installing from packages (using pkg) is slightly different compared to installing from ports. pkg uses its own database, it is only updated by pkg.
One strategy I have used with major upgrades ("major" in this case can both be from a major FreeBSD version to another (11 -> 12) and upgrading from a quite old package / ports installation to the new one) is this:
1) save a copy of all packages installed (a list of packages)
2) deinstall all packages
3) upgrade pkg or ports database
4) install everything you need from the list you made in 1), starting with major packages / ports

this neatly avoids any dependency problems caused by changes in ports / packages over time. It can cost you more time.
1) save a copy of all packages installed (a list of packages)
so, that would be pkg info ??

2) deinstall all packages
so pkg delete -a ??

3) upgrade pkg or ports database
Would I delete the ports tree and reload it like this? :
# rm -rfv /usr/ports
# portsnap fetch extract

But it seems like there should be an easier way to upgrade everything. I still feel like I made a mistake somewhere. Going back through my notes, it looks like I've always been updating different versions of FreeBSD 11 (11.0 --> 11.1 --> 11.2 --> 11.3 --> 11.4) for years and haven't done one of these major updates until now.
Rob215x said:
But it seems there should be an easier way to upgrade everything.
Portupgrade IS the recommended way to upgrade ports after a major system update, you did it the right way. However, problems like these occur sometimes. My way is somewhat similar to what tingo described: Delete the package as installed with pkg delete -f ..., then make install clean and after that resume portupgrade until finished. If a build stops with an error like in your case, I would install the package version too, so that things would work.

However, that leaves you with a problem when the next version of Apache arrives. Sometimes errors like these magically disappear, but usually they don't. Building ports requires header files from the system, you may have a problem there. Are there more prime ports that won't build? The 'union semun' in the error message is defined in /usr/include/sys/sem.h. That file, like the other files in that directory, should match the files in the source tree in directory /usr/src/sys/sys. Do you see differences there?
here are my notes from 2019 when i update from 11.x to 12.1 Those are notes for me not the actual commands that are executed during the test update in vm.
portsnap auto
#stop all public services in rc.conf so they dont start

freebsd-update -r 12.1-RELEASE upgrade

/etc/newsyslog not merged!!!!

#take a note ntpd:123
#take a note /usr/local/etc/newsyslog.conf.d/*

#install kernel
freebsd-update install
shutdown -r now

#install userland
freebsd-update install

#rebuild all installed ports
pkg-static install -f pkg

#remove old portupgrade and ruby
cd /usr/ports/ports-mgmt/portupgrade make deinstall #make clean
cd /usr/ports/lang/p5-Scalar-List-Util make deinstall make clean
pkg autoremove # to remove ruby
cd /usr/ports/ports-mgmt/portupgrade #make clean #make install

portupgrade -af
cd /usr/local/lib/compat/pkg
find . -type l -exec test ! -e {} \; -delete

freebsd-update install
shutdown -r now

#last call to freebsd-update to check if anything else need to be installed
freebsd-update install
shutdown -r now

#enable the services in /etc/rc.conf
shutdown -r now

portsclean -L
pkg autoremove
portupgrade -fu db5

# change folder
cd /usr/local/lib/compat/pkg
# print dangling links
find . -type l -exec test ! -e {} \; -print
# delete dangling links
find . -type l -exec test ! -e {} \; -delete

#upd EFI