Reinstalling ports with synth after 10.3 to 11.0 upgrade

Wapcaplet · Sep 26, 2016

Is there a preferred way to reinstall ports using ports-mgmt/synth after a major version upgrade? I'm preparing to upgrade one of my servers from 10.3 to 11.0, and I've always used ports-mgmt/portmaster in the past to rebuild everything.

Thanks!

marino · Sep 26, 2016

IIRC the only trick is that you should run pkg command manually after a major version upgrade because it emits some kind of warning. That warning messes synth up. So you need to upgrade at least ports-mgmt/pkg via official package or rebuild it from source. Then it's just a matter of:

clearing out any existing packages in Synth package directory
Generate a list of ports to install later(e.g. pkg prime-list > ~/inslist)
Generate same list, but in Synth friendly format (e.g. pkg query -e '%a == 0' '%o' > ~/synthlist
Building your package set (e.g. synth build `cat ~/synthlist`
pkg delete -af (you may need to rebuild pkg again after that)
pkg ins -r Synth `cat ~/inslist`

something like that ...

kpa · Sep 26, 2016

Re-install of ports-mgmt/pkg can be done with /usr/sbin/pkg bootstrap as the first step after the upgrade.

Wapcaplet · Sep 27, 2016

So I upgraded my system to 11.0-RELEASE and tried to rebuild ports-mgmt/synth, but the build errored out in the middle of lang/gcc6-aux:

Code:

   > tmp-constants.h                                                                                                                                                                                                                [243/870]
/bin/sh /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/../move-if-change tmp-constants.h insn-constants.h
build/genenums /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/common.md /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/config/i386/i386.md \
   > tmp-enums.c
echo timestamp > s-constants
/bin/sh /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/../move-if-change tmp-enums.c insn-enums.c
if [ xinfo = xinfo ]; then \
        makeinfo --split-size=5000000 --split-size=5000000 --no-split -I . -I /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/doc \
                -I /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/doc/include -o doc/gccint.info /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/doc/gccint.texi; \
fi
build/gengtype  \
                    -S /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc -I gtyp-input.list -w tmp-gtype.state
echo timestamp > s-enums
/usr/ports/lang/gcc6-aux/work/bootstrap/bin/g++ -c   -O2 -pipe -DLIBICONV_PLUG -fstack-protector -fno-strict-aliasing  -DLIBICONV_PLUG -DIN_GCC     -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-st
rings -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings   -DHAVE_CONFIG_H -DGENERATOR_FILE -fno-PIE -I. -Ibuild -I/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc
 -I/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/build -I/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/../include  -I/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/../libcpp/include -I/usr/local/include -DLIBICONV_PLUG \
        -o build/min-insn-modes.o min-insn-modes.c
/usr/ports/lang/gcc6-aux/work/bootstrap/bin/g++ -c   -O2 -pipe -DLIBICONV_PLUG -fstack-protector -fno-strict-aliasing  -DLIBICONV_PLUG -DIN_GCC     -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-st
rings -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings   -DHAVE_CONFIG_H -DGENERATOR_FILE -fno-PIE -I. -Ibuild -I/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc
 -I/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/build -I/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/../include  -I/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/../libcpp/include -I/usr/local/include -DLIBICONV_PLUG \
        -o build/gencheck.o /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/gencheck.c
In file included from ./tm.h:19:0,
                 from /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/gencheck.c:26:
./options.h:5711:3: error: redefinition of 'OPT_C'
   OPT_C = 129,                               /* -C */
   ^
./options.h:5708:3: note: 'OPT_C' previously defined here
   OPT_C = 126,                               /* -C */
   ^
./options.h:5719:3: error: redefinition of 'OPT_d'
   OPT_d = 137,                               /* -d */
   ^
./options.h:5717:3: note: 'OPT_d' previously defined here
   OPT_d = 135,                               /* -d */
   ^
./options.h:5720:3: error: redefinition of 'OPT_D'
   OPT_D = 138,                               /* -D */
   ^
./options.h:5718:3: note: 'OPT_D' previously defined here
   OPT_D = 136,                               /* -D */
   ^
./options.h:5723:3: error: redefinition of 'OPT_d'
   OPT_d = 141,                               /* -d */
   ^
./options.h:5717:3: note: 'OPT_d' previously defined here
   OPT_d = 135,                               /* -d */
   ^
./options.h:5724:3: error: redefinition of 'OPT_D'
   OPT_D = 142,                               /* -D */
   ^
./options.h:5718:3: note: 'OPT_D' previously defined here
   OPT_D = 136,                               /* -D */
   ^
./options.h:5725:3: error: redefinition of 'OPT_d'
   OPT_d = 143,                               /* -d */
   ^
./options.h:5717:3: note: 'OPT_d' previously defined here
   OPT_d = 135,                               /* -d */
   ^
./options.h:5733:3: error: redefinition of 'OPT_E'
   OPT_E = 151,                               /* -E */
   ^
./options.h:5731:3: note: 'OPT_E' previously defined here
   OPT_E = 149,                               /* -E */
   ^
./options.h:6415:3: error: redefinition of 'OPT_H'
   OPT_H = 833,                               /* -H */
   ^
./options.h:6413:3: note: 'OPT_H' previously defined here
   OPT_H = 831,                               /* -H */
   ^
./options.h:6418:3: error: redefinition of 'OPT_I'
   OPT_I = 836,                               /* -I */
   ^
./options.h:6416:3: note: 'OPT_I' previously defined here
   OPT_I = 834,                               /* -I */
   ^
./options.h:6491:3: error: redefinition of 'OPT_MF'
   OPT_MF = 909,                              /* -MF */
   ^
./options.h:6489:3: note: 'OPT_MF' previously defined here
   OPT_MF = 907,                              /* -MF */
   ^
./options.h:6495:3: error: redefinition of 'OPT_M'
   OPT_M = 913,                               /* -M */
   ^
./options.h:6474:3: note: 'OPT_M' previously defined here
   OPT_M = 892,                               /* -M */
   ^
./options.h:6510:3: error: redefinition of 'OPT_M'
   OPT_M = 928,                               /* -M */
   ^
./options.h:6474:3: note: 'OPT_M' previously defined here
   OPT_M = 892,                               /* -M */
   ^
./options.h:6522:3: error: redefinition of 'OPT_MM'
   OPT_MM = 940,                              /* -MM */
   ^
./options.h:6517:3: note: 'OPT_MM' previously defined here
   OPT_MM = 935,                              /* -MM */
   ^
./options.h:6524:3: error: redefinition of 'OPT_MM'
   OPT_MM = 942,                              /* -MM */
   ^
./options.h:6517:3: note: 'OPT_MM' previously defined here
   OPT_MM = 935,                              /* -MM */
   ^
./options.h:6544:3: error: redefinition of 'OPT_MP'
   OPT_MP = 962,                              /* -MP */
   ^
./options.h:6538:3: note: 'OPT_MP' previously defined here
   OPT_MP = 956,                              /* -MP */
   ^                                                                                                                                                                                                                                [131/870]
./options.h:6611:3: error: redefinition of 'OPT_o'
   OPT_o = 1029,                              /* -o */
   ^
./options.h:6608:3: note: 'OPT_o' previously defined here
   OPT_o = 1026,                              /* -o */
   ^
./options.h:6613:3: error: redefinition of 'OPT_o'
   OPT_o = 1031,                              /* -o */
   ^
./options.h:6608:3: note: 'OPT_o' previously defined here
   OPT_o = 1026,                              /* -o */
   ^
./options.h:6620:3: error: redefinition of 'OPT_P'
   OPT_P = 1038,                              /* -P */
   ^
./options.h:6616:3: note: 'OPT_P' previously defined here
   OPT_P = 1034,                              /* -P */
   ^
./options.h:6728:3: error: redefinition of 'OPT_U'
   OPT_U = 1146,                              /* -U */
   ^
./options.h:6726:3: note: 'OPT_U' previously defined here
   OPT_U = 1144,                              /* -U */
   ^
./options.h:6732:3: error: redefinition of 'OPT_v'
   OPT_v = 1150,                              /* -v */
   ^
./options.h:6730:3: note: 'OPT_v' previously defined here
   OPT_v = 1148,                              /* -v */
   ^
./options.h:6774:3: error: redefinition of 'OPT_w'
   OPT_w = 1192,                              /* -w */
   ^
./options.h:6756:3: note: 'OPT_w' previously defined here
   OPT_w = 1174,                              /* -w */
   ^
gmake[7]: *** [Makefile:2507: build/gencheck.o] Error 1
gmake[7]: *** Waiting for unfinished jobs....
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_prag.adb: In function 'Sem_Prag.Collect_Subprogram_Inputs_Outputs':
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_prag.adb:26833:13: warning: 'Spec_Id' may be used uninitialized in this function [-Wmaybe-uninitialized]
/bin/sh /usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/../move-if-change tmp-gtype.state gtype.state
build/gengtype  \
                    -r gtype.state
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_util.adb: In function 'Sem_Util.Matching_Static_Array_Bounds':
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_util.adb:14805:10: warning: 'R_Index' may be used uninitialized in this function [-Wmaybe-uninitialized]
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_util.adb:14804:10: warning: 'L_Index' may be used uninitialized in this function [-Wmaybe-uninitialized]
echo timestamp > s-gtype
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_util.adb: In function 'Sem_Util.Check_Function_Writable_Actuals':
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_util.adb:2692:48: warning: 'Others_Assoc' may be used uninitialized in this function [-Wmaybe-uninitialized]
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_prag.adb: In function 'Sem_Prag.Analyze_Pragma':
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_prag.adb:11998:13: warning: 'Str' may be used uninitialized in this function [-Wmaybe-uninitialized]
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_prag.adb:13312:13: warning: 'Typ' may be used uninitialized in this function [-Wmaybe-uninitialized]
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_prag.adb:15233:13: warning: 'Typ' may be used uninitialized in this function [-Wmaybe-uninitialized]
/usr/ports/lang/gcc6-aux/work/gcc-6.2.0/gcc/ada/sem_prag.adb:22791:25: warning: 'Chr' may be used uninitialized in this function [-Wmaybe-uninitialized]
rm gfortran.pod gcc.pod
gmake[7]: Leaving directory '/usr/ports/lang/gcc6-aux/work/build/gcc'
gmake[6]: *** [Makefile:4128: all-gcc] Error 2
gmake[6]: Leaving directory '/usr/ports/lang/gcc6-aux/work/build'
gmake[5]: *** [Makefile:876: all] Error 2
gmake[5]: Leaving directory '/usr/ports/lang/gcc6-aux/work/build'
*** Error code 2

Stop.
make[4]: stopped in /usr/ports/lang/gcc6-aux
*** Error code 1

Stop.
make[3]: stopped in /usr/ports/lang/gcc6-aux
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/misc/ini_file_manager
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/misc/ini_file_manager
*** Error code 1

Stop.
make: stopped in /usr/ports/ports-mgmt/synth

No problems building under 10.3-RELEASE. I'm running i386, if that makes a difference.

marino · Sep 27, 2016

strange, that is news to me.
FWIW, it's building in the cluster: http://portsmon.freebsd.org/portoverview.py?category=lang&portname=gcc6-aux
that's my only "test" for i386 though, I don't have any 32-bit boxes around

Wapcaplet · Sep 28, 2016

I think I discovered the problem. Apparently there is a known issue with GCC and locales in 11.0:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=210122

I tried the workaround listed there, make LC_COLLATE="C" install clean, and it worked. Should the patch in that PR be applied to lang/gcc6-aux as well?

Edit: Apparently, you commented on this other PR that referenced the same issue:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=211742

marino · Sep 28, 2016

yeah, because in my opinion it's not an individual port issue, but a global ports one. It needs a fix in the framework.

But Synth sets the locale doesn't it? You shouldn't have seen it in synth. (it doesn't set LC_COLLATE=C but LANG=C but the effect should be the same)

Wapcaplet · Sep 28, 2016

I tried to install everything from scratch. You're probably right that it works within synth (I'll test that later today or tomorrow).

There is a bit of a chicken-and-egg problem with this bug when you first install synth. Your first-time installl options are to run either pkg install synth or make -C /usr/ports/ports-mgmt/synth install clean, and if you've changed the locale variables, the make command will fail at lang/gcc6-aux without the workaround. Everyone who wants to try synth for the first time under 11.0 will hit this problem if they compile ports with non-default locale environmental variables.

marino · Sep 28, 2016

true. On a bare system, synth should be installed via official package if possible. Mainly because the gcc6-aux build requirement is a heavy one.

Somebody with strict "I-must-build-everything-myself" requirements could do this:

pkg ins synth
synth build ports-mgmt/synth and answer "yes" to new repository
pkg remove synth
pkg add <path-to-synth-repo>/synth-<version>.txz

and then go on as normal.
I think such a process is overkill, but that's how you'd bootstrap synth. I don't think people should every build "live" other than ports-mgmt/pkg and even that can be bootstrapped.

kpa · Sep 28, 2016

I'd say the big question is if the ports(7) system should clean the environment for the builds or allow environment to be passed to the builds and how. Now it just allows everything by the default environment inheritance by child processes but this is obviously wrong for package building if you want to guarantee predictable results.

marino · Sep 28, 2016

you basically just answered your own big question. Yes, the environment should be as uniform as possible and it shouldn't be that hard to enforce.