Solved Firefox, Synth or not and LTO

Geezer

Aspiring Daemon

Reaction score: 447
Messages: 765

I mainly use packages, but a few I compile from the ports because I want different options. All quarterly.

I have tried synth (a couple of times) as it seems to do exactly what I want, build recursively, create repo, register packages, and has a nice colourful curses display. A whole lot quicker and simpler to get going than poudriere.

It works great. Mostly. Except for three packages, iwmbt-firmware, chromium and firefox.

When I build these three ports on their own:

iwmbt-firmware does not fetch the file from where ever and fails. In synth, this is a fail. This package is not too important anyway.

chromium, the makefile gives an unnecessary warning (as in another thread), and in synth this is an ignore.

firefox does build but takes a bloody long time. In synth it builds for an hours and then stops with a fail.

The two packages that take the longest to build don't work in synth, which really spoils the point if it.

Anyone else had good luck/bad luck with synth?
 

johngrauvard

Member

Reaction score: 31
Messages: 22

I don't use synth, but I do use ports. firefox and firefox-esr have both failed during compilation for me after their recent updates, so I'm not sure if it's an issue peculiar to synth or the ports themselves.
 

Tieks

Well-Known Member

Reaction score: 126
Messages: 306

firefox and firefox-esr have both failed during compilation for me after their recent updates
Building firefox-esr from ports recently I got some error 'stcck-protector not found'. Which should be stack-protector of course, but that term didn't show up in the port files. I switched off config option OPTIMIZED_CFLAGS, then the build succeeded. After that I ran memtest for some time, but that didn't show errors.
 

johngrauvard

Member

Reaction score: 31
Messages: 22

Building firefox-esr from ports recently I got some error 'stcck-protector not found'. Which should be stack-protector of course, but that term didn't show up in the port files. I switched off config option OPTIMIZED_CFLAGS, then the build succeeded. After that I ran memtest for some time, but that didn't show errors.
I see, I had that option enabled for both and the error looks familiar. When I work up the nerve to install it from ports again I'll disable the option and see if that does it. It wasn't an issue before the recent updates to both of them, wondering if it's a firefox thing
 

Styrsven

New Member

Reaction score: 15
Messages: 15

I build everything with synth, and I do have OPTIMIZED_CFLAGS and LTO options set. I have fairly powerful CPU for a desktop, but sometimes when synth builds firefox in parallel with something heavy/slow, like electron or thunderbird or libreoffice or some combination of those, it fails. The log file says it is a timeout. So I think the firefox build system has a timeout, it is probably configurable and if so could maybe be fixed by a change to the port, but I am only guessing.
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 447
Messages: 765

Yes, a timeout. Tried it a dozen times, more builders few builders, more jobs, few jobs, with tmpfs, without.

Always - but not necessarily in same place - it stops and then times out. Only firefox. All the other ports get done.

Dunno why, but broadly, and unfortunately, it renders synth unusable. Great pity.
 

Argentum

Aspiring Daemon

Reaction score: 296
Messages: 623

Yes, a timeout. Tried it a dozen times, more builders few builders, more jobs, few jobs, with tmpfs, without.

Always - but not necessarily in same place - it stops and then times out. Only firefox. All the other ports get done.

Dunno why, but broadly, and unfortunately, it renders synth unusable. Great pity.
Just try a regular build.

Code:
cd /usr/ports/www/firefox/
make config
make install clean

You can also use ports-mgmt/portupgrade portinstall to build.

I built the latest (firefox-92.0_2,2) just today. Everything seems to be OK.
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 447
Messages: 765

I have had to change the subject of this thread. It is not merely synth.

Have not tried portupgrade yet, but have tried portmaster, which in general I like.
Same thing. Building firefox hangs.

In both portmaster and synth, building firefox always hangs. Not necessarily in the same place. But always. Tried changing the options. Does not matter.

But regularly building the port without any port management program always works!
 

Zirias

Son of Beastie

Reaction score: 1,698
Messages: 2,869

Quite a while ago, I've seen github pull requests aimed to allow poudriere to use binary packages while building. I don't know what's the state of this, but it would finally add the one feature people seem to like most about synth. Using poudriere has one advantage: It's well-tested, doing even the official package builds :cool:
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,704
Messages: 39,305

I don't know what's the state of this
Code:
     -b name  Specify the name of the binary package branch to use to prefetch
              packages.  Should be "latest", "quarterly", "release_*", or url.
              With this option poudriere will first try to fetch from the
              binary package repository specified the binary package prior to
              do the sanity check if the package does not already exist.

              poudriere will only use packages that:
              •   come from a repository having the same or older version of
                  pkg.
              •   do not have a locally fetched package already.
              •   are not IGNORED.
              •   match the expected local version.
              •   match the expected ABI.
              •   match the expected runtime and library dependencies.
              •   match the expected OPTIONS when CHECK_CHANGED_OPTIONS is
                  enabled (default: on).
              •   The package is NOT listed in PACKAGE_FETCH_BLACKLIST
              •   The package is NOT listed with -C, or -c, when -t is used.
              The -v flag can be used to show these decisions during build.
              Specifing twice will show more details on why some are skipped.

              !!

              poudriere has no way of determining differences outside of the
              above list.  That is, if the local ports framework, or port, has
              custom patches or special WITH_FOO knobs (not OPTIONS) then it
              is required to add its name into PACKAGE_FETCH_BLACKLIST.
              Otherwise a package may be fetched and used that lacks the
              custom patch or knob.

              !!

              See PACKAGE_FETCH_BRANCH, PACKAGE_FETCH_URL,
              PACKAGE_FETCH_BLACKLIST, and PACKAGE_FETCH_WHITELIST in
              poudriere.conf.sample.  The entries in the lists will be matched
              against package names without versions.
poudriere-bulk(8)
(Option seems to be available on ports-mgmt/poudriere-devel only)
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 447
Messages: 765

Argentum Doing portupgrade -R www/firefox now. Wait an hour or so and I will update.

Zirias Yes, I have been trying to stay clear of poudiere. I understand that it is good, that's not the point. I am trying to simplify my maintenance of ports/packages, and I do not really want to maintain a jail that would add nothing in itself to my setup.

Synth would have been the best. Sit back and watch multiple ports get upgraded in glowing colour. I do not know where the problem lies, in synth or in the firefox port.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,704
Messages: 39,305

and I do not really want to maintain a jail that would add nothing in itself to my setup.
Not a lot to maintain actually. Just run poudriere jail -u whenever there are patches. That's all the "management" you need to do.
 

Zirias

Son of Beastie

Reaction score: 1,698
Messages: 2,869

Geezer Synth uses jails as well. The only difference is that it derives them from the running system, while poudriere works with a defined base jail (but comes with tools to create and maintain them with simple single commands). This approach allows to also build packages for a different version that the host's.

Both use colors and also offer a web frontend, if that's important :cool:
 

forquare

Well-Known Member

Reaction score: 186
Messages: 335

Quite a while ago, I've seen github pull requests aimed to allow poudriere to use binary packages while building. I don't know what's the state of this, but it would finally add the one feature people seem to like most about synth.
Oh that's very cool. Looks like the PR is due to be part of 3.4.0, although it also looks like there will be a version 3.3.8 before then.
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 447
Messages: 765

Oh dear. It stopped. Right in the middle.

Specifically, the last thing that came up that is has hung at is
Code:
/usr/local/bin/clang++12 -std=gnu++17 -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-strong -fstack-clash-protection -Qunused-arguments -DLIBICONV_PLUG -isystem /usr/local/include -Qunused-arguments -Wall -Wbitfield-enum-conversion -Wdeprecated-this-capture -Wempty-body -Wformat-type-confusion -Wignored-qualifiers -Wpointer-arith -Wshadow-field-in-constructor-modified -Wshadow-uncaptured-local -Wsign-compare -Wtype-limits -Wno-error=tautological-type-limit-compare -Wunreachable-code -Wunreachable-code-return -Wno-invalid-offsetof -Wclass-varargs -Wempty-init-stmt -Wfloat-overflow-conversion -Wfloat-zero-conversion -Wloop-analysis -Wno-range-loop-analysis -Wc++2a-compat -Wcomma -Wenum-compare-conditional -Wimplicit-fallthrough -Wstring-conversion -Wno-inline-new-delete -Wno-error=deprecated-declarations -Wno-error=array-bounds -Wno-error=backend-plugin -Wno-error=free-nonheap-object -Wno-error=return-std-move -Wno-error=atomic-alignment -Wno-error=deprecated-copy -Wformat -Wformat-security -Wno-gnu-zero-variadic-macro-arguments -Wno-psabi -Wno-unknown-warning-option -fno-sized-deallocation -fno-aligned-new -O2 -pipe -O3 -DLIBICONV_PLUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -DLIBICONV_PLUG -isystem /usr/local/include -fno-exceptions -fno-strict-aliasing -fPIC -fno-rtti -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pipe -O2 -O3 -fno-omit-frame-pointer -funwind-tables  -shared -Wl,-z,defs -Wl,--warn-unresolved-symbols -Wl,--gc-sections -Wl,-h,libxul.so -o libxul.so /usr/ports/www/firefox/work/.build/toolkit/library/build/libxul_so.list  -flto=thin -Wl,-plugin-opt=-import-instr-limit=10 -Wl,-plugin-opt=new-pass-manager -Wl,-plugin-opt=-import-hot-multiplier=30 -pthread -Wl,--as-needed -fstack-protector-strong -Wl,-z,noexecstack -Wl,-z,text -Wl,-z,relro -Wl,-z,nocopyreloc -Wl,-Bsymbolic-functions -Wl,--build-id=sha1 -fstack-protector-strong -fstack-clash-protection -Wl,-rpath-link,/usr/ports/www/firefox/work/.build/dist/bin -Wl,-rpath-link,/usr/local/lib  ../../../js/src/build/libjs_static.a /usr/ports/www/firefox/work/.build/x86_64-unknown-freebsd/release/libgkrust.a ../../../config/external/lgpllibs/liblgpllibs.so ../../../config/external/sqlite/libmozsqlite3.so ../../../widget/gtk/mozgtk/libmozgtk.so ../../../widget/gtk/mozwayland/libmozwayland.so   -L/usr/local/lib -licui18n -L/usr/local/lib -licuuc -licudata -laom -ldav1d -lX11 -lX11-xcb -lxcb -lXcomposite -lXcursor -lXdamage -lXext -lXfixes -lXi -lXrender -lpthread -lffi -lplds4 -lplc4 -lnspr4 -pthread -ldl -lz -lm -lnss3 -lsmime3 -lssl3 -lnssutil3 -lfreetype -lfontconfig -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lintl -lutil -lpng -lwebpdemux -lwebp -lgraphite2 -levent -lvpx -lpixman-1 -ldbus-glib-1 -ldbus-1 -lxcb-shm

Although, it seems to have hung at different places each time it has hung.

So far:
synth, portmaster, portupgrade, none of them build firefox.
Building the port directly with make works every time.
poudriere, yet to try.

But I would like to know why it is failing in all these ports management programs.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,704
Messages: 39,305

Although, it seems to have hung at different places each time it has hung.
There is a difference between hanging and failing. You mentioned before that the build fails. When it fails, what error does it fail on?
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 447
Messages: 765

There is a difference between hanging and failing. You mentioned before that the build fails. When it fails, what error does it fail on?
Yes, there is a difference. I beg your pudding.

In all cases, it has hung. At different points.

In synth, after hanging and then timing out, it is reported as a failure.
 

Zirias

Son of Beastie

Reaction score: 1,698
Messages: 2,869

If it just hangs, it's unlikely a different build tool would help. Building these huge ports requires a somewhat powerful machine. How is CPU and I/O load during the "hang"? What about RAM (and swap) usage?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 12,704
Messages: 39,305

Yes, is it just possible it doesn't actually "hang" but it's just busy compiling? Maybe Synth thinks it takes too long and then breaks it off. Is there a time-out setting in Synth you can increase?

Building firefox takes a long time, Rust alone typically takes around 3-4 hours to complete on my "build" server (old Core i5, 4 threads, 16GB memory).
 
Top