Solved Firefox, Synth or not and LTO

Zvoni

Active Member

Reaction score: 85
Messages: 159

For terminal session, I am always (many years) using sysutils/screen. Very convenient. Screen lets you to disconnect from one terminal session and reconnect from another. You can try building your ports under screen. After starting build, you can disconnect terminal.
See - screen(1)

For me screen is a must-have on any FreeBSD installation from VM-s, servers to desktop. Wonder why it is not included in base...

View attachment 11263
An alternative to screen is tmux
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 314
Messages: 586

Yes, I switch from screen to tmux about a year ago. Both very useful.
 

Argentum

Aspiring Daemon

Reaction score: 236
Messages: 510

An alternative to screen is tmux
Installed tmux also. Cool! I can run screen under tmux :cool:
Screenshot from 2021-09-09 11-51-31.png
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 314
Messages: 586

In the end, for those who are interested, I got it sorted out.

Or to be more sincere, I did not. I filed a PR for firefox, and the developer/committer recommended I disable Link-Time Optimization in the config.

It worked a treat. Built from beginning to end with all dependencies in about twenty minutes.

I note that LTO is not enabled by default in thunderbird and chromium, and have not had a problem building those.
 

Zirias

Daemon

Reaction score: 1,423
Messages: 2,480

Well, I noticed LTO requires much more resources to build, but for me, building firefox still works when not built in parallel with other huge ports. I assume someone did some measurements how much firefox benefits from LTO before enabling that option by default, cause, you really don't do such things "just because" :what:

Anyways, disabling the option sounds more like a workaround. If you don't run out of RAM (which seems unlikely with the machine you described), the build really should complete when given enough time. If not, there's probably still something else broken. 😳
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 314
Messages: 586

Well, I noticed LTO requires much more resources to build.

I have built again, within synth, and with LTO. It hung - or appeared to - and then after nearly two hours, sprang into life and competed.

LTO seems to be having the opposite of desired effect.

I am quite happy disabling it.
 

Zirias

Daemon

Reaction score: 1,423
Messages: 2,480

The benefit of LTO is at runtime, not at compile time. It basically means to apply some suitable optimizations that are normally done during compilation also during linking of all the already compiled modules. Yes, this is very resource intensive for a large binary consisting of a huge amount of modules.
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 314
Messages: 586

The benefit of LTO is at runtime, not at compile time. It basically means to apply some suitable optimizations that are normally done during compilation also during linking of all the already compiled modules. Yes, this is very resource intensive for a large binary consisting of a huge amount of modules.

I see.
 

Zirias

Daemon

Reaction score: 1,423
Messages: 2,480

Hmm... this is interesting. I do enable LTO whenever I can. Zirias just confirmed what I was thinking all along was the good reason to do that.
The key question is of course whether the benefit at runtime is worth the pretty heavy impact on build cost. That's why I hope someone did some measurements before enabling that option by default ;)
 

Tieks

Well-Known Member

Reaction score: 108
Messages: 254

The key question is of course whether the benefit at runtime is worth the pretty heavy impact on build cost.
In case of Firefox I doubt it. Plenty of time to do things while waiting for the network.
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 314
Messages: 586

On a nice big flashy machine, building all ports together, may take less than an hour, with no LTO, or about three odd hours with LTO on those ports that have it. That is still acceptable. But for probably imperceptible increase in performance of those programs.

On an old slow machine, such as my laptop, enabling LTO may result in days worth of building. (I only use packages on my laptop.)

It could be that for building from ports, LTO only helps if you don't need it.
 

Zirias

Daemon

Reaction score: 1,423
Messages: 2,480

In case of Firefox I doubt it. Plenty of time to do things while waiting for the network.
Maybe. I don't know. Things like the performance of the javascript engine are important for today's browsers.

What LTO almost always achieves is a smaller binary size. But sure, on today's machines, you could say "who cares" 🤷‍♂️
 

Zirias

Daemon

Reaction score: 1,423
Messages: 2,480

It could be that for building from ports, LTO only helps if you don't need it.
That's why you use poudriere on a "beefy" machine to build your package repo and install the resulting packages on the weak machines 😏
 

astyle

Aspiring Daemon

Reaction score: 386
Messages: 871

The key question is of course whether the benefit at runtime is worth the pretty heavy impact on build cost. That's why I hope someone did some measurements before enabling that option by default ;)
I'd say that if a pre-compiled package was prepared with LTO by default, then it's worth it. Then anyone using pkg to install it will benefit. If you build the port yourself, the real question would be, IMHO, if they have the patience to wait for the port to finish compiling.
 

Argentum

Aspiring Daemon

Reaction score: 236
Messages: 510

On a nice big flashy machine, building all ports together, may take less than an hour, with no LTO, or about three odd hours with LTO on those ports that have it. That is still acceptable. But for probably imperceptible increase in performance of those programs.

On an old slow machine, such as my laptop, enabling LTO may result in days worth of building. (I only use packages on my laptop.)

It could be that for building from ports, LTO only helps if you don't need it.
I am trying now to build Firefox with LTO. What I can see is that the build is running on a single core. So it only utilizes the fraction of available performance. The good thing is that I can use the machine with almost no impact while the build runs on the background. Let's see how long it takes...
 

Argentum

Aspiring Daemon

Reaction score: 236
Messages: 510

I am trying now to build Firefox with LTO. What I can see is that the build is running on a single core. So it only utilizes the fraction of available performance. The good thing is that I can use the machine with almost no impact while the build runs on the background. Let's see how long it takes...
... and the result of this experiment is that build time was 1.6 times longer, but the CPU time consumed was only 5% more.
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 314
Messages: 586

Now knowing that it was LTO, and it only appears to hang, I will try building all of chromium, thunderbird and firefox, all with LTO enabled, and all in parallel within synth. I have disabled the timeout in synth and will go out and not watch it.

Before starting synth, the machine was running with xfce and various programs, and the load average was 0.25.
10:30AM up 6 days, 13:48, 1 user, load averages: 0.25, 0.51, 1.17

In a terminal window synth force www/firefox www/chromium mail/thunderbird



After ten minutes
10:41AM up 6 days, 13:59, 1 user, load averages: 125.14, 85.96, 64.00



Having gone out for a couple of hours, when I came back, it was complete with all three built successfully.
From the log files and can see that thunderbird took about 35 minutes, chromium about 100 and firefox less than 90 minutes - peculiarly, faster than when I built it on its own with LTO. I did not watch to see if any appeared to hang.
 

Argentum

Aspiring Daemon

Reaction score: 236
Messages: 510

Now knowing that it was LTO, and it only appears to hang, I will try building all of chromium, thunderbird and firefox, all with LTO enabled, and all in parallel within synth. I have disabled the timeout in synth and will go out and not watch it.
Tried to build also Thunderbird with LTO yesterday ( make) and the build failed...
 

Zirias

Daemon

Reaction score: 1,423
Messages: 2,480

The most likely reason for that to fail is indeed some resource problem (RAM, CPU, ...) – and like any optimization, it's questionable whether it's worth the hassle. But I agree with astyle – if there's even a slight advantage for firefox, enabling it by default is probably a good decision as everyone using binary packages will profit.
 

Argentum

Aspiring Daemon

Reaction score: 236
Messages: 510

The most likely reason for that to fail is indeed some resource problem (RAM, CPU, ...) – and like any optimization, it's questionable whether it's worth the hassle. But I agree with astyle – if there's even a slight advantage for firefox, enabling it by default is probably a good decision as everyone using binary packages will profit.
Had enough RAM and monitored during build. Never rised even to 50%. Some other reason. Just before that I build Firefox (which is bigger) and it completed OK. Should have been some other problem...
 
OP
Geezer

Geezer

Aspiring Daemon

Reaction score: 314
Messages: 586

Tried to build also Thunderbird with LTO yesterday ( make) and the build failed...
You are correct. When I looked afterwards, thunderbird had been built without LTO, and when I tried I again got an error.

I have also built libreoffice with LTO.
 

Argentum

Aspiring Daemon

Reaction score: 236
Messages: 510

You are correct. When I looked afterwards, thunderbird had been built without LTO, and when I tried I again got an error.

I have also built libreoffice with LTO.
My hypothesis here is that the LTO breaks something. It was a linker error (although I did not capture it). Perhaps LTO optimizes too heavily and removes something that prevents successful linking. Anyway, I do not believe that LTO can be a big benefit...
 

Zirias

Daemon

Reaction score: 1,423
Messages: 2,480

My hypothesis here is that the LTO breaks something. It was a linker error (although I did not capture it). Perhaps LTO optimizes too heavily and removes something that prevents successful linking.
Without a log, there's not much to analyze. If you use poudriere, you'll always have a complete log ;)

If building with LTO is broken, a PR would make sense. It should be fixed, or, if not possible, the option should be removed.

Anyway, I do not believe that LTO can be a big benefit...
You can never tell, it depends a lot on the actual software built. The only way to tell is benchmarking ;)
 

Argentum

Aspiring Daemon

Reaction score: 236
Messages: 510

Without a log, there's not much to analyze. If you use poudriere, you'll always have a complete log ;)
Of course, I can run this once again, one day. But now OP also wrote that the build failed. Everybody can try this...


You can never tell, it depends a lot on the actual software built. The only way to tell is benchmarking ;)
Agree. How to benchmark a web browser? There are online tests available, but to get a reliable reading seems hard. I have not seen any e-mail client benchmarks...
 
Top