www/firefox 'too few semaphores' configure error

Y

YuryG

Using poudriere I tried to upgrade www/firefox, but it gives an error during configure stage:
Code: 
memory/jemalloc/src> config.status: executing include/jemalloc/jemalloc.h commands

Creating config.status
Traceback (most recent call last):
  File "/wrkdirs/usr/ports/www/firefox/work/firefox-53.0/configure.py", line 124, in <module>
    sys.exit(main(sys.argv))
  File "/wrkdirs/usr/ports/www/firefox/work/firefox-53.0/configure.py", line 34, in main
    return config_status(config)
  File "/wrkdirs/usr/ports/www/firefox/work/firefox-53.0/configure.py", line 119, in config_status
    return config_status(args=[], **encode(sanitized_config, encoding))
  File "/wrkdirs/usr/ports/www/firefox/work/firefox-53.0/python/mozbuild/mozbuild/config_status.py", line 136, in config_status
    reader = BuildReader(env)
  File "/wrkdirs/usr/ports/www/firefox/work/firefox-53.0/python/mozbuild/mozbuild/frontend/reader.py", line 886, in __init__
    self._gyp_worker_pool = ProcessPoolExecutor(max_workers=max_workers)
  File "/wrkdirs/usr/ports/www/firefox/work/firefox-53.0/python/futures/concurrent/futures/process.py", line 274, in __init__
    _check_system_limits()
  File "/wrkdirs/usr/ports/www/firefox/work/firefox-53.0/python/futures/concurrent/futures/process.py", line 263, in _check_system_limits
    raise NotImplementedError(_system_limited)
NotImplementedError: system provides too few semaphores (30 available, 256 necessary)
*** Fix above errors and then restart with\
               "gmake -f client.mk build"
gmake[1]: *** [/wrkdirs/usr/ports/www/firefox/work/firefox-53.0/client.mk:383: configure] Error 1
gmake[1]: Leaving directory '/usr/ports/www/firefox'
===>  Script "configure" failed unexpectedly.
Please report the problem to gecko@FreeBSD.org [maintainer] and attach the
"/wrkdirs/usr/ports/www/firefox/work/firefox-53.0/config.log" including the
output of the failure of your make command. Also, it might be a good idea to
provide an overview of all packages installed on your system (e.g. a
/usr/local/sbin/pkg-static info -g -Ea).
*** Error code 1
What can I do?
 
Rebuild kernel because of a port???
So, less and less kernel options are good (or at least equal to corresponding modules).
 
If the sem(4) module is loaded, or the corresponding functionality has statically been linked into the kernel, the number of available semaphores can be increased on the fly using the
Code: 
p1003_1b.sem_nsems_max
sysctl(8) tuneable. I don't know at what overhead this comes, but I assume it to be neglectible when compared to the resources usually consumed by firefox.

Increasing it to more than 256 fixes the build of firefox over here without having to reboot the system.
 
@YuriG
20170411:
AFFECTS: www/firefox
AUTHOR: gecko@FreeBSD.org

Firefox 53 or later may fail to build if sem(4) is present (see below)
because sysconf(3) still consults the old limits. If you don't plan to
run FreeBSD 8.x or older binaries make sure sem(4) isn't loaded during
boot or built statically into the kernel.

Creating config.status
Traceback (most recent call last):
File "configure.py", line 124, in <module>
sys.exit(main(sys.argv))
File "configure.py", line 34, in main
return config_status(config)
File "configure.py", line 119, in config_status
return config_status(args=[], **encode(sanitized_config, encoding))
File "python/mozbuild/mozbuild/config_status.py", line 136, in config_status
reader = BuildReader(env)
File "python/mozbuild/mozbuild/frontend/reader.py", line 886, in __init__
self._gyp_worker_pool = ProcessPoolExecutor(max_workers=max_workers)
File "python/futures/concurrent/futures/process.py", line 274, in __init__
_check_system_limits()
File "python/futures/concurrent/futures/process.py", line 263, in _check_system_limits
raise NotImplementedError(_system_limited)
NotImplementedError: system provides too few semaphores (30 available, 256 necessary)
Click to expand...
 
UPDATING was extended with a record on this issue in revision 438359, although this happened only after the firefox port was updated in revision 438264, so there was a timeslot (of about one day) with the sem(4) limited firefox in ports but without it's corresponding UPDATING record.

drhowarddrfine , As sem(4) is critical to me, and probably also others, I posted another option to build firefox, in despite of having seen the UPDATING record.
 
By the way, inserting notes not at the top of UPDATING file is a very bad practice, imho. People used to read it everytime they are updating ports' tree, do not feel need to read it twice, reading after already read records.
 
xibo said:
If the sem(4) module is loaded, or the corresponding functionality has statically been linked into the kernel, the number of available semaphores can be increased on the fly using the
Code: 
p1003_1b.sem_nsems_max
sysctl(8) tuneable. I don't know at what overhead this comes, but I assume it to be neglectible when compared to the resources usually consumed by firefox.

Increasing it to more than 256 fixes the build of firefox over here without having to reboot the system.
Click to expand...
Will be much obliged if you told me how this sysctl variable can be set to anything but "0".
Not at least using a normal sysctl command like sysctl p1003_1b.sem_nsems_max=512 , cause that would return:
Code: 
p1003_1b.sem_nsems_max: 0 -> 0

EDIT: Sorry, this is because sem.ko is NOT loader here, of course. But this is exactly what throws into confusion: /usr/ports/UPDATING says www/firefox
Code: 
"may fail to build if sem(4) is present... make sure sem(4) isn't loaded during boot or built statically into the kernel."
Well then, sem is NOT loaded on my system, yet the build fails. Presumably I can load it and set the above variable as mentioned above (haven't tried yet), but what's the meaning of it all??
 
xibo said:
If the sem(4) module is loaded, or the corresponding functionality has statically been linked into the kernel, the number of available semaphores can be increased on the fly using the
Code: 
p1003_1b.sem_nsems_max
sysctl(8) tuneable. I don't know at what overhead this comes, but I assume it to be neglectible when compared to the resources usually consumed by firefox.

Increasing it to more than 256 fixes the build of firefox over here without having to reboot the system.
Click to expand...
Well, with sem.ko loaded and the limit set above 256 the build fails with the following message:
Code: 
Traceback (most recent call last):
  File "/construction/xports/www/firefox/work/firefox-53.0/configure.py", line 124, in <module>
    sys.exit(main(sys.argv))
  File "/construction/xports/www/firefox/work/firefox-53.0/configure.py", line 34, in main
    return config_status(config)
  File "/construction/xports/www/firefox/work/firefox-53.0/configure.py", line 119, in config_status
    return config_status(args=[], **encode(sanitized_config, encoding))
  File "/construction/xports/www/firefox/work/firefox-53.0/python/mozbuild/mozbuild/config_status.py", line 136, in config_status
    reader = BuildReader(env)
  File "/construction/xports/www/firefox/work/firefox-53.0/python/mozbuild/mozbuild/frontend/reader.py", line 886, in __init__
    self._gyp_worker_pool = ProcessPoolExecutor(max_workers=max_workers)
  File "/construction/xports/www/firefox/work/firefox-53.0/python/futures/concurrent/futures/process.py", line 285, in __init__
    EXTRA_QUEUED_CALLS)
  File "/usr/local/lib/python2.7/multiprocessing/__init__.py", line 217, in Queue
    from multiprocessing.queues import Queue
  File "/construction/xports/www/firefox/work/firefox-53.0/build/mach_bootstrap.py", line 374, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/usr/local/lib/python2.7/multiprocessing/queues.py", line 48, in <module>
    from .synchronize import Lock, BoundedSemaphore, Semaphore, Condition
  File "/construction/xports/www/firefox/work/firefox-53.0/build/mach_bootstrap.py", line 374, in __call__
    module = self._original_import(name, globals, locals, fromlist, level)
  File "/usr/local/lib/python2.7/multiprocessing/synchronize.py", line 59, in <module>
    " function, see issue 3770.")
ImportError: This platform lacks a functioning sem_open implementation, therefore, the required synchronization primitives needed will not function, see issue 3770.
*** Fix above errors and then restart with\
               "gmake -f client.mk build"
gmake[1]: *** [/construction/xports/www/firefox/work/firefox-53.0/client.mk:383: configure] Error 1
 
And, contrary to my initial suspicions, this has nothing to do with python shebang issue discussed elsewhere here just recently and which was kind of fixed by commenting out DEVELOPER=yes in /etc/make.conf. That one had to do with putting the python intepreter version into scripts, if I understand it correctly.

However, www/firefox port specifically mentions lang/python27 as its build dependency, and that one is by default built with semaphores disabled. Which seems to be the cause of the build configure error above.
 
I seem to be spending too much time of recently with various ports failing to compile while other ports stop functioning after upgrades. Even full rebuild of ports didn't heal this situation.
 
free-and-bsd said:
And, contrary to my initial suspicions, this has nothing to do with python shebang issue discussed elsewhere here just recently and which was kind of fixed by commenting out DEVELOPER=yes in /etc/make.conf. That one had to do with putting the python intepreter version into scripts, if I understand it correctly.

However, www/firefox port specifically mentions lang/python27 as its build dependency, and that one is by default built with semaphores disabled. Which seems to be the cause of the build configure error above.
Click to expand...
All right, I finally succeeded with www/firefox build after rebuilding lang/python27 with SEM option enabled. The funny thing is (cannot be helped, I guess) the total of 140 related packages were rebuilt (synth), but only 3 of them were upgraded in my installation.
 
free-and-bsd said:
…the total of 140 related packages were rebuilt (synth), but only 3 of them were upgraded in my installation.
Click to expand...
Python is a huge dependency, and changing its options make synth to rebuild many of the staff. By the way, SEM is by default ON for quite a long time for python. May be you had old options?
Code: 
OPTIONS_DEFAULT=        LIBFFI PYMALLOC SEM THREADS UCS4
 
YuryG said:
Python is a huge dependency, and changing its options make synth to rebuild many of the staff. By the way, SEM is by default ON for quite a long time for python. May be you had old options?
Code: 
OPTIONS_DEFAULT=        LIBFFI PYMALLOC SEM THREADS UCS4
Click to expand...
That's possible, for I've kept my OPTIONS from the time I switched over to FreeBSD (9.0 at the time) from Linux. Kept them in order to avoid things like HAL from infiltrating my system without me knowing about it. And I normally don't disable default options unless it is to get rid of some dependency I don't want on my system.

Anyways, I can see now why ports-mgmt/synth is designed to remove old options when defaults change in a port.
 
You must log in or register to reply here.
Back
Top