Synth: Introducing new custom package repository builder for FreeBSD and DragonFly

the items are not "obsolete".
They are likely build dependencies.
For example, "gmake". You need "gmake" to build a package, but you don't need it after that.
If you deleted "gmake" package, synth would have to build it again.
Synth deletes truly obsolete packages (older versions) when the repository is generated, so the actual repository is 100% current.

Bottom line: the packages you don't use are not worth worrying about. I'm surprised the number is so low. I would expect a lot more than 3 packages out of 152 to be "build-only".
 
Ok, I just did a compare and these are the files left over :
Code: 
jpeg-8_7.txz
nasm-2.11.08_1,1.txz
liblz4-1.7.5,1.txz
jpeg is not longer used and was replaced with jpeg-turbo (so that's not a build dependency)

I'm not sure what the other files are. I won't worry about though.
 
Hy, I'm not at my FreeBSD computer but maybe you have a good suggestion. I try to install py-pytoport. synth status ports-mgmt/py-pytoport show me that it is marked with N, so it should be installed. But if I run synth install ports-mgmt/py-pytoport , it didn't compiled it and so pkg can't find a packages. Do you need some further informations?
I haven't a custom kernel or anything else only a few ports installed through synth.
 
There are several logs in the log directory that start with the number zero. Reviewing these overall build logs usually answers questions like this. With no real details, I'd say py-pytoport was either skipped or ignored. The reason why would be available in the logs.
 
Oh, by the way, there is a detailed HTML report with the complete build history in the "Report" subdirectory of the log directory. It will also clearly show skips and ignores and is searchable/filterable. It is always reflecting the ongoing or last build, then overwritten. If you aren't aware of this web report, you should check it out.
 
Thanks, with your advice I found the problem really fast. Maybe a verbose output would be great, unfortunately I found no hints in the man page.
But the problem comes from the Makefile itself and from the fact that a package should be built.
Code: 
.if defined(PACKAGE_BUILDING) && !defined(PACKAGE_BUILDING_FLAVORS) && \
    ${PYTHON_VER} != ${PYTHON_DEFAULT}
IGNORE= you have python ${PYTHON_DEFAULT} set as the default, and this needs ${PYTHON_VER}
.endif
Would it be useful to set PACKAGE_BUILDING_FLAVORS automatically from synth? According to this https://svnweb.freebsd.org/ports/head/ports-mgmt/py-pytoport/Makefile?view=log the package build is disabled because of poudriere.
 
it's a known flaw with ports that won't be properly corrected until flavors/subpackages is implemented. poudriere by now probably implements the rather nasty framework workaround, but I'm not going to. Without looking, I'm guessing other ports get around this using slave ports. I don't really have a suggestion for you regarding this known issue.
 
I assume that the modifications to the other gcc ports to block the installation of fixed-headers also needs to be applied to gcc-aux ports. Anybody can request this; they are unmaintained ports.
 
I took a look at the patches for the other gcc ports, and it looks like the patches just delete the /include-fixed directory during post-stage. lang/gcc6-aux already does this if the BOOTSTRAP option is turned on, so should that line be taken out of the BOOTSTRAP option requirement and simply made mandatory during post-stage?

This is a bit above my knowledge level, unfortunately.
 
Wapcaplet said:
so should that line be taken out of the BOOTSTRAP option requirement and simply made mandatory during post-stage?
Click to expand...

That sounds about right. I haven't looked at it, but yeah, the change would be to remove fixed headers unconditionally.
 
Is this a bug or something else?

I noticed when I updated my last 3 ports (each on separate days), the log file shows that PKG is not found and installs it. PKG has always existed on the system though.
Code: 
--------------------------------------------------------------------------------
--  Phase: pkg-depends
--------------------------------------------------------------------------------
===>   py27-pyinotify-0.9.6 depends on file: /usr/local/sbin/pkg - not found
===>   Installing existing package /packages/All/pkg-1.10.1.txz
Installing pkg-1.10.1...
Extracting pkg-1.10.1: .......... done
===>   py27-pyinotify-0.9.6 depends on file: /usr/local/sbin/pkg - found
===>   Returning to build of py27-pyinotify-0.9.6



--------------------------------------------------------------------------------
--  Phase: pkg-depends
--------------------------------------------------------------------------------
===>   unbound-1.6.4_1 depends on file: /usr/local/sbin/pkg - not found
===>   Installing existing package /packages/All/pkg-1.10.1.txz
Installing pkg-1.10.1...
Extracting pkg-1.10.1: .......... done
===>   unbound-1.6.4_1 depends on file: /usr/local/sbin/pkg - found
===>   Returning to build of unbound-1.6.4_1



--------------------------------------------------------------------------------
--  Phase: pkg-depends
--------------------------------------------------------------------------------
===>   nano-2.8.6 depends on file: /usr/local/sbin/pkg - not found
===>   Installing existing package /packages/All/pkg-1.10.1.txz
Installing pkg-1.10.1...
Extracting pkg-1.10.1: .......... done
===>   nano-2.8.6 depends on file: /usr/local/sbin/pkg - found
===>   Returning to build of nano-2.8.6

The file is where it should be :
Code: 
$ ls /usr/local/sbin/pkg
/usr/local/sbin/pkg
 
it's installing pkg(8) in the builder jail (internal build environment).
synth doesn't modify the localhost environment which is where the pkg you're referring to is located.
 
I'm curious if this is working as intended? Maybe I'm just misunderstanding how this works.

There was an update for lang/python27 today.

After running portsnap fetch update, I did a synth upgrade-system and it rebuilt the following 24 packages.
Code: 
python27-2.7.13_7
python2-2_3
py27-Babel-2.3.4
py27-Jinja2-2.9.5
py27-MarkupSafe-1.0
py27-alabaster-0.7.6
py27-authres-1.0.0
py27-dns-2.3.6_1
py27-docutils-0.13.1
py27-fail2ban-0.9.7
py27-imagesize-0.7.1
py27-ipaddr-2.1.11
py27-postfix-policyd-spf-python-1.3.2_1
py27-pygments-2.2.0
py27-pyinotify-0.9.6
py27-pyspf-2.0.12_4
py27-pytz-2017.2,1
py27-setuptools-36.0.1
py27-six-1.10.0
py27-snowballstemmer-1.2.0_1
py27-sphinx-1.4.8_1,1
py27-sphinx_rtd_theme-0.2.4
py27-sqlite3-2.7.13_7
scons-2.5.1_1
After doing so, it only installed "python27-2.7.13_7" and did nothing with the 23 other packages.
Code: 
Number of packages to be upgraded: 1
Shouldn't the other 23 packages been forced upgraded - since they depend on the new updated python version that was just installed?
 
Hi,

1. I have 16GB RAM + 32GB Swap (SSD). I am using 4 builders and 8 jobs and it is working fine, but the maximum swap usage I got were about 14%. Usually it does not goes more than 4% usage. However if I increase the build or jobs numbers I begin to have some ports failing to build.

Is it supposedly to use more swap, allowing me to increase these numbers? Or should I have something mis-configured, possible what?

2. I was thinking if it would be feasible to implement the ability of setting the total jobs number instead of per builder. I mean, (e.g.) I currently have 32 possible jobs in total but if I have just two ports building, with two slots in idle/shutdown state (I understand) ports-mgmt/synth will be using just half of the total possible jobs, 16.

Or, in a event of several ports to be built, but with two idle slots and two working, each working slot could receive 16 jobs. When one of these be built, eventually allowing more three new jobs, the total jobs could be re-distributed to the new ports, then 4 for each one.

Thank you. :)
 
I also have 16GB of ram, and I use 6 builders and 4 jobs and it's never failed to build a port or touched any swap. It builds all my 150 ports in less than 20 minutes.

Other than that, I'm the last guy you want giving any advice on this :p
 
lebarondemerde said:
2. I was thinking if it would be feasible to implement the ability of setting the total jobs number instead of per builder. I mean, (e.g.) I currently have 32 possible jobs in total but if I have just two ports building, with two slots in idle/shutdown state (I understand) ports-mgmt/synth will be using just half of the total possible jobs, 16.
Click to expand...

I wholeheartedly second this. At the very least once a builder is shut down, reallocate jobs to the next task that kicks off in a still-running builder. Or when there is a choke-point of dependencies that bring it down to one builder...

Otherwise no complaints; a very happy "customer!"
 
i have installed synth via pkg install not from ports.

when i run synth status or synth system-upgrade i get

Code: 
$ sudo env http_proxy=http://10.1.1.1:8080/ synth upgrade-system
Password:
Builder mounts detected; attempting to remove them automatically ...
Dismounting successful!
Querying system about current package installations.
Stand by, comparing installed packages against the ports tree.
Stand by, building pkg(8) first ... Failed!!  (Synth must exit)
Unfortunately, the system upgrade failed.
 
Yes, mine is a server (not desktop).

Those ports you listed above have some rather large dependencies ... now I see why you're trying to tweak things.
 
hi,as mentioned above i did not manage to make synth work ,in pkg update

was getting :
Code: 
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating Synth repository catalogue...
pkg: Repository Synth load error: access repo file(/var/db/pkg/repo-Synth.sqlite) failed: No such file or directory
pkg: file:///var/synth/live_packages/meta.txz: No such file or directory
repository Synth has no meta file, using default settings
pkg: file:///var/synth/live_packages/packagesite.txz: No such file or directory
Unable to update repository Synth
Error updating repositories!
i have removed the packages required for synth and synth via : pkg remove synth and pkg remove gcc6-aux ncurses

but seems its still there ,even in search i get :
Code: 
pkg: Repository Synth missing. 'pkg update' required
pkg: Repository Synth load error: access repo file(/var/db/pkg/repo-Synth.sqlite) failed: No such file or directory
pkg: Repository Synth cannot be opened. 'pkg update' required

any ideas to complete remove it ?

Code: 
freebsd# sudo pkg install desktop-installer
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
Updating Synth repository catalogue...
pkg: Repository Synth load error: access repo file(/var/db/pkg/repo-Synth.sqlite) failed: No such file or directory
pkg: file:///var/synth/live_packages/meta.txz: No such file or directory
repository Synth has no meta file, using default settings
pkg: file:///var/synth/live_packages/packagesite.txz: No such file or directory
Unable to update repository Synth
Error updating repositories!
seem i really messed it ,now cant install pkgs at all .tryed to remove and install pkg from ports with no luck same error .
 
You must log in or register to reply here.
Back
Top