Synth: Introducing new custom package repository builder for FreeBSD and DragonFly

Mayhem30

Well-Known Member

Thanks: 17
Messages: 341

What is the best method to reinstall all ports after doing an OS upgrade?

Should I be doing it differently? or am I missing any steps?
Code:
sudo pkg delete -a
delete all files in: /var/synth/live_packages/All
cd /usr/ports/ports-mgmt/synth
sudo make install clean
sudo synth build /home/me/build.list
 

mefizto

Well-Known Member

Thanks: 6
Messages: 382

Greetings all,

I had been using mgmt-ports/synth to compile ports, instead of the make, install, clean.

Recently, I decided to install GUI using the guide at https://forums.freebsd.org/threads/35308/. The instructions recommend to disable sysutils/hal in the x11-servers/xorg-server. O.K., no problem, I used make -C /usr/ports/x11-servers/xorg-server config to set the desirable options before starting the compilation. However, the instructions recommend to further disable sysutils/hal in all other ports that might have HAL option. This gave me a pause because I do not know how to ascertain that one of the dependencies does not have sysutils/hal option. The make, install, clean interrupts the compilation at every dependency with options, so one could check. However, with mgmt-ports/synth it seems practically impossible to set the configuration for all the dependencies using the make -C /usr/ports/[category]/[portname] config.

Thinking about it, the issue is even more complex. How does one ensure that the option settings is consistent not only for the particular port dependencies, but among all the ports? Any advice is welcomed.

Kindest regards,

M
 

Mayhem30

Well-Known Member

Thanks: 17
Messages: 341

In order to disable an option for all ports, you need to edit your /usr/local/etc/synth/LiveSystem-make.conf file and add :
Code:
OPTIONS_UNSET = HAL
If you want to disable multiple options, it needs to be space-delimited.
Code:
OPTIONS_UNSET = CUPS HAL
Also, I usually run synth prepare-system afterwards to let Synth rebuild any changes.
 

Mayhem30

Well-Known Member

Thanks: 17
Messages: 341

the items are not "obsolete".
They are likely build dependencies.
For example, "gmake". You need "gmake" to build a package, but you don't need it after that.
If you deleted "gmake" package, synth would have to build it again.
Synth deletes truly obsolete packages (older versions) when the repository is generated, so the actual repository is 100% current.

Bottom line: the packages you don't use are not worth worrying about. I'm surprised the number is so low. I would expect a lot more than 3 packages out of 152 to be "build-only".
I figured out what the issue was. I had to pkg delete all the packages off the server and reinstall them. That is an important step if your migrating from portupgrade/portmaster, otherwise you have all sorts of build dependencies installed on the system for no reason.

What I did was :
Code:
cd /var/synth/live_packages/All
rm -rf *.txz
synth just-build `cat /home/build.list`
synth rebuild-repository
pkg delete -af
/usr/sbin/pkg bootstrap
pkg install `cat /home/build.list`
Now it shows :
Code:
$ pkg stats          
Local package database:
        Installed packages: 103
        Disk space occupied: 567 MiB

Remote package database(s):
        Number of repositories: 1
        Packages available: 142
        Unique packages: 142
        Total size of packages: 193 MiB
That's a big difference. Almost 40 packages that were not needed to be installed on the system.
 

Matt Joras

New Member
Developer


Messages: 2

So I randomly seem to get this error when building ~500 ports (just started using synth):

Code:
thread <something> exits with resources held!' at line 316 in file /usr/src/lib/libthr/thread/thr_exit.c (errno = 2)
Any ideas what might be causing that? It doesn't seem to be related to any particular port. Anything I can do to help debug it? This is on a FreeBSD-CURRENT machine, fwiw.
 

Matt Joras

New Member
Developer


Messages: 2

Matt Joras

Have you watched the memory/swap usage?
Admittedly not very closely prior to the crash. Watching it now it doesn't look like it gets anywhere close to the 32GB mem + 4GB swap.

Anyway, the message doesn't seem memory-related. Per my cursory reading of libthr it looks like a thread exiting while holding a lock:
Code:
#if defined(_PTHREADS_INVARIANTS)                                                                                                                                                             
        if (THR_IN_CRITICAL(curthread))                                                                                                                                                       
                PANIC("thread %p exits with resources held!", curthread);                                                                                                                     
#endif
...

#define THR_IN_CRITICAL(thrd)                           \                                                                                                                                     
        (((thrd)->locklevel > 0) ||                     \                                                                                                                                     
        ((thrd)->critical_count > 0))
 

Eric A. Borisch

Well-Known Member

Thanks: 211
Messages: 331

Sooo.. any hints on what to do post python-flavors change 11/30/2017? Everything python related (and above) fails due to requiring "py27-setuptools>0:devel/py-setyptools@py27" which "does not exist".
 
OP
OP
marino

marino

Daemon

Thanks: 435
Messages: 1,091

In my defense, we were told we would have 6 months to work on the support.
This introduction of flavors predates that by at least 3 months.
Secondly, the flavors developers did not provide a separate flavor branch in order to test new tools.
AFAIK, portmaster would also be caught by this.

support was planned, but introducing it 3 months early catches me by surprise.
On the plus side, at least I now how real ports to test it against.
 

lebarondemerde

Daemon

Thanks: 690
Messages: 1,563

Upstream should be probably very keen of getting rid of those many, many @python slave ports. :D

EDIT: I am also surprised BTW. When Poudriere 3.2 come out (the version with FLAVOUR support) I asked around when upstream would start accepting FLAVOURED ports and I got no reply.

So, I assumed it would still take a while.
 
OP
OP
marino

marino

Daemon

Thanks: 435
Messages: 1,091

dunno, probably a few days. I recommend synth users lock in tree at specific revision, no more than rev 455209. That's the one before ports are converted to flavors. I'm assuming the previous flavors commits don't break the tree yet. In any case, the last "good" revision is sometime earlier today.
 

Mayhem30

Well-Known Member

Thanks: 17
Messages: 341

You've been busy! I noticed on GitHub that you were constantly working on this day and night to get it done.

Thank you for all your hard work on this.
 

jdp_03

New Member


Messages: 11

I am trying to bring my system up to date after a while away due to illness. After running synth prepare-system request, I get:

Code:
Stand by, updating external repository catalogs ... done.
Scanning existing packages.
Scanning existing packages.  
Queue integrity lost!   (Synth must exit)
Unfortunately, the system upgrade failed.
Please could you advise on what I need to do next?
 
Top