Synth version 2.00: Flavor support

No worries John, I'm just thankful that you made Synth. My 'play' server is a Dell PowerEdge 1950 (2x quad-core 2.66GHz Xeon E5430 Harpertown processors, 16 GB RAM, 2x 143GB SAS hard drives). I am researching hardware (and trying to convince my wife) to let me build a brand new box using Intel Core i5-8400 but even that will have only a moderate gain over dual E5430 Harpertown processors. I use the same server (a 2nd one actually) with net-p2p/btsync to do the building for my slower machines.

Thanks for your effort with this FLAVORS thing; I'm far from sold on it, but it's early days. Wish there was more I could read up on it, but no one has offered any URLs yet.

Thanks again.
 
Just saw that ports-mgmt/portmaster was updated and now supports flavors. I see that portmaster checks the /usr/ports/MOVED file to work things out.

Couldn't Synth use that file instead of having to build it's own index? (which is quite time consuming).
Code: 
devel/py34-setuptools|devel/py-setuptools@py34|2017-11-30|Moved to a flavored, generic, version
devel/py35-setuptools|devel/py-setuptools@py35|2017-11-30|Moved to a flavored, generic, version
devel/py36-setuptools|devel/py-setuptools@py36|2017-11-30|Moved to a flavored, generic, version
 
For people that this index generations seems like a problem, I suggest doing the following:

Write a small shell script that first updates the ports tree, then executes synth status. Update cron to run that script at like 0500 each morning. Then your machine will have a current index every time you need to use it.

If somebody wants to publish such a script and post it here, maybe that would help others.
 
That's exactly what I've done. I have cron set to run portsnap cron update at 5AM - and then at 6:30AM it runs my script.

Note that portsnap cron update sleeps a random amount of time between 1 and 3600 seconds. So you need to wait a minimum of 1 hour after it fires before running the script.
Code: 
0  5 * * * /usr/sbin/portsnap cron update >/dev/null 2>&1
30  6 * * * /bin/echo y | /home/check-updates.sh
When running the script in cron, you need to add "/bin/echo y" to tell synth to rebuild the index (if prompted), otherwise it will hang and do nothing.

If updates are available, I "echo" it so it emails me the results. If no updates are available, it echo's nothing and then exits.

check-updates.sh
Code: 
#!/usr/local/bin/bash

read UPDATES <<< $(/usr/local/bin/synth status | awk '/Total packages that would be built:/ { print $7 }')

if [ "$UPDATES" -eq "0" ]; then
   exit;
fi

if [ "$UPDATES" -eq "1" ]; then
   UPDSTR="package update is"
else
   UPDSTR="package updates are"
fi

echo "Synth status: $UPDATES $UPDSTR available for download"
I hope this helps someone, as it took me a bit to figure it all out.
 
i would have added the "portsnap" command to the check-updates.sh script so that you don't have to time it.
Or in other words, the second part doesn't run if the first part fails.
 
Some tweaks. Remove the portsnap cron line from crontab, and just run this every morning.

Code: 
#!/bin/sh

portsnap auto > /dev/null || exit 1

[ -x /usr/local/bin/synth ] || { echo "Error: synth not installed?"; exit 1 ; }

SYNOUT=$(echo y | /usr/local/bin/synth status)

BLDS=$(echo -e "${SYNOUT}" | awk '/Tot.*built:/{print $7}')
DLS=$(echo -e "${SYNOUT}" | awk '/Tot.*fetched:/{print $7}')

[ "$BLDS" = "0" ] && [ -z "$DLS" ] && exit

[ -z "$DLS" ] || DLSTAT=" ${DLS} to download,"

echo "Synth status:${DLSTAT} $BLDS to build."
 
portsnap auto is not valid command on FreeBSD 10.4
Code: 
$ sudo portsnap auto
usage: portsnap [options] command ... [path]

Options:
 ...
 ...
 ...
Commands:
  fetch        -- Fetch a compressed snapshot of the ports tree,
                  or update an existing snapshot.
  cron         -- Sleep rand(3600) seconds, and then fetch updates.
  extract      -- Extract snapshot of ports tree, replacing existing
                  files and directories.
  update       -- Update ports tree to match current snapshot, replacing
                  files and directories which have changed.
 
Mayhem30 said:
portsnap auto is not valid command on FreeBSD 10.4
Code: 
$ sudo portsnap auto
usage: portsnap [options] command ... [path]

Options:
 ...
 ...
 ...
Commands:
  fetch        -- Fetch a compressed snapshot of the ports tree,
                  or update an existing snapshot.
  cron         -- Sleep rand(3600) seconds, and then fetch updates.
  extract      -- Extract snapshot of ports tree, replacing existing
                  files and directories.
  update       -- Update ports tree to match current snapshot, replacing
                  files and directories which have changed.
Click to expand...

True. All my systems are on 11.1-p*. Adapt as needed. ;)

[edit]
Fixed typo in version #.
For those with 11, “auto” is just what you would hope: “Run fetch or cron depending on whether stdin is a terminal; then run update or extract depending on whether portsdir exists.”
 
On one of my machines, I have an auto-update cronjob script:

Code: 
# ...
export TERM=xterm
# ...
echo "***" | tee -a ${LOG_FILE}
echo "*** Looking for ports to update..." | tee -a ${LOG_FILE}
echo "***" | tee -a ${LOG_FILE}
/usr/local/bin/synth rebuild-repository | tee -a ${LOG_FILE}
/usr/local/bin/synth upgrade-system | tee -a ${LOG_FILE}
/usr/local/bin/synth purge-distfiles | tee -a ${LOG_FILE}

However, Synth just halts:

***
*** Looking for ports to update...
***
Synth is already running on this system.
Synth is already running on this system.
Synth is already running on this system.
Click to expand...

Executing the commands in the file manually does not have this effect, I am not prompted for anything either. What am I missing?
 
That's a dangerous script. You should limit this to synth prepare-system.
Plus it doesn't do what I think you want it to do.
"Rebuild repository" doesn't build anything, so "upgrade" isn't going to do anything.
But if you meant to put synth prepare-system instead of synth rebuild-repository I would not recommend that.
If there's a build failure and the full ports build doesn't work, the system upgrade will be partial at best. System upgrades should always be manual in my option.

As for the actual question, there's a active synth pid somewhere (at least there was). I think the messages are legit at the time of the cron job. When you try it later, there wasn't any instance of synth running.
 
Thank you - I have misunderstood the README, probably.
But why are there suddenly active synth PIDs every three days (the cronjob time) without me starting any? I thought that maybe synth doesn't find itself, it just made no sense to me.

Yes, system upgrades should be manual and I actually do them manually on all non-playground servers. :)
But I'm lazy sometimes.
 
evidence suggests another cron job is starting synth up at the same time. Or your script is forking and racing. but something is going on and the error messages are most likely real.
 
For the record, here's where I ended, which seems to be working when called via cron: updates portfiles, rebuild synth flavors index, report status back to cron via stdout...

(For 10.x, change 'portsnap auto' to two lines with 'portsnap cron' and 'portsnap update'...)

Code: 
#!/bin/sh

# Synth requires TERM to be set
export TERM=xterm

portsnap auto > /dev/null || exit 1

[ -x /usr/local/bin/synth ] || \
    { echo "Error: synth not installed?"; exit 1 ; }

SYNOUT=$(echo y | /usr/local/bin/synth status)

BLDS=$(echo -e "${SYNOUT}" | awk '/Tot.*built:/{print $7}')
DLS=$(echo -e "${SYNOUT}" | awk '/Tot.*fetched:/{print $7}')

[ "$BLDS" = "0" ] && [ -z "$DLS" ] && exit

[ -z "$DLS" ] || DLSTAT=" ${DLS} to download,"

echo "Synth status:${DLSTAT} $BLDS to build."
 
What is this "Total fetched" line the code is trying to scrape? I have never seen it in any Synth output.
Code: 
DLS=$(echo -e "${SYNOUT}" | awk '/Tot.*fetched:/{print $7}')
 
Eric A. Borisch said:
If you have ‘leverage pre-built packages’ enabled in synth.
Click to expand...

Ok, that is one option that I'm still not clear about. When enabled, providing my port does not have custom options, it will grab the pre-build package (if available) from an external repo?

What file do I need to define the external repo in?
 
Mayhem30 said:
Ok, that is one option that I'm still not clear about. When enabled, providing my port does not have custom options, it will grab the pre-build package (if available) from an external repo?

What file do I need to define the external repo in?
Click to expand...

In such a way the pkg is aware of them.

See synth(1) for the description on when a pre-compiled port is considered usable. It’s my favorite part of synth over “the other builder.” Only those ports where I need a non-standard option (and ports depending on them) need to be built. Most of the ports where I need a non-standard option are high on the dependency tree (apps, rather than libs) so many ports can be fetched.
 
Ok, so only 14 of my ports will have to be built, and the other 130 (if available and considered usable) will use a pre-built package.

Why is this option not enabled by default?
 
Synth appears to have crashed this morning.
Code: 
Regenerating flavor index: this may take a while ...
Scanning entire ports tree.

culprit: databases/postgis24
Scan aborted because 'make' encounted an error in the Makefile.
databases/postgis24 (return code = 1)
Also, it left all of these behind :
Code: 
$ df -h
Filesystem                  Size    Used   Avail Capacity  Mounted on
/dev/mirror/gm0s1a          105G    7.2G     89G     7%    /
devfs                       1.0K    1.0K      0B   100%    /dev
tmpfs                        13G    228K     13G     0%    /usr/obj/synth-live/SL09
/bin                        105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/bin
/sbin                       105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/sbin
/lib                        105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/lib
/libexec                    105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/libexec
/usr/bin                    105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/usr/bin
/usr/include                105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/usr/include
/usr/lib                    105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/usr/lib
/usr/libdata                105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/usr/libdata
/usr/libexec                105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/usr/libexec
/usr/sbin                   105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/usr/sbin
/usr/share                  105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/usr/share
/usr/ports                  105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/xports
/var/db/ports               105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/options
/var/synth/live_packages    105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/packages
/usr/ports/distfiles        105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/distfiles
tmpfs                        12G    4.0K     12G     0%    /usr/obj/synth-live/SL09/construction
tmpfs                        12G    4.0K     12G     0%    /usr/obj/synth-live/SL09/usr/local
/usr/lib32                  105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/usr/lib32
/boot                       105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/boot
tmpfs                       100M    4.0K    100M     0%    /usr/obj/synth-live/SL09/boot/modules
/usr/games                  105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/usr/games
/usr/src                    105G    7.2G     89G     7%    /usr/obj/synth-live/SL09/usr/src
devfs                       1.0K    1.0K      0B   100%    /usr/obj/synth-live/SL09/dev
Do I need to umount all of them manually?
 
no. any synth command, e.g. synth configure cleans them up.

It didn't crash. The port tree is bad, so it stopped (as it should).
 
You must log in or register to reply here.
Back
Top