Synth v1.00 has been released

When start synth I got this:
Code: 
03  --:--:--  check-sanity  get_catport: invalid key_cursor

And found this in file 00_last_results.log:
Code: 
00:00:18 [03] => emulators/linux_base-c6
 
Is your port tree stock? is mars.pkglist got any strange characters in it?
Are there ports involved that are not in the ports tree?
 
This is from file mars.pkglist:
Code: 
archivers/arj
archivers/bzip2
archivers/file-roller
archivers/p5-Archive-Zip
archivers/p7zip
archivers/thunar-archive-plugin
archivers/unzip
archivers/zip
audio/clementine-player
audio/gstreamer-plugins-faad
audio/mate-media
deskutils/mate-utils
devel/bison
devel/cmake
devel/cppunit
devel/dmake
devel/doxygen
devel/git
devel/gitflow
devel/gmake
devel/gperf
devel/maven31
devel/mdds
devel/ninja
devel/patch
devel/ruby-gems
devel/tmake
devel/ucpp
devel/xorg-macros
devel/yajl
editors/libreoffice-sl
editors/vim
emulators/linux-c6
emulators/linux_base-c6
ftp/filezilla
graphics/cairo
graphics/evince
graphics/gimp
graphics/gstreamer-plugins-aalib
devel/git
devel/gitflow
devel/gmake
devel/gperf
devel/maven31
devel/mdds
devel/ninja
devel/patch
devel/ruby-gems
devel/tmake
devel/ucpp
devel/xorg-macros
devel/yajl
editors/libreoffice-sl
editors/vim
emulators/linux-c6
emulators/linux_base-c6
ftp/filezilla
graphics/cairo
graphics/evince
graphics/gimp
graphics/gstreamer-plugins-aalib
graphics/mupdf
graphics/vigra
java/eclipse
java/netbeans
java/openjdk8
lang/gawk
lang/python27
mail/ssmtp
mail/thunderbird
mail/thunderbird-dictionaries
mail/thunderbird-i18n
math/galculator
misc/mc
multimedia/gstreamer1-plugins-all
multimedia/gstreamer1-plugins-bad
multimedia/gstreamer1-plugins-good
multimedia/gstreamer1-plugins-ugly
net/vnc
net/openntpd
ports-mgmt/dialog4ports
ports-mgmt/portmaster
ports-mgmt/poudriere
print/cups
print/cups-filters
print/hplip
security/ca_root_nss
security/sudo
sysutils/apcupsd
sysutils/automount
sysutils/ezjail
sysutils/fusefs-ext4fuse
sysutils/fusefs-ntfs
sysutils/grub2-bhyve
sysutils/libcdio
sysutils/mate-system-monitor
sysutils/monit
textproc/aspell
textproc/gsed
textproc/intltool
textproc/linux-c6-expat
textproc/sl-aspell
www/chromium
www/firefox
www/firefox-i18n
www/midori
www/opera
www/xpi-adblock_plus
www/xpi-colorfultabs
www/xpi-firebug
www/xpi-flagfox
www/xpi-quick-locale-switcher
x11/glproto
x11/mate
x11/mate-desktop
x11/nvidia-driver-340
x11/slim
x11/xorg
x11-drivers/xf86-video-fbdev
x11-fonts/webfonts
x11-themes/icons-tango
x11-themes/icons-tango-extras
x11-themes/slim-themes
x11-toolkits/open-motif

The same file is used by ports-mgmt/poudriere without any problems.
 
It's a directory at /usr/ports and you can't delete it.
Something on your mars list is pulling it in.
 
That list seems complex. why not just use "primes"?
I mean stuff like Dmake, etc, those are build dependencies. you don't need to specify stuff like that. specify the final port that you want.

unless you really want "Bison" ?


You can bisect the list --- break it to 2 parts, see which part pulls in Linux, and keep splitting it into two until you find the port.
 
Well, mystery solved:

Code: 
emulators/linux-c6
emulators/linux_base-c6

you specified it.

Opera probably needs it too.
 
I recommend that you really review that list and remove all the build dependencies. Those will be pulled in as necessary. Just put on the list what you actually want to be in the repository.
 
For everyone:
I'm going to be on vacation starting pretty much right now and going on for at least 8 days. I'll bring a laptop with me but I don't know how much I'll connect in the next few days.

If there is anything unsolvable, just document it at:
https://github.com/jrmarino/synth/issues

And I'll look when I get back (and also work on version 1.1)
 
Hi Marino, thanks for your work! You described your tool at BSDnow to be easy for beginners, but I've got some questions:

First, your Github page lists the synth repo as the highest priority system repo, but when I want to install a built port through pkg, the official package gets installed instead. Why is this the case? I have to specify the full path for it to work – the „-r“ option doesn't work because „No repositories are enabled“.

Second, I want to install a custom built port into a jail, but neither pkg -j X add nor pkg -j X install work with full paths to a built port – but on the host machine it does. Can someone help me here, please?

Third, does your program provide an option to delete the whole local repo or individual packages, or can I safely delete /var/synth/ or particular packages followed by synth rebuild-repository?

Last, will your program provide a „-j“ jail option in the future? Why do some operations (rebuild-repository, purge distfiles) take for so long while just a few ports were compiled? And why is pkg being built when already being installed on the system?

Best regards and have a nice holiday,
Frank
 
Murdock said:
First, your Github page lists the synth repo as the highest priority system repo, but when I want to install a built port through pkg, the official package gets installed instead. Why is this the case? I have to specify the full path for it to work – the „-r“ option doesn't work because „No repositories are enabled“.
Click to expand...

It sounds like you have never run a synth command that would create the pkg config file. pkg -vv would confirm that the Synth repository is not defined, but it sounds like pkg is saying NO repositories are defined. The solution to the problem is to install through synth (once). The correct way to install a package before that is pkg add and providing a full path to the package.

Second, I want to install a custom built port into a jail, but neither pkg -j X add nor pkg -j X install work with full paths to a built port – but on the host machine it does. Can someone help me here, please?
Click to expand...

you can run package directly after you install the missing pkg conf file. you can also add a pkg conf file to the jail itself and call package from the jail.

Third, does your program provide an option to delete the whole local repo or individual packages, or can I safely delete /var/synth/ or particular packages followed by synth rebuild-repository?
Click to expand...

No, it is not safe, nor can I imagine a reason you would want to do that. The repo is constructed as you said it should be and doesn't contain anything extra.

Last, will your program provide a „-j“ jail option in the future?
Click to expand...

no, that's a pkg command and it works on pkg when it's configured correctly. Synth builds repos, not installs packages (at best it commands pkg to install them; that's just for convenience)

Why do some operations (rebuild-repository, purge distfiles) take for so long while just a few ports were compiled?
Click to expand...
currently they scan the entire tree which takes a long time on FreeBSD (not DragonFly) because nobody is fixing the absurd USES=compiler:features implementation. V1.1 will remove the full tree scan when I figure out how to do it safely on the repository rebuild. Scanning the whole tree is unavoidable for purging distfiles, but is not like you run that command every day. More like every 2-3 months.

And why is pkg being built when already being installed on the system?
Click to expand...
Because each builder needs pkg in order to do queries and scanning options. Synth doesn't use nor trust anything on the system other than the system root it's building on.[/quote]
 
I just noticed when scanning the ports tree Synth takes around 4 times as long for me with version 1.02. What used to take less than a minute in version 1.00 now takes around 4 minutes. Not really that big of a deal but I can see it taking far longer for other people as I have a pretty beefy machine with a ton of RAM.
 
marino@ said:
No, it is not safe, nor can I imagine a reason you would want to do that. The repo is constructed as you said it should be and doesn't contain anything extra.
Click to expand...

I just upgraded my system from php56 to php70. The method I used was to dump all php56 ports with pkg info -ao | grep php56 | awk {'print $2'} | sed -e 's/php56/php70/g' > portlist, then I ran synth build ./portlist, followed by pkg delete php56\*, and pkg install -r Synth `cat portlist`.

This seemed to work as expected, I had replaced all php56 ports with php70 ones. However then all of the old php56 ports were still found in the repo so I just deleted /var/synth/live_packages/All/php56* and ran synth rebuild-repository to make things consistent again.

I'm guessing you're going to say now that synth would have deleted the files automatically if I had run another command and there was no need at all to do this? But if not then that's an example of when you might want to do this?
 
xtaz said:
This seemed to work as expected, I had replaced all php56 ports with php70 ones. However then all of the old php56 ports were still found in the repo so I just deleted /var/synth/live_packages/All/php56* and ran synth rebuild-repository to make things consistent again.
Click to expand...

It's not necessary to do. At some point (probably sooner rather than later) the packages will be considered obsolete and just deleted automatically before the next repository rebuild.

I'm guessing you're going to say now that synth would have deleted the files automatically if I had run another command and there was no need at all to do this? But if not then that's an example of when you might want to do this?
Click to expand...

Think about it this way. You are a member of a library. You need 100 books, but the library has 100,000 books. You don't need to burn the 99,900 books that you don't use. You don't care if they are there or not. This case isn't nearly as extreme. Maybe there are 10% more packages in the repo that you won't use in the future. All it costs is disk space and maybe a few extra seconds when generating the repo digests. In practical terms, there's not much to be gained by actively removing valid packages. Just leave them there until they get removed naturally by bitrot.
 
protocelt said:
I just noticed when scanning the ports tree Synth takes around 4 times as long for me with version 1.02. What used to take less than a minute in version 1.00 now takes around 4 minutes. Not really that big of a deal but I can see it taking far longer for other people as I have a pretty beefy machine with a ton of RAM.
Click to expand...
On my FreeBSD 10.2-RELEASE (amd64) it happened the same as on your machine.
 
I'm not imagining it then? To be honest I can't remember how long it took on the older versions, but I do know that it takes ages on my server with the current version. I just timed it doing the "Scanning entire ports tree" stage and it took 31 minutes. It takes longer than compiling the actual packages themselves! Hopefully it's a lot faster in version 1.1 as it looks like it's going to be done a different way without a full tree scan.
 
it takes about 2.5 minutes on dragonfly and about 10 minutes on FreeBSD. 31 minutes is absurdly extreme. if you are seeing that, your server is doing something else i/o-related at the same time.

edit: and no, nothing changed b/w 1.00 and 1.02. I never got kpa's claim of 30 seconds for a full scan.
 
I can see that it's not doing anything else at all. It's just a low powered home-server box. An Intel Atom D525 1.8Ghz 4 core, 4GB RAM, and 2.5" 5400 rpm laptop hard drive. Running 10.3-BETA2 amd64. Not the fastest of things. But it does the job. Bit silly though that it usually compiles whatever package updates there are in 3 or 4 minutes and then takes 31 minutes to do the port tree scan. Hopefully 1.1 takes the problem away if it won't be doing a full tree scan any more.
 
nothing silly about it at all, the i/o for a full tree scan is magnitudes greater than a single port build. it's why the INDEX file exists. synth doesn't use the INDEX though. but yes, changing the 5400 mech out with an SSD would cut that scan time dramatically....
 
You must log in or register to reply here.
Back
Top