Synth: Introducing new custom package repository builder for FreeBSD and DragonFly

OP
OP
marino

marino

Daemon

Reaction score: 441
Messages: 1,091

ah, it might be line 803. It seems it prints sorry upon success instead of failure.
 

kpa

Beastie's Twin

Reaction score: 1,814
Messages: 6,318

This was the update of today:

Code:
..snip...
Local repository successfully rebuilt
Updating Synth repository catalogue...
Fetching meta.txz: 100%    260 B   0.3kB/s    00:01    
Fetching packagesite.txz: 100%   54 KiB  54.9kB/s    00:01    
Processing entries: 100%
Synth repository update completed. 207 packages processed.
Checking for upgrades (3 candidates): 100%
Processing candidates (3 candidates): 100%
Checking integrity... done (0 conflicting)
The following 2 package(s) will be affected (of 0 checked):

Installed packages to be UPGRADED:
        synth: 0.98_2 -> 0.98_4

Installed packages to be REINSTALLED:
        curl-7.46.0_2 (options changed)

The process will require 8 KiB more space.
[1/2] Upgrading synth from 0.98_2 to 0.98_4...
[1/2] Extracting synth-0.98_4: 100%
[2/2] Reinstalling curl-7.46.0_2...
[2/2] Extracting curl-7.46.0_2: 100%
Unfortunately, the system upgraded failed.
freebsd10 ~ %
 
OP
OP
marino

marino

Daemon

Reaction score: 441
Messages: 1,091

well, if you want to try, run "make patch" and then change line 803 from
Code:
if Unix.external_command (command)
to
Code:
if not Unix.external_command (command)
and "make deinstall; make install"

I expect that is the fix. it may be a couple of days before the next update because I'm doing an extended test.
 
  • Thanks
Reactions: kpa

kpa

Beastie's Twin

Reaction score: 1,814
Messages: 6,318

I don't have time to test the fix but it looks right, under a shell the status code 0 means success but in almost every programming language 0 means false that translates to failure.
 

kpa

Beastie's Twin

Reaction score: 1,814
Messages: 6,318

As a programmer I would write constructs like that in this fashion to make it clear to anyone reading the code what the success status code is (in C):

Code:
char *externalcommand = ... ;
int status;
...

status = system(externalcommand);

if (status != 0) {
    /* Failure */
...
}
 

Chris_H

Daemon

Reaction score: 195
Messages: 1,076

WOW! Brilliant work, marino@ !
The options are many, but are intuitive enough so as to not be overwhelmed -- thanks! Thank you also for the work you put into the FAQ/documentation. It's lengthy, yet concise -- perfect! While I haven't yet actually used it to upgrade/install. I have used it to evaluate the list of ~960 ports I want it to update -- synth status ./PORTLIST. Which brings me to my only nit, so far;
It doesn't appear that synth evaluates make configure the same way that the FreeBSD ports() framework does. Case in point; synth evaluated the list I fed it, and returned ~60 ports that it indicated I need to (re)configure -- make configure || rmconfig/configure. So I went through the list, and about half of those returned "configuration found for blah-blah-NN". So after finishing the list, I figured that synth would also mark those off the list. It didn't. So it appears to me, I will now have to memorize the options for each of those, prior to a rmconfig followed by a configure. A bit of a PITA, IMHO. Nothing I can't manage. Just wished I didn't have to. :)
If you can think of anything here. I'd love to hear about it -- aside from keeping the ports updated more often. ;)

A huge thanks again, John! Really appreciate all the work, and thought you put into this!

--Chris
 
OP
OP
marino

marino

Daemon

Reaction score: 441
Messages: 1,091

So after finishing the list, I figured that synth would also mark those off the list. It didn't.
But it does. you can't even advance it if didn't. From the symptoms, I'd say you might have two locations for options, especially if you had a list of 60 obsolete cached options.

to iterate, if you make -C /usr/ports/<cat>/<port> rmconfig and then synth status returns it as still bad, synth isn't looking at the same place as /usr/ports is storing them.
 

Chris_H

Daemon

Reaction score: 195
Messages: 1,076

But it does. you can't even advance it if didn't. From the symptoms, I'd say you might have two locations for options, especially if you had a list of 60 obsolete cached options.

to iterate, if you "make -C /usr/ports/<cat>/<port> rmconfig" and then synth status returns it as still bad, synth isn't looking at the same place as /usr/ports is storing them.
Thanks for the prompt reply, John.
I think it must be my explanation. Let me re-iterate;
synth status ./PORTLIST returned a list of ~60 ports that the configure options were out of date, and hence; needed a make configure. I went through them all, performing a make configure about two thirds of them brought up the configuration dialog(). The other third returned Found configuration for port-blah-NN.... My point is; if the configuration is considered up to date enough for the port itself. Then why doesn't synth agree (it still says I need to (re)configure the port)?

Thanks again, John.

--Chis
 
OP
OP
marino

marino

Daemon

Reaction score: 441
Messages: 1,091

by the way, you should rmconfig any saved configuration that is the same as the default options. synth assumes the defaults. most people have unnecessary cached options because they just hit "yes", "yes", yes" when the dialog window keeps coming up.
 
OP
OP
marino

marino

Daemon

Reaction score: 441
Messages: 1,091

i should have read further.
Then why doesn't synth agree (it still says I need to (re)configure the port)?
the port isn't checking the validity of the config file, only that it exists.
If synth says it's bad, it's bad. It checked the options against the current port and something doesn't match up.
If you "make config" and save, it will be fixed. Likewise "rmconfig" will obviously fix it.
 
OP
OP
marino

marino

Daemon

Reaction score: 441
Messages: 1,091

The other third returned Found configuration for port-blah-NN....
"make config" brings up the dialog no matter what. It won't print "found configuration" and not bring up the dialog.
 

Chris_H

Daemon

Reaction score: 195
Messages: 1,076

OK. Got it! That makes sense enough. In the end; if I'd kept this particular box more up to date, synth would currently building/updating my ports, instead of warning (scolding) me. :)

Really, John. I can't thank you enough for this tool. It's a real godsend. I can't believe the load it takes off me.
Job well done!

--Chris

Yea. I'm using make config. :)

...and it's performing a build ./PORTLIST now! :)
 

Chris_H

Daemon

Reaction score: 195
Messages: 1,076

OK in case anyone's interested. I cobbled up a script to build a complete list of ports installed on the system. So I could feed it to ports-mgmt/synth
It goes as follows, it's heavily commented, and should run without error. but has no real accommodations for error -- you have been warned! ;)

Code:
#!/bin/sh -
# first create a list of all currently installed ports
pkg info >>./PORTLIST-RAW
Code:
#!/bin/sh -
# pkg(8) also prints a description of the ports listed
# we only want the port's names, get them
awk '{print $1;}' ./PORTLIST-RAW >./PORTLIST-CLEAN
Code:
#!/bin/sh -
# now all we need to do is prepend the category to
# those port names. I created the following script to
# accomplish just that

# create a categorized ports list to feed to
# ports-mgmt/synth

# place a list of port names, one per line *between*
# the following quotes

ports="

"
for name in $ports

do
   pkg origin $name >>./PORTLIST-DO
# the following line is just to produce some status/feedback
   echo $name
done
That's actually 3 scripts. You could actually run the first 2 commands w/o
the use of sh. I just ran the commands pkg(8), and awk(1) at my console. It's up to you. The third one should be run as a shell script. As you can see, there is some manual intervention involved. You will need to feed it the contents of PORTSLIST-CLEAN. It should finally look something like the following:
Code:
ports="
xterm-314
xtrans-1.3.5
zip-3.0_1
zziplib-0.13.62_2
"
Just put the list just like that between the 2 double-quotes. You can then run the script to create your final list, to feed to ports-mgmt/synth.
Worked well for me. Hope it works well for you! :)

--Chris
 

protocelt

Daemon

Reaction score: 411
Messages: 1,253

Yikes! That seems a lot of work. :)

As you said, you could just use pkg(8) and awk(1) in a one-liner, something like the following: # synth build/force `pkg info -oa | awk -F ' ' '{ print $2 }'`
 

Chris_H

Daemon

Reaction score: 195
Messages: 1,076

Indeed, protocelt
It was more of a brain dump. Those were all the pieces I used to arrive at a final list. But when I pasted them all in the post, and saw them all together. I quickly arrived at much the same as you. :)
Oh well. It's the thought that counts, right? :)

--Chris
 
OP
OP
marino

marino

Daemon

Reaction score: 441
Messages: 1,091

OK in case anyone's interested. I cobbled up a script to build a complete list of ports installed on the system. So I could feed it to ports-mgmt/synth
I don't understand. Why don't you simply use synth rebuild-repository or synth upgrade-system?

Synth knows what you have installed, and it just builds those installed ports with those commands.

It seems to me that the whole script is unnecessary.
 

Chris_H

Daemon

Reaction score: 195
Messages: 1,076

I don't understand. Why don't you simply use synth rebuild-repository or synth upgrade-system?

Synth knows what you have installed, and it just builds those installed ports with those commands.

It seems to me that the whole script is unnecessary.
Thank you for the information, marino@
While that is indeed good to know, and I would surely be better off using that method. The ports-mgmt/synth exit status seems contrary:
Code:
The task is complete.  Final tally:
Initial queue size: 1024
  packages built: 852
  ignored: 1
  skipped: 168
  failed: 3

Duration: 14:27:24
The build logs can be found at: /var/log/synth
Would you like to rebuild the local repository (Y/N)? Y
Scanning entire ports tree.
progress: 98.05%
culprit: packages/All
Scan aborted for an unknown reason.
bad input for 'Value: ""
Failed to scan ports tree  (Synth must exit)
Aside from the obvious portions. I'm not exactly sure what to make of it. Can you please translate why the rebuild of the repository failed? Clearly packages/All wasn't properly prepended with /usr/ports/. But my synth.ini completely checks out. No anomalies, what so ever.

Thanks again, John.

--Chris
 

Chris_H

Daemon

Reaction score: 195
Messages: 1,076

Additional information, in case it helps:

Code:
FreeBSD dev-box 11.0-CURRENT FreeBSD 11.0-CURRENT #0 r294112:
Mon Jan 18 14:25:01 PST 2016 root@dev-box:/usr/obj/usr/src/sys/DEVBOX amd64

Path: /usr/src
Working Copy Root Path: /usr/src
URL: svn://svn.freebsd.org/base/head
Relative URL: ^/head
Repository Root: svn://svn.freebsd.org/base
Repository UUID: ccf9f872-aa2e-dd11-9fc8-001c23d0bc1f
Revision: 294112
Node Kind: directory
Schedule: normal
Last Changed Author: ak
Last Changed Rev: 294111
Last Changed Date: 2016-01-15 15:13:01 -0800 (Fri, 15 Jan 2016)

Path: /usr/ports
Working Copy Root Path: /usr/ports
URL: svn://svn.freebsd.org/ports/head
Relative URL: ^/head
Repository Root: svn://svn.freebsd.org/ports
Repository UUID: 35697150-7ecd-e111-bb59-0022644237b5
Revision: 406193
Node Kind: directory
Schedule: normal
Last Changed Author: madpilot
Last Changed Rev: 406193
Last Changed Date: 2016-01-15 14:38:36 -0800 (Fri, 15 Jan 2016)

synth-0.98_2
Name  : synth
Version  : 0.98_2
Installed on  : Wed Jan 20 07:43:43 2016 PST
Origin  : ports-mgmt/synth
Architecture  : freebsd:11:x86:64
Prefix  : /usr/local
Categories  : ports-mgmt
Licenses  : ISCL
Maintainer  : marino@FreeBSD.org
WWW  : https://github.com/jrmarino/synth
Comment  : Custom package repository builder for FreeBSD and DragonFly
Thanks, again.

--Chris
 
OP
OP
marino

marino

Daemon

Reaction score: 441
Messages: 1,091

Additional information, in case it helps:

Code:
synth-0.98_2
Name  : synth
Version  : 0.98_2
That's been fixed in 0.98_4. The version you installed is too old. You can pkg upgrade synth because portsmon says 0.98_4 is available everywhere
 

Chris_H

Daemon

Reaction score: 195
Messages: 1,076

OK. I started a 2nd run, after telling pkg() to upgrade ports-mgmt/synth, and you were right; the package clusters had 0.98_4. Anyway I chose another synth build /BUILDLIST. Before you remind me about rebuild-repository, or upgrade-system, I just want to know that It'll finish a task completely, and uneventfully, before I make a full commitment, and let alter my system. :)
Anyway, it, or rather, one of the builds failed. Shortly after, a (system?) message was cast across the bottom of the screen:
Code:
ELF interpreter /libexec/ld-elf.so.1 not found, error 2
As you might guess from the number of ports it needs to build, this box is lagging a bit. But still, if it had anything to do with the synth install, I would have figured that pkg() would reconcile any dependencies, or forbid the install, for lack of prerequisites. So I'm not sure where to point, or look, exactly. I've instructed synth to finish what it's doing, and end the current build session (ESC). So I'll give the log(s) a look (system && synth), when synth's finished. But thought I'd mention it, in case you've seen, or heard anything like this, where synth is concerned.

Thanks!

--Chris
 
OP
OP
marino

marino

Daemon

Reaction score: 441
Messages: 1,091

Before you remind me about rebuild-repository, or upgrade-system, I just want to know that It'll finish a task completely, and uneventfully, before I make a full commitment, and let alter my system
synth build-repository doesn't alter the system. It's just assembling information about all the build packages, but that's it.

I would have figured that pkg() would reconcile any dependencies.
Why would you think that? It's Synth's job.

The error sometimes happens as a result of failed builds. I've see it before on occasion.

Synth isn't at release status yet, I'm still looking for, finding, and fixing some bugs (many reported here)
 

Chris_H

Daemon

Reaction score: 195
Messages: 1,076

N.P. I know that it's still a young application. But I really like it, and am anxious to provide anything I can to help it become stable. :)
As to pkg() being responsible for chasing dependencies. Yes. I knew. But was sure synth couldn't possibly be wrong. ;)
Seriously. After posting my message, and going back to monitor the progress. It occurred to me that the only thing that wouldn't/couldn't find /libexec/ld-elf.so.1 would be something in a chroot(), or jail(). So concluded that either the chroot wasn't setup correctly, or something in a chroot crashed badly.
Anyway. I'll report back with anything that looks like it might be of any interest to you.

Thanks for the reply, John.

--Chris
 
Top