Synth: Introducing new custom package repository builder for FreeBSD and DragonFly

OP
OP
marino

marino

Daemon

Thanks: 435
Messages: 1,091

Oh, so I'm doomed to either
- not upgrade the ports in my system at all;
or
- upgrade selected ports hoping they won't cause a break in firefox again via [firefox dependencies, firefox dependencies dependencies, firefox dependencies dependencies dependencies, ...] and so on.

Neither of options is optimal for me.
This means you didn't comprehend what I was telling you. If you lock a binary package, you're going to have issues with dependencies updating underneath it. If you build from source (by controlling the source tree, e.g. with svn) then you never have the issue of the dependencies being mismatched. The issue you have is the locked source port falls out of sync with the tree. Two totally different consequences to two totally different attempts of freezing a port.

For the latter, you can update the frozen port to keep working. But you're doing port maintenance at that point.

And nobody said "don't update ports at all". At best you could interpret this as "don't use firefox if you insist on an obsolete version".
or alternatively, use ravenports that will have firefox 52 for a long time.
 

vit.

New Member


Messages: 2

Effectively, you're talking about a container where all app dependencies are shipped together with the app. Like Ubuntu snaps or Windows side-by-side. With the app package being completely isolated from the rest of the system. Which is doable but requires one to maintain a custom port builds tree / deployment location. In other systems, that custom deps tree is implicitly maintained by somebody else, like the app developers themselves, but it does exist.
On FBSD you'll have to maintain it yourself. Then you can deploy it in a jail, as one of the options.
 

Mayhem30

Well-Known Member

Thanks: 17
Messages: 341

I'm having an issue and not sure how to move forward with it.
Code:
$ sudo synth status
Regenerating flavor index: this may take a while ...
Scanning entire ports tree.
 progress: 35.51%             
culprit: emulators/xen
  Scan aborted because 'make' encounted an error in the Makefile.
  emulators/xen (return code = 1)
Flavor index generation failed: ports scan
The /usr/ports/emulators/xen directory does not exist. What should I do now?
 
OP
OP
marino

marino

Daemon

Thanks: 435
Messages: 1,091

nov1c3

New Member


Messages: 8

Hello,

Would it be ok if the version of FreeBSD where Synth is installed is higher than the FreeBSD version of the servers I'm building pkg's for? For example the server where Synth is used runs 11.2-RELEASE and all other servers run 11.1-RELEASE.

Thank you.
 
OP
OP
marino

marino

Daemon

Thanks: 435
Messages: 1,091

it should work at least 99% of the time. Ideally the builder would be the older machine though. It's guaranteed that packages from older minor releases work on newer minor releases, but not the other way around.
 

mfoacs

New Member


Messages: 10

Probably an old topic but I couldn't find an answer: every time I synth upgrade-system, it ignores sysutils/lsof and sysutils/htop. There are no logs to look into - that's what "ignored" really means I guess -, so I move to a manual install with synth install sysutils/lsof. This results in "The most recent version of packages are already installed", which again, is true.

With sysutils/htop however, there is something else: synth install sysutils/htop or any of "build, force" gives me "pkg: No packages available to install matching 'htop-2.2.0' have been found in the repositories". It seems to make sense since htop requires lsof and nothing was built, so synth is happy, but I am confused...

I had no issues installing both of the packages it with pkg, portmaster or make install.

And that begs the question: if the package is already installed with the latest available version why is synth selecting it for an upgrade? and since lsof it's a dependency from another package - also already installed and up to date, shouldn't synth ignore them both completely?
 
OP
OP
marino

marino

Daemon

Thanks: 435
Messages: 1,091

the log starting with 00_* will explain why its getting ignored. go to logs directory, find it, open it, and look for for lines involving lsof and htop.
 

mfoacs

New Member


Messages: 10

the log starting with 00_* will explain why its getting ignored. go to logs directory, find it, open it, and look for for lines involving lsof and htop.
Hmm, yes that makes sense: lsof requires the kernel sources and I've installed the kernel sources on a non-standard location. Lesson learned.
 

free-and-bsd

Aspiring Daemon

Thanks: 67
Messages: 643

Hello, I'm having this funny message upon the completion of a synth install command:
Code:
The task is complete.  Final tally:
Initial queue size: 2
    packages built: 2
           ignored: 0
           skipped: 0
            failed: 0
Duration: 00:01:35
The build logs can be found at: /var/log/synth
Stand by, recursively scanning 2 ports serially.
Scanning existing packages.
Packages validated, rebuilding local repository.
Local repository successfully rebuilt
pkg: Unknown repository: Synth
Unfortunately, the system upgraded failed
Interestingly, this last pkg error prevents pkg from installing whatever has been built by synth.
Interesting, too, that when I serve the same repository via httpd to other computers, the local pkg find no problems updating from it.
It's pkg-1.10.5_1 and synth-2.05.

Here is my repo.conf which pkg pretends not to see (although it did see it until now):
Code:
# Automatically generated.
Synth: {
  url      : "file:///var/synth/11.1-packages",
  priority : 0,
  enabled  : yes
}
 

free-and-bsd

Aspiring Daemon

Thanks: 67
Messages: 643

Hello, I'm having this funny message upon the completion of a synth install command:
Code:
The task is complete.  Final tally:
Initial queue size: 2
    packages built: 2
           ignored: 0
           skipped: 0
            failed: 0
Duration: 00:01:35
The build logs can be found at: /var/log/synth
Stand by, recursively scanning 2 ports serially.
Scanning existing packages.
Packages validated, rebuilding local repository.
Local repository successfully rebuilt
pkg: Unknown repository: Synth
Unfortunately, the system upgraded failed
Interestingly, this last pkg error prevents pkg from installing whatever has been built by synth.
Interesting, too, that when I serve the same repository via httpd to other computers, the local pkg find no problems updating from it.
It's pkg-1.10.5_1 and synth-2.05.

Here is my repo.conf which pkg pretends not to see (although it did see it until now):
Code:
# Automatically generated.
Synth: {
  url      : "file:///var/synth/11.1-packages",
  priority : 0,
  enabled  : yes
}
OK, I've fixed it :) Just mixed up a bit the repo.conf files. Now they look fine and everything is running again.
Sorry for the false alarm... just when I get nervous I become distracted and can mess up things a bit.
 

Mayhem30

Well-Known Member

Thanks: 17
Messages: 341

Hi Marino, I've run in to a weird problem.

I've updated my make.conf file to use the security/openssl111 port and Synth only want to build the new OpenSSL version, and not any of the 70 ports that depend on it.

Any ideas what I am doing wrong?

/usr/local/etc/synth/LiveSystem-make.conf :
Code:
OPTIONS_UNSET = X11 CUPS

DEFAULT_VERSIONS+=ssl=openssl111 php=7.1 mysql=5.6
Code:
$ sudo synth status
Querying system about current package installations.
Stand by, comparing installed packages against the ports tree.
Scanning existing packages.
These are the ports that would be built ([N]ew, [R]ebuild, [U]pgrade):
  N => security/openssl111
Total packages that would be built: 1
 

aht0

Member

Thanks: 29
Messages: 99

I think these 70 ports may need adjustments because OpenSSL internals changed and just forcing a new version on ports would not just work. So they are still hardwired to use older openssl version. Just wait until port maintainers perform their magic.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Thanks: 6,337
Messages: 27,518

openssl111 is not a valid option:
Code:
# Possible values: base, openssl, openssl-devel, libressl, libressl-devel
See /usr/ports/Mk/bsd.default-versions.mk
 

kpa

Beastie's Twin

Thanks: 1,755
Messages: 6,235

It's not listed as a valid option there but it does work as a custom setting for selecting an exact OpenSSL port to be used. Lot of ports seem to break though at the moment with that setting, I was lucky to have only one broken port and I don't really need that one.
 

SirDice

Administrator
Staff member
Administrator
Moderator

Thanks: 6,337
Messages: 27,518

Right, I had to dig through /usr/ports/Mk/Uses/ssl.mk to see what's happening. The code itself doesn't actually check what's put in, it simply uses security/${SSL_PORT}. So it automagically works for anything you put in. It actually allows bad input too, setting SSL_DEFAULT=gnutls for example, will actually install security/gnutls but then fail later during dependency checks. Interesting.
 

mfoacs

New Member


Messages: 10

Another one: synth is failing to install databases/mongodb36.

/usr/bin/ld: final link failed: No space left on device
c++: error: linker command failed with exit code 1 (use -v to see invocation)
scons: *** [build/opt/mongo/mongoperf] Error 1
scons: building terminated because of errors.
build/opt/mongo/mongoperf failed: Error 1
*** Error code 2

Stop.
make: stopped in /xports/databases/mongodb36

--------------------------------------------------
-- Termination
--------------------------------------------------
Finished: Thursday, 20 SEP 2018 at 18:13:42 UTC
Duration: 01:00:57


At some point I see synth using 99.9% of the 9G swap available, but only roughly 10% of RAM.
The machine has 12 cores, 20G RAM and fast enough disks so I guess my synth configuration is not optimal.
I currently have 6 builders, 3 jobs per builder and I could, according to the man pages, go up to 12*0.75=9 builders and 4 jobs per builder.
What else can I do? disable tempfs?
 
OP
OP
marino

marino

Daemon

Thanks: 435
Messages: 1,091

The rule of thumb is swap should be ideally be set at 4 times the ram, so 80G in your case.
If you build mongo by itself, you should be okay. one builder, 3 jobs, 20G ram, I would be surprised if you touched swap at all.
Don't disable tmpfs.
 
Top