Best practice for meeting extra rubygem dependencies for installed ports?

[ovirt]

In the case e.g. where an installed port has additional dependencies due to adding extra features outside the scope of the FreeBSD ports system. Current case in hand is plugins for www/redmine60.

I started down the path of creating local ports to provide missing rubygems not currently in ports, but that has quickly hit dependency conflicts with existing ports.

It now seems the simplest answer is to use the bundle installer and just let it have its own way. The downsides are that the jail normally doesn't require internet or network access other than providing a web service via a reverse web proxy on the host, and perhaps the risk of encountering dependency conflicts that way, too. The connectivity issue could perhaps be addressed by downloading the required .gem files and placing them inside the jail and using bundle install --local.

My duckduckgo-fu has proved inadequate to glean any further guidance, so am hoping The Wise ones here may be able to provide further enlightenment.

 

[ralphbsz]

There are several aspects to your question.

First, the package installation system. FreeBSD has its own system for that, namely using the pkg command to install pre-built packages, which are customized (patched) for FreeBSD; the same package can also be installed using the ports system. This works great when it does. But it is run by a large set of volunteers, and many pieces of software are not available as a package (or port), and sometimes packages are outdated.

On the other hand, most programming languages have their own package system: Perl uses the CPAN site, Python uses the pip command, Rust installs its crates automatically, and as you point out, Rubygem uses bundle. Usually, these language-specific systems have a huge variety and number of things: Perl has 220K packages on CPAN, and in Rust there seem to be 2-3 (incompatible) crates for every problem. No way the FreeBSD package system and volunteers can keep up.

But how to combine FreeBSD packages with language-specific systems? That's a tough one. As described above, one can't live purely in the FreeBSD package system, since there is so much missing in it. On the other hand, some language-specific packages won't work correctly on FreeBSD, since they need to be patched (things that do hardware IO, such as serial ports or GPIO pins are often in that category). So one ends up having to use a mix of both. To make matters worse, some language-specific package systems (such as pip for Python) now lock down the install, and by default don't allow packages to be installed system-wide on OSes (such as FreeBSD) where OS-specific package managers exist. (Pip's solution of using user-specific venv is particularly insane, since it just fragments the problem per user, and the user isn't the correct granularity for managing dependencies, the application or project is. This is a huge problem area, and Docker and Flatpacks are attempts at a partial solution.

But given all that, we still need to make progress. Here is what I do: (a) Start with installing everything I can using FreeBSD's pkg system. (b) If something I need is not available there, then use language-specific commands (such as for me pip and Rust's crates). (c) Spend lots of time making sure both pkg and pip/crate/... remain up to date. (d) Deal manually with conflicts between the two systems, by letting pkg upgrade things first, and not touching pkg's packages using other tools.

All this assumes that your environment has access to the internet. As you point out, downloading and using local would work, but it is a lot of work. I would instead propose allowing your jail to access the internet, at least for the duration of package installation, and perhaps restricted to only the repository you need to read from (with a jail-specific pf.conf file and the packet filer).

All of this is annoying and tedious.

 

[ovirt]

Thanks ralph for the thoughtful and detailed response. It's very helpful to know that I haven't simply overlooked something obvious or readily known.

Having slept on it I can see a couple of possibilities that could simplify the update process somewhat for rubygems. bundle check gives a list of missing rubygems - so that should be scriptable to automate the download of the missing ones to enable running bundle --local inside the jail. Another possibility is getting bundle to run on the host similarly to pkg -r <path-to-jail>.

So TIMTOWTDI still applies and hope continues to spring eternal. Now to find the path of least resistance/annoyance/tedium. Thank you again for the reply.