Help me ! I'm going to make a mistake... migrate to GNU/Linux ...

M

MetaOxy

Hello,

I have been using FreeBSD for my home-box server for years now; I started with FreeBSD 9.x, something like this .
I have always been very satisfied with the ecosystem, the community, the overall quality of the distribution, and its design.
I like the jails, I like packet filtering, I like the robustness of the design and the product. I'm really a FreeBSD enthousiast !
In fact, I have always tried to evangelize and promote it in various ways on forums and my blog.

Except that for the past few months, I have been facing a problem when I want to deploy new software that I find on GitHub...

Most of the time, they are intended to be installed with a “dockerfile”... and that's it !

In the best case scenario, there is documentation for building for dev peoples with npm/cargo... but most of the time, manual compilation are painfull and fails because a library does not recognize FreeBSD...
Some softwares are very easy to build (often those created with Golang or Rust), but as soon as there is a little javascript npm frontend or backend , it quickly becomes very complicated. Especially for modern webapp.

It becomes really, really frustrating to find a new software that I like and would like to test, but that I know I'm going to have to struggle with, try to translate the docker file, commit to the repo, and fight to justify why this or that library doesn't compile with FreeBSD... I don't have the energy or time for that anymore...

So I know that FreeBSD isn't the problem; the problem is that developers think that dockerfile is the new standard for deploying applications especially for javascript apps !
But reality is what it is.

I tried playing around with Podman, but it's really not production-ready, and you have to do a lot of tinkering to get software other than a "simple Python image" to work.

One solution might be to boot a VM with Bhyve based on Alpine and run Docker in that VM
But that would require me to use multiple technologies, review my backup policy, run tests, and so on. And my ressources on my main box are limitated (N100 + 16G or Ram)
This will takes me further away from what I'm trying to do: have a simple, consistent system consisting exclusively of jails easy to manage and restore.

This is why I'm considering to migrate my small n100 box to Rocky or Debian with docker only .... 😢
 
I've done a little research and here are my two cents.

1) You are facing a common conundrum.

2) The most practical solution seems to be, as you yourself point out:
MetaOxy said:
boot a VM with Bhyve based on Alpine and run Docker in that VM
Click to expand...

3) To simplify your backups, you may place the VM's disk image in a dedicated dataset and snapshot/replicate it alongside the jail datasets. You probably already know this. I'm assuming you are using ZFS.

4) Don't leave FreeBSD. You know you'll regret it in the long term.

5) To make some "sacrifices" in order to use a technology as cool as FreeBSD is worth it. I have three monitors, one is connected to a different GPU. FreeBSD can't handle this that I know of. Solution: I'm using two monitors since I switched to FreeBSD and I'm happier than ever.
 
  • Thanks
Reactions: Pap
My 2c: use tools that enable your success.

I'm using FreeBSD since 4.9, migrated from Slackware. But I'm not forcing FreeBSD on setups where it's clearly creating issues that a) cannot be solved b) require lots of my time when there is cleaner solution.

As you mentioned VM is an option (there are other choices next to bhyve too). From what you've described I think it's a good way to test your setup and see that in action.

MetaOxy said:
So I know that FreeBSD isn't the problem; the problem is that developers think that dockerfile is the new standard for deploying applications especially for javascript apps !
But reality is what it is.
Click to expand...
Well, developers of said application are aiming for Linux and it's convenient way of distributing application to end users. It's not ideal solution for everything on Linux either but certainly has a place to shine.
 
I've stopped fighting, a rocky VM in Bhyve solved my problems, but I confess I am not a big docker user, not a big fan, I like to try things and docker is probably the easiest way to do that.

For me jails are good for FreeBSD stuff mostly or even if a Linux binary is available, but if a compilation is required and needs too much tinkering, then jails loose their purpose (but it could be a me problem though).

You might also give a try to Illumos with their LX zones, combining Unix spirit and Linux "jails"/zones is a nice feeling, easy to setup and it works pretty well, the downside for me is the very small community, docs that relies mostly on Oracle Solaris10 and the low development which leads you to consider it as a good or bad choice for a long term solution.
 
Suggestion to the OP: Why don't you change the title of this thread to something like "What's the Best Way of Running Dockerfile-only apps on FreeBSD?"
 
======= Ignore - Just a RANT, not help...from poorly informed semi-noob=======

Docker is the reason Linux is "beating" FreeBSD as headless server alternative....right now....and the reason behind WSL on Windows (embrace -> suffocate pipeline)
Another one is lack of HW pass-through capability (can't pass-through just one device from one PCI) and the notorious nVidia GPU (it is not easy on Docker Linux either).

Both of these are a feature, not a bug. Docker (or Podman) run as root, vulnerable libraries in container and free-for-all pass-through is disaster waiting to happen - security and/or functional.
My prediction is, Docker will be the reason behind major security flop and will be replaced.

Is it ad-hoc user app bundle? Use something like appimage.
Is it a server? Install it in to the OS or OS-native container (jails, LXC) and for pete's sake run it under dedicated least necessary privileged account.
Is it permanent user app? Again, install it - for better efficiency, updates and audit.

Do you need to pass-through some USB dongle in to the VM? Buy or create separate Ethernet connected variant and pass it through TCP stream. Do not give PCI access to VMs, that is how hypervisor gets compromised.
 
DLL hell and 30 year old exe files were trademarks of Windows but seems Linux decided they want some of that cake too...

I for one more prefer the Windows way where everything is dumped into two or three directories which look horrible, but there's that, instead of Docker abstracting the filesystem which makes it even harder to manage the shitty old lib/executable and there are even more chances that it gets forgotten about, buried under those fs-layers.
 
  • Like
Reactions: mer
DanielBowman said:
Is it ad-hoc user app bundle? Use something like appimage.
Click to expand...

What issues do appimages/snaps/flatpaks resolve, that cannot be resolved with static builds?
Is it that glibc on Linux moves so much, it is essentially impossible to run a freshly built static program on a 5 yr old system?

In any case we take care of compatibility in a different fashion and fat binaries is all that's needed.
Even a large GUI program such as Chromium, could be 100% static on FreeBSD, it could just dlopen the few components it doesn't build itself like Xlib, libGL, check them for version and symbol disposition and then map appropriate internal code against.
 
Zare said:
What issues do appimages/snaps/flatpaks resolve
Click to expand...
For me, I am wary of running something I just casually want to use with sudo first.
I hear you...where are the days apps were one binary with their own GUI even.

But I suspect the LLMs will (indirectly) change the GUI game and thus the need of plethora of DLLS - most of them just GUI adjacent.
Now apps are slaves to GUI - either use our current calls or package your own (older) versions. I.E. APPS call the GUI functions.

Why not the other way around? I have an app, here is a bunch of functions it can perform. You choose your favorite GUI (maybe dynamically generated depending on the app I/O style and where you are in the workflow) - a blank canvas for general user input (keyboard, gestures, STT).
"Play me some Music" - DE generates GUI in form of visualization, in full screen with few big azz buttons.
"Edit this song" - DE renders GUI with several windows full of dials and graphs.
All in the style you prefer or the app suggests. No need for menus - "Show me what options for editing there is" - generates menu with LLM generated annotations.

You just code a tool, specify the API/options and it is up to the user if they want to run it using ASCII terminal or in VR or send to neuralink implant....wanna go old school and need everything uniform with mouse controls? Download GtK67 kit, LLM will map the API to that and simulate the legacy layout of controls in single pseudo-window.
 
Zare said:
I for one more prefer the Windows way where everything is dumped into two or three directories which look horrible, but there's that, instead of Docker abstracting the filesystem which makes it even harder to manage the shitty old lib/executable and there are even more chances that it gets forgotten about, buried under those fs-layers.
Click to expand...
I couldn't agree more.

I know that docker images are usually huge security holes worse than samba :D That the developers who create the images have no idea how to secure all this...

I used to use containerds at my work, and it creates a huge mess... Fundamentally, the divide between Docker/Podman/Containerd (crictl/ctrctl, etc.) creates a big mess if you're on an RHEL-like or Debian system what command to use ... Then when you have to operate it makes with dozens of mount points all over /run/. The “df -h” commands are just unreadable without “grep -v”... Containers can write outside the container, which is a real mess that puts files everywhere.

For my home env, I'd like to avoid using this, but seriously, I'm fed up with JavaScript stuff that doesn't build... Also, I don't think I'm going to open the application on the internet behind my reverse proxy as I usually do; I'm going to restrict it with WireGuard. Because I don't trust it.

What's more, podman/FreeBSD it's experimental. The management of ZFS datasets is terrible (all the jails are in the same dataset), which also creates dozens of mount points in the file system 😱

On the other hand, I have to admin that it's really nice not to have to worry about backing up the application anymore, just the data.
 
You must log in or register to reply here.
Back
Top