general/other Best solution for using docker with docker compose on FreeBSD

icodeforyou said:
kpedersen it is strange though that the GitHub page of containerd itself claims that it is "available as a daemon for Linux and Windows". Maybe that is not true then.
Click to expand...
Indeed. Just clever marketing. Check out the Makefiles here:

https://github.com/containerd/containerd/blob/main/Makefile.windows#L25
https://github.com/containerd/containerd/blob/main/Makefile.freebsd#L17

Notice on those lines it is called containerd-shim? This is basically just a shim layer ultimately calling into a Linux emulator (they call this the "runtime" which is an incorrect name for it). Whereas the real meat of it is:

https://github.com/containerd/containerd/blob/main/Makefile#L84

There is a shim for Linux too because some people do opt to run the main instance in i.e Qemu.
 
So wait...

I can install nerdctl and containerd locally on FreeBSD? They are both in the ports tree I realize just now.

so getting
sysutils/nerdctl
together with
sysutils/containerd

and nerdctl would connect to a locally running containerd which is internally running a Linux emulator?

But that would mean my manual setup with the VM would become useless. Which would be great. Do I get this correctly?

Edit: Reading a bit more about it, I was way too optimistic it appears 😄
 
icodeforyou said:
connect to a locally running containerd which is internally running a Linux emulator?

But that would mean my manual setup with the VM would become useless. Which would be great. Do I get this correctly?
Click to expand...
You still need to provide a runtime for containerd. The runtime is basically a Linux emulator with the correct boilerplate.
So if you create a bunch of scripts around your manual setup and give it the correct names / functions for containerd to use then you have basically created a runtime.

Another option for a runtime would be to create similar kinds of scripts around FreeBSD's jails. That would be an OCI compliant runtime but nothing from DockerHub will work because as mentioned before, they aren't fully OCI compliant.
 
I did find two more interesting things. One the one hand lima-vm, which appears to attract some people trying to get rid of docker desktop on macOS. It appears to work in principle also on FreeBSD according to this discussion and this blog post.

And then I found this PR for containerd concerning Linux containers on FreeBSD. I am not sure I understand what the functionality entails though...
 
icodeforyou said:
It appears to work in principle also on FreeBSD according to this discussion and this blog post.
Click to expand...
Keep in mind that the FreeBSD alternative to KVM is Bhyve and Qemu doesn't support Bhyve. So your Qemu VM running Linux will be very slow, falling back to the TCG JIT and not virtualization. It might be enough for you though(?).

Are there any scripts around macOS's Xhyve Docker Linux emulation? I would have thought some of them could be quite transferrable.
 
icodeforyou said:
1) Getting the docker compose command on the FreeBSD machine working properly.
Click to expand...
Slightly different direction, but devel/py-molecule has a DOCKER option nowadays. Haven't tried it yet, so I don't know how good it works. It's off by default, you will need to build it from ports to use it. You could set up some Ansible roles that will spawn any docker container you need. Then have docker itself running on a local bhyve(8) vm.
 
Just to throw this out there. I'm betting that 80% of all those wanting Docker started wanting Docker cause they read somewhere that they should want Docker but had no compelling reason or need beforehand. The same is true of Kubernetes cause Google pushed it so hard.

It's a similar effect for web technologies. Some big company announces some new tech and all of a sudden there's a whole pile of questions on Stack Overflow and other forums about how they are implementing it. Too often by sole developers who think that, if Google needs it, so do they and everyone else.
 
kpedersen You are right of course, the performance penalty on lima-vm might be too high. I might still test it though...

Any ideas on this discussion and this PR on Github? It would appear that if I understand correctly through this recent PR containerd can now detect that it is on FreeBSD and use the linux compatibility layer to run Linux container images?

drhowarddrfine the main reason I want to have docker or a working docker equivalent is for the sake of interoperability between my FreeBSD laptop and my Linux machines. I want to be able to work on the same projects and have the same containerized dev environment defined by the docker / docker-compose files on both OS.

General remark: Actually I'd say this thread has been quite a success so far. Even though the best solution has not been found yet, to there is a lot of stuff I now know I did not know before 😄
 
drhowarddrfine said:
Just to throw this out there. I'm betting that 80% of all those wanting Docker started wanting Docker cause they read somewhere that they should want Docker but had no compelling reason or need beforehand. The same is true of Kubernetes cause Google pushed it so hard.

It's a similar effect for web technologies. Some big company announces some new tech and all of a sudden there's a whole pile of questions on Stack Overflow and other forums about how they are implementing it. Too often by sole developers who think that, if Google needs it, so do they and everyone else.
Click to expand...
Isn't that what's been known as a market & demand driven (self) propelling process? ;)
 
drhowarddrfine said:
We find this is the typical case of those who never used FreeBSD. Like you, those who come to learn and understand sometimes get amazed.
Click to expand...
Even though that might sound a bit odd to those who advised me in the beginning of this thread to just run my containers on Linux: What I like about working with FreeBSD is that you kinda need to look deeper at the options available when stuff doesn't just work out of the box. And in the process you learn a lot of implementation details, both about the operating system and your software, that you might never have discovered staying on Linux or even something else entirely...

And for some reason I like debugging FreeBSD more than I like debugging Linux.
 
icodeforyou said:
forquare thanks very much for these resources. nerdctl looks like it could be the way to go here. I do not particularly mind getting rid of Docker's daemon, as long as I can build a list of containers from custom dockerfiles specified in a docker-compose.yml ... I might give it a try. The only thing I cannot spot immediately is how to connect nerdctl to a remotely running instance of containerd. For docker you just have to specify an environment variable. Any ideas on that?
Click to expand...
Alas now i look into it more, it does not 😭 See here.

It seems when i was using nerdctl on macOS with colima, the installer actually replaced my local nerdctl with a shell script:

Code: 
.colima %>cat /usr/local/bin/nerdctl.lima
#!/bin/sh
set -eu
exec lima nerdctl "$@"

But maybe that information is good enough to you with Docker? Could you perhaps install the Docker CLI in your VM and write a small shell script to ssh to it (like the FAQ link I posted above)
 
I will be gone for a few days, then I will try out what works best I suppose. We got quite a few options now.

But I for sure need to do some testing regarding the containerd pull request I mentioned earlier which still appears to me to enable Linux containers on FreeBSD via the linux compatibility layer a.k.a linuxulator. If that solution were to work, then I would just install a local containerd via linuxulator + nerdctl on bare FreeBSD.
 
I will always suggest that people have far fewer problems if they quit trying to make FreeBSD act like Linux and use the tools of FreeBSD to work like FreeBSD.
 
drhowarddrfine I guess that makes sense from a certain perspective, but it sadly does not meet the requirement of interoperability that I have. In fact it quite clearly simply ignores it.
 
zirias@ said:
Seriously, why?

Don't get me wrong, but the most important aspect in chosing an OS is the application it must support. So if docker is a prio for you, use Linux.
Click to expand...
Agreed. The ABI in the vast majority of Docker and Podman containers are built using the Linux ABI. Some have tried to use the Windows ABI but those are few and far between.

I suppose a person could for the sake of saying they've done it build FreeBSD containers but I doubt there will be much uptake in the community.

Also remember Linux Cgroups + Namespaces is not the same as FreeBSD jails. Though cgroups + namespaces is IMO inferior to jails (it is less secure), cgroups + namespaces is the industry standard. There are differences that docker/podman containers rely on.

Adding cgroups + namespaces to the linuxlator might be the path forward but the cost in terms of reduced security compared to jails should also be considered.

But native FreeBSD docker/podman images might have limited utility.
 
icodeforyou said:
That is the whole point I am trying to make about interoperability I'd say
Click to expand...
Then you need to interoperate and use Linux.

Contrary to what cy@ mentioned, a "standard" that dictates a specific OS and software implementation can never be a standard. It might be crazy popular but then again... so is Windows which Microsoft continuously self-declares a "standard".
 
kpedersen said:
Then you need to interoperate and use Linux.

Contrary to what cy@ mentioned, a "standard" that dictates a specific OS and software implementation can never be a standard. It might be crazy popular but then again... so is Windows which Microsoft continuously self-declares a "standard".
Click to expand...
The standard is whatever most of the people accept. You and I may not like it but that's life.

If you care to notice who M$ is hiring, it's Linux developers. Windows isn't the thing it used to be. Their focus is cloud. Cloud for the reason of continuous revenue stream. (Follow the money.)

You and I will likely be surprised (or horrified) by what this industry looks like in five years.
 
I was away for a few days... no one had a useful comment on the containerd PR I already mentioned several times?

I might just try building the newest containerd from github with nerdctl and see if it actually runs linux containers now...

btw. interoperability has the prefix inter- within it which means: between different things... FreeBSD cannot inter-operate with itself, neither can Linux.

That being said: I already mentioned that I mainly(!) operate on Linux. I just also started liking FreeBSD in the not so long ago. And this thread is about finding a solution for my FreeBSD laptop to inter-operate effectively with by development set which includes docker.
 
Okay, so first feedback for anyone interested:

1) The current state of nerdctl on FreeBSD itself makes it not very useful. Neither does nerdctl compose work, nor can it handle networking between containers it appears, due to missing CNI plugin.

2) Lima-vm on the other hand actually does appear to work in principle. Building containers however is horrendously slow. Up to the point where building the app is actually not working because one container won't build. Meanwhile the whole system becomes unresponsive with the X server not moving the mouse any more. Not sure why that is.

I will later explore the other options.
 
You must log in or register to reply here.
Back
Top