general/other Looking for UK/EU VPS or bare-metal hosting recommendations (ZFS, Jails, Bhyve)

asteriskRoss

asteriskRoss

I'm looking for some advice on hosting providers in the UK or EU for my (small) company's DevOps infrastructure; a small number of services including source control as well as continuous integration (CI) runners. Our main product is a virtual appliance so the CI runners will be compiling code, building the appliance and spinning up VMs with bhyve for integration testing.

To give some background, we are currently hosted in AWS but have found the platform quite cumbersome to manage. It just doesn't feel like it's designed for a small company. We had quite a generous amount of free credits but they're about to run out and I'm appalled by the pricing for larger servers on their platform.

VPS would be fine, except that most providers I have looked at don't offer nested virtualisation and spinning up ephemeral machines looks awkward.

I've looked at a few alternatives so far:
  • Vultr: I have used Vultr in the past for a cheap personal VPS running FreeBSD and had a good experience (once I fixed the network adapter performance). However, minimum charging period for an instance is an hour, so creating a tearing down instances frequently wouldn't be cost effective. They look expensive for bare-metal.
  • Contabo: This provider looks cheap for VPS, with this warning from Peter2121 in Thread 21472 but again no nested virtualisation
  • Hetzner: I was very tempted by Hetzner's dedicated servers and auction pricing. However, I was put off by the fact that they do not officially support FreeBSD and by the complex workarounds detailed in Thread 85399. I fear losing hours of engineering time if there is a server issue.
Does anyone have recommendations for UK or EU-based providers that offer reasonably priced VPS that supports nested virtualisation or bare-metal servers with good, native FreeBSD support?

Thanks in advance for your help!
 
asteriskRoss said:
  • Hetzner: I was very tempted by Hetzner's dedicated servers and auction pricing. However, I was put off by the fact that they do not officially support FreeBSD and by the complex workarounds detailed in Thread 85399. I fear losing hours of engineering time if there is a server issue.
Click to expand...

Hetzner is great. In case of problems just write an email and usually within an hour you get access console access via a web-kvm.

Alternative: netcup

Currently I am moving quite some big projects over from US based clouds.
 
netcup vps amd64, 6 cpus, 8Gb memory, 160 GB disk. 12 Euro/mo. Additional 600 GB storage + 8 Euro/mo.
no complains (they hiked the price somewhat because ram/storage/energy but they all did that). Prices above are current.
I also had digital ocean in the past, it was ok but netcup was a lot cheaper for the same stuff or better.

also https://forums.freebsd.org/threads/vps-v-cloud-hosting.103178/page-5#post-773510

also used https://www.arubacloud.com/ some years ago, was ok (i had some 1 euro instances in france, uk, de). (to break netflix geo fencing :))
they got incrementally more expensive so i closed them. no problems just some annoying italian emails and parts of interface also italian
they also seem to offer bare metal now

tested contabo and it sucked (probably extremely oversubscribed, console lagged, etc)

used my.frantech.ca because they had the cheapest block storage (looks like they still have it) for remote zfs backup but moved to netcup (cheaper combo ram + cpu + storage)
 
rootbert said:
Hetzner is great. In case of problems just write an email and usually within an hour you get access console access via a web-kvm.
Click to expand...
That's very useful to know. So let's say I'm an idiot and manage to mess up the network configuration, it's possible to log in at the console without needing specialist client software (the thread I linked was talking about needing a Java client)? What method did you use to get FreeBSD installed in the first place?
 
A friend of mine rents bare metal at Hetzner but he runs linux. There is no unattended ipmi / serial console access but you can set the machine to boot in recovery mode (tftp) from the admin webpage.
It tftp boots some linux stuff. probably the linux kernel that boots in recovery mode has no zfs support (i don't really know)
he had a nvme disk failure so i used it once

this is what google ai says about it
Hetzner's standard Linux rescue system does not load ZFS kernel modules by default, but you can manually install and activate ZFS support inside the rescue environment to import or repair your pools. [1, 2, 3]

Using ZFS in the Hetzner Rescue System
    • Boot into Rescue Mode: Activate the rescue system via the Hetzner Robot (for dedicated servers) or the Cloud Console, then reboot your server. [1, 2]
    • Install ZFS Packages: Connect via SSH and install the ZFS utilities and kernel modules using the rescue system's package manager (e.g., apt-get update && apt-get install zfsutils-linux on a Debian/Ubuntu-based rescue system). [1]
    • Load the Module: Ensure the ZFS module is loaded into the kernel by running modprobe zfs.
    • Import Your Pool: Locate your pools using zpool import and mount them with zpool import -f <pool_name> or specify an alternate root using -R /mnt.
 
covacat said:
There is no unattended ipmi / serial console access but you can set the machine to boot in recovery mode (tftp) from the admin webpage.
It tftp boots some linux stuff.
Click to expand...
Thanks for clarifying: that is the level of hassle I am keen to avoid. Engineer time (including mine) costs a lot more than the saving from cheaper but inconvenient hosting. For personal stuff, the challenge of unsupported tech is a welcome one but not so much for work stuff!
 
manitu.de also added vservers to their portfolio a while ago. I've had/have several root servers and web hosting packages there over the years and also always recommend them to friends/family etc when asked for a good web/mailhoster. I've worked with them for ~20 years now and they are always my go-to provider if I need something reliable that needs to just work and has proper support if something does not.

They run their own datacenter and infrastructure at their location in germany, so no dependencies on larger providers that would take them down with them. Support is *extremely* fast, friendly and professional - you never have to navigate through various layers of 'first-level-bulwarks' until you get someone with some expertise. you always get proper help from someone that knows their stuff AND has the power to move the necessary stuff to help you right from the first reply. E.g. I've had the occasional HDD failure/bad blocks back when root servers always had spinning rust in them, and the disk replacement was always faster than it took me to diagnose the problem, find the offending HDD and write the ticket.
That being said, they don't offer FreeBSD preinstalled nor do they 'officially' support it, but they have some FreeBSD-savvy people in their rows which were always happy to help if the OS was actually relevant to the problem or solution. You can just install FreeBSD via the rescue image or e.g. via mfsBSD (IIRC on their root servers I just dd'ed mfsBSD to the HDD from the rescue image). I haven't used their VPS servers yet, so I can't give any details on how it might work there (e.g. custom images).

Regarding contabo and netcup: their KVM networking for VPS is completely broken regarding ipv6 routing - there are several threads about that. They both had tickets about that for several years without any solution or change it their botched setups. You also have occational short outages or performance degradation, so I wouldn't recommend them for important stuff. That being said I still run a VPS at netcup for a personal mailserver - mainly because I was too busy/lazy to migrate it to manitu since they offer VPS.
Also beware of their automated backups - last time I tried (~2-3 years ago) their snapshots were not zfs-compatible, i.e. the images were unusable. This might be different now that bitterFS is finally dead and linux folks also use ZFS.

The few times I had to deal with Hetzner it was always tedious to get proper support and then they were pretty elitist and always pointed to others (me/my customer) even if the problem was 100% on their side (their mailservers also have some weird and occasionally broken configuration...). Strato is pretty much the same but worse - I think both have become way too big to care about pesky "small" customers anymore...
 
you can run linux on bare metal @ hetzner and create as many freebsd vm-s as you can. Problem is you won't get freebsd support anywhere and if they have it today they may not have it tomorrow. You won't get a good price, stable, no out of business tomorrow and ipmi anywhere. and as of FreeBSD support ... good luck with that.
 
covacat said:
you can run linux on bare metal @ hetzner and create as many freebsd vm-s as you can. Problem is you won't get freebsd support anywhere and if they have it today they may not have it tomorrow. You won't get a good price, stable, no out of business tomorrow and ipmi anywhere. and as of FreeBSD support ... good luck with that.
Click to expand...
why search for freebsd support? search for .iso boot support then boot freebsd iso and install it
 
What exactly do you mean by "they don't support FreeBSD"? With dedicated HW it's up to you what you install there. You have an option to upload your custom ISO there too (you can run your own OS too).
VMs are, as far as I know, qemu+kvm so it's largely supported.

If you are asking about having commercial support on top of the hosting services, I don't know what services Hetzner provide.

Currently I have one VM in Hetzner, geli/zfs, and had zero issues installing and running it for some time now.
 
vm's have kvm console at hetzner via js but bare metals don't.

you can run freebsd but you have no serial console.if some hardware fails or you make a config mistake / bothced upgrade and the system won't go multiuser what do you do ? your recovery boot is linux without zfs support. i dont say its not possible to do it, its just not as easy as running linux. its like when you ask your isp to help with your pppoe connection and telling them you are running mpd5
 
covacat said:
A friend of mine rents bare metal at Hetzner but he runs linux. There is no unattended ipmi / serial console access but you can set the machine to boot in recovery mode (tftp) from the admin webpage.
It tftp boots some linux stuff. probably the linux kernel that boots in recovery mode has no zfs support (i don't really know)
he had a nvme disk failure so i used it once

this is what google ai says about it
Hetzner's standard Linux rescue system does not load ZFS kernel modules by default, but you can manually install and activate ZFS support inside the rescue environment to import or repair your pools. [1, 2, 3]

Using ZFS in the Hetzner Rescue System
    • Boot into Rescue Mode: Activate the rescue system via the Hetzner Robot (for dedicated servers) or the Cloud Console, then reboot your server. [1, 2]
    • Install ZFS Packages: Connect via SSH and install the ZFS utilities and kernel modules using the rescue system's package manager (e.g., apt-get update && apt-get install zfsutils-linux on a Debian/Ubuntu-based rescue system). [1]
    • Load the Module: Ensure the ZFS module is loaded into the kernel by running modprobe zfs.
    • Import Your Pool: Locate your pools using zpool import and mount them with zpool import -f <pool_name> or specify an alternate root using -R /mnt.
Click to expand...

A Linux ZFS kernel cannot necessarily access a FreeBSD-created poll, though. If FreeBSD activated a single feature in the pool that Linux doesn't have it won't import.
 
cracauer@ said:
A Linux ZFS kernel cannot necessarily access a FreeBSD-created poll, though. If FreeBSD activated a single feature in the pool that Linux doesn't have it won't import.
Click to expand...
then you get creative and use iscsi over ssh and fix it on your freebsd box :)
 
some of them may have but the box my friend rents does not. also they have some kvm support but you setup a kind of meeting and a guy comes with an ip kvm at your box (or it may be a robot but kvm its not always available) but thats an annoying solution. also the time for kvm usage is/was limited to one hour or something

but in the end whatever i don't care, im not hetzners competitor or anything
_martin said:
covacat When I look at Differences between managed servers and bare metal servers I see full KVM/IPMI is provided. KVM being keyboard/video/mouse in this context.
Click to expand...
 
_martin said:
What exactly do you mean by "they don't support FreeBSD"? With dedicated HW it's up to you what you install there. You have an option to upload your custom ISO there too (you can run your own OS too).
VMs are, as far as I know, qemu+kvm so it's largely supported.
Click to expand...

I mean it in the sense as there is no option to get a VPS (or root server) pre-installed with FreeBSD, only various linux variants. Of course you can always install whatever you want (if you know how), but since OP asked for "native FreeBSD support", which pretty much boils down to "can be ordered/set up with FreeBSD from the backend", I added the information that manitu doesn't provide this.
 
sko said:
I mean it in the sense as there is no option to get a VPS (or root server) pre-installed with FreeBSD, only various linux variants
Click to expand...
If that's what is implied by "no support" then covacat's solution seems to be the best one for the situation in Hetzner - deploy HW with Linux, spool VMs on top of it.

I checked in Azure (GWC) -- there are preinstalled images available. That's also an option.
 
Thanks very much to all for your replies and suggestions. Replying in one big message:

sko said:
OP asked for "native FreeBSD support", which pretty much boils down to "can be ordered/set up with FreeBSD from the backend"
Click to expand...
For clarity, I'm happy with the ISO boot support that gergeskirolos59 mentioned, though in that case I expect I might need to do a bit more work to configure hardware drivers etc. Indeed, imho, installing from your own image is preferable since it reduces the risk of a supply chain attack where the pre-configured base image has been modified (appreciating you need to trust your hoster nevertheless!)

covacat said:
you can run linux on bare metal @ hetzner and create as many freebsd vm-s as you can
Click to expand...
I made a conscious switch from Linux some years ago and have not looked back. Our team is small and we don't have a dedicated infrastructure role so having fewer things to learn is an advantage. Running FreeBSD as a VM on a Linux host would be viable (and is effectively what you are doing on a VPS) but definitely not my first choice; I would much prefer FreeBSD as the base operating system we're looking after since we already know how to secure and maintain it.

_martin said:
You have an option to upload your custom ISO [to Hetzner] too
Click to expand...
Thanks, _martin , via the link you posted I found Hetzner's "Installation via KVM Console and USB drive". So it looks like it's totally achievable on a dedicated server, with the caveat that you need to make a support request rather than just clicking through a web interface at any time for console access. I'm on the fence as to whether this is an acceptable level of inconvenience. For my particular use, close to 100% uptime isn't crucial. It's less about time waiting and more about how much extra time it might take to fix something if it goes wrong.

_martin said:
I checked in Azure (GWC) -- there are preinstalled images available
Click to expand...
Both AWS and Azure have excellent support for FreeBSD. Our FreeBSD-based appliance runs nicely in both and even after our move we'll maintain a presence with the hyperscalers for testing. Azure has a similar level of administrative complication to AWS and also similar pricing when it comes to running instances with plenty of RAM and/or compute power. I'm prepared to pay for hosting to reduce our management overhead but imho both AWS and Azure add quite considerably to that overhead, hence the appeal of running our own jails and VMs inside our own box where we can be masters of our own destiny rather than doing battle with AWS/Azure configuration settings.

Edit: Fixed "Installation via KVM Console and USB drive" link
 
_martin said:
deploy HW with Linux, spool VMs on top of it.
Click to expand...
why should i deal with something like linux underneath? By now they removed every sane unix-related tool that made their "cli-focused-windows-derivate" barely tolerateable.
I'd rather cut off my fingers and never touch a computer again...

asteriskRoss said:
imho, installing from your own image is preferable since it reduces the risk of a supply chain attack
Click to expand...
100% this. although my biggest concern wouldn't be a supply chain attack, but usually pre-configured images from most VPS-hosters come with some kind of cloud-init that for absolutely no reason is often python-based and therefore drags in several tons of garbage, which is a far bigger attack surface.
 
sko said:
why should i deal with something like linux underneath? By now they removed every sane unix-related tool that made their "cli-focused-windows-derivate" barely tolerateable.
I'd rather cut off my fingers and never touch a computer again...
Click to expand...
First of all this was covacat's idea:
covacat said:
you can run linux on bare metal @ hetzner and create as many freebsd vm-s as you can.
Click to expand...

so if you want to quote somebody with childish reaction quote the original poster. However I do agree with him.
If you want to cut your fingers, do. Here: ✂️.

If you've followed the thread with comprehension you'd see my first reaction where I said there's no problem installing FreeBSD in Hetzner and you can even pull your own ISO image. However, OP, asteriskRoss, has some contraints in this regard on hours and support required to do so. There's where our suggestions are stemming from.
 
asteriskRoss said:
So it looks like it's totally achievable on a dedicated server, with the caveat that you need to make a support request rather than just clicking through a web interface at any time for console access. I'm on the fence as to whether this is an acceptable level of inconvenience.
Click to expand...
I asked my colleague who is using Hetzner as part of their business infrastructure (DR in Azure). They rent physical HW (ESX, Linux). KVM can be requested, in contract they have few hours for TTR (time to respond) but usually do that in few minutes. Linux boxes are usually troubleshooted by booting to rescue image. He didn't complain about the KVM usage time constraint but it makes sense if the KVM is shared among more customers.

This problem exists in Azure too, lack of console access can be painful. Also many things can't be solved by just having serial console. Then you have same approach - use the disk in different VM and troubleshoot.
 
This whole adventure has been far more difficult than I was expecting. I've looked through a lot of websites and spoken with some sales people. I think I've now accepted that cheap and good are difficult to achieve in one place! Licencing Windows, which we need for testing, is another factor. We need to test against lots of different versions, though not every day, which makes licencing for bare-metal prohibitively expensive. Cloud providers are definitely a win for that one, where the licences are included automatically in the instance cost.

Currently in the running:
  • Stick with AWS (or Azure) and just try to make our use of it much more efficient by minimising the specs on instances and using ephemeral instances for CI. We need a presence in both of those anyway, since that's where some of our customers are.
  • Move to Krystal, which is a Vultr-like VPS provider and looks to charge per second, which would make our ephemeral machines viable. No official FreeBSD support, but support for uploading your own image
  • Hetzner for bare metal due to a combination of choice, short contracts and cost
  • Some hybrid of the above: Move our always-on services off AWS to a cheaper VPS provider and then use AWS or Azure for our ephemeral machines, or use a Hetzner box for predictable cost and performance, whilst not relying on its uptime for (semi-)critical services.
 
You must log in or register to reply here.
Back
Top