Solved [GELI] getting rid of bootpool?

Oh wow, that beast finally boots directly from the encrypted pool.

The last problem was the efi bootloader. While gptzfsboot supports decryption, boot1.efi does not. But the trick is, you can instead put loader.efi directly on the efi partition. One of the rare cases where I find official FreeBSD docs a bit lacking (or did I look in all the wrong places?)

Of course, my efi partitions were too small, had to resize them first. 🧐

Now, this bootpool can finally die for good, and my reward is working boot environment selection in the loader menu. 🥳

Marking this solved, maybe I'll condense it in a little howto later…
 
One of the rare cases where I find official FreeBSD docs a bit lacking (or did I look in all the wrong places?)
I have to correct myself: while uefi(8) isn't totally clear when you still have boot1.efi in mind, there's also boot1.efi(8) (newly added in 13) which would have helped me, stating it is deprecated and doesn't support encryption. Well, it was the way FreeBSD 11 booted from efi :rolleyes:.
 
Hi Zirias,

I apologize for the off-topic, but I have a question to Mjölnir - why is it a malpractice to have the firewall/packet filter in a VM? Is it not the idea of VM to insulate the OS?

Kindest regards,

M
 
I apologize for the off-topic, but I have a question to Mjölnir - why is it a malpractice to have the firewall/packet filter in a VM? Is it not the idea of VM to insulate the OS?
Read a book about firewalls (it's general architectural properties). When an attacker succeeds to break out of a VM/jail'ed service on that very same physical machine, AND succeeds to break out of that service's VM or jail, then s/he can access the packet filter. That's less than sub-optimal... Therefore, a clean implementation is to put IIRC the internal packet filter on it's own dedicated physical machine, with separate management LAN (ideally own cables to dedicated management laptop), and no other services on that machine. Then it can neither be attacked from outside nor from the inside, except from the management laptop.
 
Hi Mjölnir,

thank you for the response. I understand what you are saying. But, it then begs two questions (1) what is the purpose of VM/jail, and (2) since the separate machine with the packet filter is running on underlying OS and is connected to the reminder of the network, cannot the attacker take control of said separate machine and continue atttack on the rest of the network?

Kindest regards,

M
 
  • open a separate thread if you're interested in these topics (e.g. Networking -> Security)
  • get a good book on firewalls (packet filter is one part of) from store or your local public library, and/or search online for a good introductory tutorial
 
To put things into perspective here: Security isn't something that can be "solved", it's a continuous process and an essential part of it is understanding risks and attack vectors, and of course, assessment and management of these risks.

What Mjölnir describes here is the possible attack vector with any firewall in a VM. If this is the only one left, it is already pretty good, of course depending on what you compare with.

A business operating their own infrastructure will typically operate many physical machines anyways, so they would be crazy not to reduce the risk further by using a dedicated machine (most of the time a special-purpose appliance) for firewalling. But using separate physical cables is a different story; in a larger network, you WILL find some shared trunks transporting multiple VLANs, one of them being the management LAN. The additional cost for cabling often isn't considered justified; assuming VLANs are used and configured correctly, you'd need strange bugs in switch firmware to "intrude" (and then you'd still have to gain remote access on the actual firewall).

But then, look at a home or (very) small business scenario, with just one server. A dedicated firewall appliance might be over the top. So, do you want to use one of these consumer "plastic routers" for firewalling? I'd have my doubts there about the quality and maintenance of their firmware, and there sure have been problems in the past. Putting a firewall in a VM is IMHO the second best solution after "dedicated hardware", but you have to do it right. If you wire up your firewall on virtual networks provided by the host system, you're doing it wrong: the host's networking code would still handle all packages first, including those the firewall would block, so any remote vulnerability in networking code could lead to a breach of the whole physical machine. Using PCI-passthrough for all NICs, giving the firewall VM exclusive access to the hardware, is already a lot better.

What's left is indeed the risk an attacker could gain control of the host by breaking out of a jail or vm. Flaws in CPUs (hello intel) render this a bit more realistic. Still, to even attempt such an attack, you'd first have to get access to a jail or vm.

Well, just some context, and there are many more things to take into account for every decision regarding security – bottom line is, for firewalling, a dedicated machine is the best solution (but needs trusted and maintained software as well of course, see "plastic routers"), and if you can afford it, you should. Still, a VM can be a reasonable alternative…
 
Hi Zirias,

first thank you for the detailed explanation. I do understand, at least on the conceptual level, the problem, the fact that security is not absolute, and that the more defenses one puts up, the better. But, in my naive brain, I still cannot fully distinguish the two scenarios.

In the first one, the attacker breaks out of a jail. But, does he still not need gain access to the root? In the second scenario, the firewall running on the dedicated machine is subject to the same problem, with flaws in the hardware and software, thus, like in the first one, he can get access to the dedicated machine. And as a corollary, nothing prevents him form continuing the attack on other machines on the network, i.e., gain access to their roots.

From a more conceptual level, there is lot to gain from accessing BigCompany(tm) machines: know-how, trade secrets, business posture, etc. Thus an effort, requiring quite a sophistication is justified. What is to gain from attacking small people like many of us? Perhaps access to computing resource for illicit activities or ransom. Does that justify the continued effort once the attacker find itself in a jail, on a rather obscure OS, (no disrespect to FreeBSD), or is his time better spent by trying elsewhere?

Kindest regards,

M
 
OT since 6 posts... This will be much easier to find by others, who can then also give their 2¢, if you move over to "Server & Networking -> Firewalls".
 
In the first one, the attacker breaks out of a jail. But, does he still not need gain access to the root? In the second scenario, the firewall running on the dedicated machine is subject to the same problem, with flaws in the hardware and software, thus, like in the first one, he can get access to the dedicated machine. And as a corollary, nothing prevents him form continuing the attack on other machines on the network, i.e., gain access to their roots.
The first scenario is breaking out of some jail or vm running on the physical host OTHER than the firewall vm. Once you manage to break out of a VM, getting root on the host won't be the problem any more. Breaking out of a VM is the difficult part, it's possible in theory (e.g. thanks to CPU bugs), but not *that* likely. But once you control the physical host, you can also control the firewall VM hosted on it.

This isn't possible when the firewall is a dedicated physical machine.

Whether an attacker can gain access to some other VM or jail in the first place depends on what's running there (services) and when some of them are exposed to the internet (e.g. from some DMZ zone), the risk increases.

Getting remote access to the firewall itself is even less likely, it typically doesn't run any services except e.g. an sshd only reachable from your management network, so you'd need a bug in its networking or firewalling code itself for that.
 
Why don't you just open a thread like e.g. "Howto design a firewall (system) for SOHO vs. professional grade use case?" in "Server & Networking -> Firewalls". Then chances are it's not a 2-person dialog.
 
Hi Zirias,

thank you for the reply. I did not realize that as you wrote: "Once you manage to break out of a VM, getting root on the host won't be the problem any more", I thought that getting a root is not such an easy proposition. Now, once one controls the root, all bets are off.

Regarding running sshd on the firewall, I understand that this may be necessary if one wants to access the system form outside. However, when I read about firewalls when setting up a pf(4), the attack against ssh are the most prevalent.

I think that based on the attack danger I perceive, your solution is very elegant, if one wants to run only a single computer. Though, with the low cost of Raspberry Pi4, it might be viable to have a firewall on a separate hardware if one is really paranoid. In that regards, I think I would prefer that solution because I agree with you about the "plastic routers". My landlord has one and it is no longer supported, so who knows what bugs are present in the closed firmware. Hence, I run firewall on each of my individual computers, though not as sophisticated as yours. Do you think that it would be worth for you to write a How-to?

Kindest regards,

M
 
Just went through this procedure. Thought I’d share my process. To suss this out safely, I made a copy of my raw disks into qcow2 images and performed testing in a VM. I highly recommend this as it allows for trying out procedures without fear of affecting the live system.

  1. Even with testing, it’s probably a good idea to have a FreeBSD “livecd” handy in case things go awry. And, as always, have some backups.
  2. Ensure that there are no GELI keyfiles in the bootpool. If there are, move them elsewhere or remove them entirely with e.g. geli setkey -v -k /boot/keyfile /dev/ada0. Likewise, should update geli_*_keyfile* entries in /boot/loader.conf.
  3. Replace the existing /boot → /bootpool/boot with rm /boot && cp -r /bootpool/boot /.
  4. If your rootpool is encrypted, then ensure it’s configured for boot with geli configure -b ada0 && geli configure -g ada0
  5. Ensure that the rootpool has bootfs set with zpool set bootfs=zroot/ROOT/default zroot.
  6. Reboot to validate.
  7. If successful, zpool destroy bootpool. If not, then, well, the bootpool is still there so it’s workable.
  8. Reboot to validate
Of course, modify device, pool, and dataset names accordingly. Note that my system uses BIOS, not UEFI boot. Some things may be different in that case. My original install was FreeBSD 10 and is presently running 12.
 
Thanks amarshall ! Your steps helped me a lot. There is one thing I'd like to add:

After the reboot in step 6 the system still boots from the bootpool (given that the partition for the bootpool comes before the partition for the zroot pool). My GPT partition layout was created by an earlier FreeBSD installation (maybe version 9 or 10):

Bash:
gpart show ada0
=>         40  15628053088  ada0  GPT  (7.3T)
           40         1024     1  freebsd-boot  (512K)
         1064          984        - free -  (492K)
         2048      4194304     2  freebsd-zfs  (2.0G)
      4196352      4194304     3  freebsd-swap  (2.0G)
      8390656  15619662472     4  freebsd-zfs  (7.3T)

The bootpool resides in partition index 2 with type freebsd-zfs. To "hide" this partition from gptzfsboot I changed its type:

Bash:
gpart modify -i 2 -t freebsd-ufs ada0

Then gptzfsboot boots from the zroot pool in partition index 4.

After changing loader.conf in /boot copied to the zroot pool I noticed that there was no change during the boot process. Only after I changed the partition type the changes in loader.conf were applied.
 
Back
Top