Some questions about FreeBSD-SA-19:07.mds

Zirias

Well-Known Member

Reaction score: 175
Messages: 486

I'm having a few doubts here. First about the microcode:
Ensure that the BIOS update or devcpu-data package is dated after 2014-05-14.
Is this a typo? Or does it mean there is no microcode update addressing this specific problem yet, but the mitigation needs some instruction introduced much earlier?

Mitigation Configuration

Systems with users, processes, or virtual machines in different trust
domains should disable Hyper-Threading
[...]

To activate the MDS mitigation set the hw.mds_disable sysctl.
[...]
What exactly does this mitigation do? Why is there the recommendation to still disable HT? Does that mean the mitigation is known not to address all possible problems or does it mean it's just unknown whether there's still a risk left with HT?
 
OP
OP
Zirias

Zirias

Well-Known Member

Reaction score: 175
Messages: 486

Is this a typo?
It obviously was, the date now reads "2019-05-14" ;)

Anyways, I'm still curious about the background of the mitigation and the recommendation to disable HT ...
 
OP
OP
Zirias

Zirias

Well-Known Member

Reaction score: 175
Messages: 486

Does this help?
Not so much, unfortunately.

Without knowing what FreeBSD's mitigation does, it's hard to assess whether you should disable HT. Does FreeBSD's mitigation somehow classify processes in "security domains"? If so, how?

For example, I have a virtual machine with Windows 7 running on my server, it's for testing purposes and only has internet access -- no way to reach anything in my local network. So, if this machine catches some malware, it's not a big deal -- once noticed, just reset it to an earlier ZFS snapshot. Now, without disabling HT, do I have to worry that malware on this machine could somehow spy on processes running on my FreeBSD host? If for example the mitigation would make sure that bhyve processes will never share a core with processes of the host or of other bhyve VMs, I'd say it's fine for me to keep HT enabled. But is this the case?
 

SirDice

Administrator
Staff member
Administrator
Moderator

Reaction score: 7,293
Messages: 29,742

Without knowing what FreeBSD's mitigation does, it's hard to assess whether you should disable HT. Does FreeBSD's mitigation somehow classify processes in "security domains"? If so, how?
That question is probably best asked on the freebsd-security mailing list. I'm not sure if we have forum members with that level of intimate knowledge.
 

PMc

Well-Known Member

Reaction score: 81
Messages: 295

Not so much, unfortunately.

Without knowing what FreeBSD's mitigation does
Looking at the code, it does bring in some assembler code differenciated for various CPU types from intel core gen2 upwards, and/or some use of an assember CPU instruction verw and a sysctl(8) switch hw.mds_disable.

Concerning that instruction, I could find this snippet:
The Intel CPU instruction called "VERW" is enhanced by the new CPU Microcode such that it flushes all buffers and ports. The VERW instruction will be called during task switch or VM switch by the patched kernels and hypervisors.
This explanation comes from some Linux, but one might assume it's the same here.

, it's hard to assess whether you should disable HT.
It is indeed, but that gets someway clear from the description of the thing in some of the papers floating around:
They say there are basically two different attack methods. One is to obtain unaccessible data when a thread transites from kernel to userland (probably difficult to implement such an attack), the other is that two threads are running on the same core in hyperthreading mode, and one obtains data owned by the other (the more common and demonstrated way).
It further says that cleaning the microarch structures does not fully help against the second scheme, because this cleaning happens within one thread, while the other could still obtain something.
Therefore they recommend to swich off HT in cases where a core could be shared by hostile threads.

Does FreeBSD's mitigation somehow classify processes in "security domains"? If so, how?
Sorry, I have no idea.

For example, I have a virtual machine with Windows 7 running on my server, it's for testing purposes and only has internet access -- no way to reach anything in my local network. So, if this machine catches some malware, it's not a big deal -- once noticed, just reset it to an earlier ZFS snapshot. Now, without disabling HT, do I have to worry that malware on this machine could somehow spy on processes running on my FreeBSD host? If for example the mitigation would make sure that bhyve processes will never share a core with processes of the host or of other bhyve VMs, I'd say it's fine for me to keep HT enabled. But is this the case?
Wouldn't it be possible to dedicate or reserve specific cores (and only complete cores) to each installation? On FreeBSD this could be done with cpuset(1) (I don't touch windows).
 
OP
OP
Zirias

Zirias

Well-Known Member

Reaction score: 175
Messages: 486

It further says that cleaning the microarch structures does not fully help against the second scheme, because this cleaning happens within one thread, while the other could still obtain something.
Therefore they recommend to swich off HT in cases where a core could be shared by hostile threads.
Thanks, this already explains a lot. So probably, FreeBSD's mitigation just makes sure to issue a VERW at appropriate times, but doesn't change scheduling so that bhyve and host processes don't share the same core...
That's bad news for me, and I guess I'll have to live with the risk here.
Wouldn't it be possible to dedicate or reserve specific cores (and only complete cores) to each installation?
Just pinning bhyve to a specific core wouldn't help, I'd also have to make sure any other process doesn't use that core. As my server only has 4 cores, doing that (possibly with several virtual machines) would be even worse than disabling HT. A good solution for my specific usecase would be if the scheduler would block the core as long as a bhyve process is active on it...
 

hsw

New Member

Reaction score: 3
Messages: 16

Just hit this problem system was really unstable random memory corruption random segfaults
because pkg repo does not have up-to-date devcpu-data (v1.21) had to build from ports (v1.22)
should have realised somthing was wrong when "sysctl hw.mds_disable_state" showed "Software ..." and not "VERW"
 

PMc

Well-Known Member

Reaction score: 81
Messages: 295

Yeah, here it illuminates only half of the monitor anymore...
 

CyberCr33p

Active Member

Reaction score: 14
Messages: 176

In mds advisory they say:

Code:
Perform one of the following:

Update CPU microcode, upgrade your vulnerable system to a supported FreeBSD
stable or release / security branch (releng) dated after the correction date,
evaluate mitigation and Hyper Threading controls, and reboot the system.
So we either need to use updated microcode or upgrade to 12.0-RELEASE-p5?

I just upgrade to p5 and I see:

Code:
sysctl hw.mds_disable_state
hw.mds_disable_state: inactive
So it's disabled by default and we have to set hw.mds_disable_state during boot.

They recommend to use hw.mds_disable=3 to autoselect if the microcode or the code will be used.

Is the microcode update better for performance?
 

PMc

Well-Known Member

Reaction score: 81
Messages: 295

In mds advisory they say:

Code:
Perform one of the following:

Update CPU microcode, upgrade your vulnerable system to a supported FreeBSD
stable or release / security branch (releng) dated after the correction date,
evaluate mitigation and Hyper Threading controls, and reboot the system.
So we either need to use updated microcode or upgrade to 12.0-RELEASE-p5?
NO! Both!

This advisory is horrible. It uses a terrible language, a creepy formatting, and needs a lawyer to understand what it says. :(

The stance "Perform one of the following" does not apply to the directly following paragraph!
It does apply to the numbered list that follows much later.

So it's disabled by default and we have to set hw.mds_disable_state during boot.

They recommend to use hw.mds_disable=3 to autoselect if the microcode or the code will be used.
Yepp.

Is the microcode update better for performance?
I might suppose so, but I don't know for certain.

I have set
Code:
pmc@disp:501:1~$ grep mds /etc/sysctl.conf 
hw.mds_disable=3
pmc@disp:502:1~$
and now it shows

Code:
pmc@disp:504:1~$ /sbin/sysctl -a | grep mds_
hw.mds_disable: 3
hw.mds_disable_state: software IvyBridge
I have not installed the microcode - if there is any for IvyBridge - because nobody did yet bother to put it into the 2019Q2 ports branch.
 
Top