Prehistory: I want to build kernel and world on a fast virtual machine for other ones (all the virtual machines are hosted on VMware EXSi hypervisors, which have slightly different physical CPUs). I had read ``23.6. Tracking for Multiple Machines'' chapter in the handbook, and shared /usr/src and /usr/obj on the build machine to be accessed from other virtual hosts, which mount the shares.
Problem: After
I believe the problem isn't specific to virtualized CPUs or VMware ESXi hypervisor, because hardware failure is just an expected consequence. Therefore, I posted this on FreeBSD forums, not in VMware forums.
Details/steps to reproduce:
Build machine
FreeBSD-13.0-CURRENT r354802 (custom kernel, amd64).
Both /etc/make.conf and /etc/src.conf don't exist.
CPU capabilities:
Bit 08 in EBX is set -- the CPU supports BMI2.
Build succeeded, check/view the instructions:
Client machine
FreeBSD-13.0-CURRENT r354802 (custom kernel, amd64).
Bit 08 in EBX is cleared -- the CPU doesn’t support BMI2; therefore:
I get that error from above during
Moreover, the generated install program also contains AVX512F's instruction vpbroadcastq (supported by build machine's CPU):
And that causes FreeBSD-13.0-CURRENT running on physical Intel Core i3-8300T (not supporting AVX512) to fail on that program too.
Question: How to disable using those instruction set extensions on my build machine? Or may be I do something wrong, and to be able to share built world and kernel between several machines I must do something else?
Problem: After
make -j16 buildworld has finished successfully on the build machine, I get on that machine, for example, /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/install program having the shlxq instruction (one from the BMI2 instruction set extensions). This eventually causes make installkernel and make installworld to fail with SIGILL on a virtual machine which must consume built world and kernel, and which is hosted on another ESXi instance, with older physical CPU (read: with CPU not implementing shlxq).I believe the problem isn't specific to virtualized CPUs or VMware ESXi hypervisor, because hardware failure is just an expected consequence. Therefore, I posted this on FreeBSD forums, not in VMware forums.
Details/steps to reproduce:
Build machine
FreeBSD-13.0-CURRENT r354802 (custom kernel, amd64).
Both /etc/make.conf and /etc/src.conf don't exist.
CPU capabilities:
Bash:
# cpucontrol -i 7 /dev/cpuctl0
cpuid level 0x7: 0x00000000 0xd19f6ffb 0x00000018 0xbc000000Bit 08 in EBX is set -- the CPU supports BMI2.
Bash:
# make clean && rm -r /usr/obj/*
# make -j16 buildworldBuild succeeded, check/view the instructions:
Code:
$ lldb /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/install
(lldb) disassemble -a 0x26152b
...
install[0x26152b] <+139>: shlxq %rdx, %rsi, %rsi
...Client machine
FreeBSD-13.0-CURRENT r354802 (custom kernel, amd64).
Bash:
# cpucontrol -i 7 /dev/cpuctl0
cpuid level 0x7: 0x00000000 0x00000002 0x00000000 0xbc000000Bit 08 in EBX is cleared -- the CPU doesn’t support BMI2; therefore:
Bash:
root@:/usr/src # /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/install
pid 880 (install), jid 0, uid 0: exited on signal 4
illegal instructionI get that error from above during
make installkernel and/or make installworld.
Bash:
root@:/usr/src # lldb /usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/install
Current executable set to '/usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/install' (x86_64)
(lldb) r
Process 878 launching
Process 878 launched: '/usr/obj/usr/src/amd64.amd64/tmp/legacy/bin/install' (x86_64)
Process 878 stopped
* thread #1, name = 'install', stop reason = signal SIGILL: privileged instruction
install`base_block_alloc:
-> 0x26152b <+139>: shlxq %rdx, %rsi, %rsi
0x261530 <+144>: movq $-0x1000, %rdx ; imm = 0xF000Moreover, the generated install program also contains AVX512F's instruction vpbroadcastq (supported by build machine's CPU):
Code:
$ lldb install
(lldb) disassemble -a 0x280821
...
install[0x280821] <+225>: vpbroadcastq %rcx, %zmm0
...And that causes FreeBSD-13.0-CURRENT running on physical Intel Core i3-8300T (not supporting AVX512) to fail on that program too.
Question: How to disable using those instruction set extensions on my build machine? Or may be I do something wrong, and to be able to share built world and kernel between several machines I must do something else?
