bhyve Bhyve: Linux kernel 4.19 doesn't boot, but 4.14 does

U

Utumno

Hello,

I am trying to run Debian Linux under bhyve. Results:

1) Debian 9 with kernel 4.9 boots
2) self-compiled vanilla 4.14 from kernel.org boots
3) Debian 10 with kernel 4.19 does not boot
4) self compiled vanilla 4.19 from kernel.org does not boot
5) another Debian kernel v. 5.6 does not boot

It's really starting to look like sometime between 4.14 and 4.19 they added something which breaks booting under bhyve, or at least my installation of bhyve.

When it 'does not boot' it always stops at precisely the same place. Log below:
Code: 
[    0.000000] Linux version 4.19.0-13-amd64 ([email]debian-kernel@lists.debian.org[/email]) (gcc version 8.3.0 (Debian 8.3.0-6)) #1 SMP Debian 4.19.160-2 (2020-11-28)
[    0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.19.0-13-amd64 root=UUID=77172eeb-a5cc-444b-88f8-244015ff9673 ro console=ttyS0
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[    0.000000] BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009ffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007eaa7fff] usable
[    0.000000] BIOS-e820: [mem 0x000000007eaa8000-0x000000007eab8fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007eab9000-0x000000007fb38fff] usable
[    0.000000] BIOS-e820: [mem 0x000000007fb39000-0x000000007fb68fff] type 20
[    0.000000] BIOS-e820: [mem 0x000000007fb69000-0x000000007fb90fff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007fb91000-0x000000007fb98fff] ACPI data
[    0.000000] BIOS-e820: [mem 0x000000007fb99000-0x000000007fb9cfff] ACPI NVS
[    0.000000] BIOS-e820: [mem 0x000000007fb9d000-0x000000007ffcffff] usable
[    0.000000] BIOS-e820: [mem 0x000000007ffd0000-0x000000007ffeffff] reserved
[    0.000000] BIOS-e820: [mem 0x000000007fff0000-0x000000007fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] efi: EFI v2.40 by BHYVE
[    0.000000] efi:  SMBIOS=0x7fb6b000  ACPI=0x7fb98000  ACPI 2.0=0x7fb98014 
[    0.000000] secureboot: Secure boot could not be determined (mode 0)
[    0.000000] SMBIOS 3.0 present.
[    0.000000] DMI:  BHYVE, BIOS 1.00 03/14/2014
[    0.000000] last_pfn = 0x80000 max_arch_pfn = 0x400000000
[    0.000000] Disabled
[    0.000000] x86/PAT: MTRRs disabled, skipping PAT initialization too.
[    0.000000] CPU MTRRs all blank - virtualized system.
[    0.000000] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WB  WT  UC- UC  
[    0.000000] found SMP MP-table at [mem 0x000f0000-0x000f000f]
[    0.000000] Using GB pages for direct mapping
[    0.000000] RAMDISK: [mem 0x34e85000-0x36739fff]
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI: RSDP 0x000000007FB98014 000024 (v02 BHYVE )
[    0.000000] ACPI: XSDT 0x000000007FB970E8 00004C (v01 BHYVE  BVFACP   00000001      01000013)
[    0.000000] ACPI: FACP 0x000000007FB96000 0000F4 (v04 BHYVE  BVFACP   00000001 BHYV 00000001)
[    0.000000] ACPI: DSDT 0x000000007FB91000 0009A3 (v02 BHYVE  BVDSDT   00000001 INTL 20170303)
[    0.000000] ACPI: FACS 0x000000007FB9C000 000040
[    0.000000] ACPI: HPET 0x000000007FB95000 000038 (v01 BHYVE  BVHPET   00000001 BHYV 00000001)
[    0.000000] ACPI: APIC 0x000000007FB94000 000052 (v01 BHYVE  BVMADT   00000001 BHYV 00000001)
[    0.000000] ACPI: MCFG 0x000000007FB93000 00003C (v01 BHYVE  BVMCFG   00000001 BHYV 00000001)
[    0.000000] ACPI: SPCR 0x000000007FB92000 000050 (v01 BHYVE  BVSPCR   00000001 BHYV 00000001)
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at [mem 0x0000000000000000-0x000000007fffffff]
[    0.000000] NODE_DATA(0) allocated [mem 0x7fffb000-0x7fffffff]
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.000000]   DMA32    [mem 0x0000000001000000-0x000000007fffffff]
[    0.000000]   Normal   empty
[    0.000000]   Device   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000001000-0x000000000009ffff]
[    0.000000]   node   0: [mem 0x0000000000100000-0x000000007eaa7fff]
[    0.000000]   node   0: [mem 0x000000007eab9000-0x000000007fb38fff]
[    0.000000]   node   0: [mem 0x000000007fb9d000-0x000000007ffcffff]
[    0.000000]   node   0: [mem 0x000000007fff0000-0x000000007fffffff]
[    0.000000] Zeroed struct page in unavailable ranges: 246 pages
[    0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000007fffffff]
[    0.000000] ACPI: PM-Timer IO Port: 0x408
[    0.000000] IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-31
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0xa400 base: 0xfed00000
[    0.000000] ACPI: SPCR: SPCR table version 1
[    0.000000] ACPI: SPCR: console: uart,io,0x3f8,115200
[    0.000000] smpboot: Allowing 2 CPUs, 0 hotplug CPUs
[    0.000000] PM: Registered nosave memory: [mem 0x00000000-0x00000fff]
[    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]
[    0.000000] PM: Registered nosave memory: [mem 0x7eaa8000-0x7eab8fff]
[    0.000000] PM: Registered nosave memory: [mem 0x7fb39000-0x7fb68fff]
[    0.000000] PM: Registered nosave memory: [mem 0x7fb69000-0x7fb90fff]
[    0.000000] PM: Registered nosave memory: [mem 0x7fb91000-0x7fb98fff]
[    0.000000] PM: Registered nosave memory: [mem 0x7fb99000-0x7fb9cfff]
[    0.000000] PM: Registered nosave memory: [mem 0x7ffd0000-0x7ffeffff]
[    0.000000] [mem 0x80000000-0xffffffff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on bare hardware
[    0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645519600211568 ns
[    0.000000] random: get_random_bytes called from start_kernel+0x93/0x52a with crng_init=0
[    0.000000] setup_percpu: NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:2 nr_node_ids:1
[    0.000000] percpu: Embedded 45 pages/cpu s144536 r8192 d31592 u1048576
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 513779
[    0.000000] Policy zone: DMA32
[    0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.19.0-13-amd64 root=UUID=77172eeb-a5cc-444b-88f8-244015ff9673 ro console=ttyS0
[    0.000000] Memory: 1983152K/2096168K available (10252K kernel code, 1242K rwdata, 3320K rodata, 1596K init, 2264K bss, 113016K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.000000] Kernel/User page tables isolation: enabled
[    0.000000] ftrace: allocating 31960 entries in 125 pages
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=512 to nr_cpu_ids=2.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[    0.000000] NR_IRQS: 33024, nr_irqs: 512, preallocated irqs: 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] ACPI: Core revision 20180810
[    0.000000] clocksource: hpet: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 113919999973 ns
[    0.000000] APIC: Switch to symmetric I/O mode setup
[    0.000000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.024000] tsc: PIT calibration matches PMTIMER. 1 loops
[    0.024000] tsc: Detected 2098.398 MHz processor
[    0.000014] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x1e3f471f330, max_idle_ns: 440795257933 ns
[    0.002491] Calibrating delay loop (skipped), value calculated using timer frequency.. 4196.79 BogoMIPS (lpj=8393592)
[    0.006478] pid_max: default: 32768 minimum: 301
[    0.007898] Security Framework initialized
[    0.008879] Yama: disabled by default; enable with sysctl kernel.yama.*
[    0.010525] AppArmor: AppArmor initialized
[    0.016725] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.019808] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.021480] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes)
[    0.022482] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes)
[    0.024728] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
[    0.026477] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
[    0.027901] Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
[    0.029902] Spectre V2 : Mitigation: Full generic retpoline
[    0.030476] Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
[    0.032415] Speculative Store Bypass: Vulnerable
[    0.033564] TAA: Mitigation: Clear CPU buffers
[    0.034476] MDS: Mitigation: Clear CPU buffers
[    0.035734] Freeing SMP alternatives memory: 24K
[    0.148787] smpboot: CPU0: Intel(R) Xeon(R) CPU E5-2620 v4 @ 2.10GHz (family: 0x6, model: 0x4f, stepping: 0x1)
[    0.150475] Performance Events: unsupported p6 CPU model 79 no PMU driver, software events only.
[    0.150543] rcu: Hierarchical SRCU implementation.
[    0.152108] random: crng done (trusting CPU's manufacturer)
[    0.153452] NMI watchdog: Perf NMI watchdog permanently disabled
[    0.154527] smp: Bringing up secondary CPUs ...
[    0.155695] x86: Booting SMP configuration:
[    0.156697] .... node  #0, CPUs:      #1
[    0.242508] smp: Brought up 1 node, 2 CPUs
[    0.244416] smpboot: Max logical packages: 2
[    0.245444] smpboot: Total of 2 processors activated (8393.27 BogoMIPS)
[    0.246700] devtmpfs: initialized
[    0.247347] x86/mm: Memory block size: 128MB
[    0.248699] PM: Registering ACPI NVS region [mem 0x7fb99000-0x7fb9cfff] (16384 bytes)
[    0.250533] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.252828] futex hash table entries: 512 (order: 3, 32768 bytes)
[    0.254537] pinctrl core: initialized pinctrl subsystem
[    0.255978] NET: Registered protocol family 16
[    0.257134] audit: initializing netlink subsys (disabled)
[    0.258515] audit: type=2000 audit(1612452650.284:1): state=initialized audit_enabled=0 res=1
[    0.260515] cpuidle: using governor ladder
[    0.262482] cpuidle: using governor menu
[    0.263459] ACPI FADT declares the system doesn't support PCIe ASPM, so disable it
[    0.265239] ACPI: bus type PCI registered
[    0.266198] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.266667] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    0.268859] PCI: not using MMCONFIG
[    0.270477] PCI: Using configuration type 1 for base access
[    0.271835] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.274486] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.276102] ACPI: Added _OSI(Module Device)
[    0.276102] ACPI: Added _OSI(Processor Device)
[    0.278482] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.279619] ACPI: Added _OSI(Processor Aggregator Device)
[    0.280912] ACPI: Added _OSI(Linux-Dell-Video)
[    0.281983] ACPI: Added _OSI(Linux-Lenovo-NV-HDMI-Audio)
[    0.282804] ACPI: 1 ACPI AML tables successfully acquired and loaded
[    0.285174] ACPI: Interpreter enabled
[    0.286488] ACPI: (supports S0 S5)
[    0.287318] ACPI: Using IOAPIC for interrupt routing
[    0.288530] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xe0000000-0xefffffff] (base 0xe0000000)
[    0.290557] PCI: MMCONFIG at [mem 0xe0000000-0xefffffff] reserved in ACPI motherboard resources
[    0.292627] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.295918] ACPI: PCI Root Bridge [PC00] (domain 0000 [bus 00-ff])
[    0.298481] acpi PNP0A03:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    0.300430] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[    0.302022] acpi PNP0A03:00: host bridge window expanded to [io  0x0000-0x0cf7]; [io  0x0000-0x0cf7 window] ignored
[    0.302539] PCI host bridge to bus 0000:00
[    0.303539] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
[    0.306478] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff window]
[    0.308100] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[    0.310479] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xdfffffff window]
[    0.312263] pci_bus 0000:00: root bus resource [mem 0xf0000000-0xf07fffff window]
[    0.314050] pci_bus 0000:00: root bus resource [mem 0xd000000000-0xd0000fffff window]
[    0.314478] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.330355] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 *5 6 7 9 10 11 12 14 15)
[    0.330583] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
[    0.332680] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 *6 7 9 10 11 12 14 15)
[    0.334595] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 *7 9 10 11 12 14 15)
[    0.336414] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
[    0.338579] ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
[    0.340681] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 9 *10 11 12 14 15)
[    0.342573] ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 9 10 11 12 14 15) *0, disabled.
[    0.344730] vgaarb: loaded
[    0.346531] pps_core: LinuxPPS API ver. 1 registered
[    0.347730] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <[email]giometti@linux.it[/email]>
[    0.349901] PTP clock support registered
[    0.350495] EDAC MC: Ver: 3.0.0
[    0.351583] Registered efivars operations
[    0.351583] PCI: Using ACPI for IRQ routing
[    0.501554] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0, 0, 0, 0, 0, 0
[    0.502478] hpet0: 8 comparators, 32-bit 16.777216 MHz counter
[    0.506504] clocksource: Switched to clocksource tsc-early
[    0.517671] VFS: Disk quotas dquot_6.6.0
[    0.518651] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.520388] AppArmor: AppArmor Filesystem Enabled
[    0.521531] pnp: PnP ACPI init
[    0.522650] system 00:02: [io  0x0220-0x0223] has been reserved
[    0.524053] system 00:02: [io  0x0224-0x0227] has been reserved
[    0.525453] system 00:02: [io  0x04d0-0x04d1] has been reserved
[    0.526853] system 00:02: [io  0x0400-0x0407] has been reserved
[    0.528256] system 00:02: [mem 0xe0000000-0xefffffff] has been reserved
[    0.529876] pnp: PnP ACPI: found 6 devices
[    0.538274] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[    0.540366] pci 0000:00:03.0: BAR 6: assigned [mem 0xc000a000-0xc000a7ff pref]
[    0.542067] pci 0000:00:05.0: BAR 6: assigned [mem 0xc000a800-0xc000afff pref]
[    0.543769] pci 0000:00:06.0: BAR 6: assigned [mem 0xc000b000-0xc000b7ff pref]
[    0.545469] pci 0000:00:09.0: BAR 6: assigned [mem 0xc000b800-0xc000bfff pref]
[    0.547166] pci 0000:00:1e.0: BAR 6: assigned [mem 0xc000c000-0xc000c7ff pref]
[    0.548863] pci 0000:00:1f.0: BAR 6: assigned [mem 0xc000c800-0xc000cfff pref]
[    0.550612] NET: Registered protocol family 2
[    0.551789] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes)
[    0.553641] TCP established hash table entries: 16384 (order: 5, 131072 bytes)
[    0.555377] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
[    0.556964] TCP: Hash tables configured (established 16384 bind 16384)
[    0.558531] UDP hash table entries: 1024 (order: 3, 32768 bytes)
[    0.559955] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
[    0.561509] NET: Registered protocol family 1
[    0.562554] NET: Registered protocol family 44
[    0.563732] pci 0000:00:1f.0: Activating ISA DMA hang workarounds
[    0.565228] Unpacking initramfs...
[    1.144856] Freeing initrd memory: 25300K
[    1.146648] Initialise system trusted keyrings
[    1.147728] Key type blacklist registered
[    1.148783] workingset: timestamp_bits=40 max_order=19 bucket_order=0
[    1.152242] zbud: loaded
[    1.325751] Key type asymmetric registered
[    1.326787] Asymmetric key parser 'x509' registered
[    1.327957] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
[    1.329753] io scheduler noop registered
[    1.330712] io scheduler deadline registered
[    1.331764] io scheduler cfq registered (default)
[    1.332872] io scheduler mq-deadline registered
[    1.334154] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[    1.335898] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    1.360812] 00:03: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    1.386068] 00:04: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
[    1.388370] Linux agpgart interface v0.103
[    1.389389] AMD IOMMUv2 driver by Joerg Roedel <[email]jroedel@suse.de[/email]>
[    1.390810] AMD IOMMUv2 functionality not available on this system
[    1.392525] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f03:MOU] at 0x60,0x64 irq 1,12
[    1.394625] i8042: Warning: Keylock active
[    1.398174] serio: i8042 KBD port at 0x60,0x64 irq 1
[    1.399431] serio: i8042 AUX port at 0x60,0x64 irq 12
[    1.400794] mousedev: PS/2 mouse device common for all mice
[    1.403307] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0
[    1.405698] rtc_cmos 00:05: registered as rtc0
[    1.406816] rtc_cmos 00:05: alarms up to one day, y3k, 114 bytes nvram
[    1.408404] ledtrig-cpu: registered to indicate activity on CPUs
[    1.410136] NET: Registered protocol family 10
[    1.419321] Segment Routing with IPv6
[    1.420235] mip6: Mobile IPv6
[    1.420956] NET: Registered protocol family 17
[    1.422072] mpls_gso: MPLS GSO support
[    1.423097] mce: Using 0 MCE banks
[    1.423935] sched_clock: Marking stable (1448605245, -25524956)->(1438986829, -15906540)
[    1.426263] registered taskstats version 1
[    1.427263] Loading compiled-in X.509 certificates
[    1.467921] Loaded X.509 cert 'Debian Secure Boot CA: 6ccece7e4c6c0d1f6149f3dd27dfcc5cbb419ea1'
[    1.469985] Loaded X.509 cert 'Debian Secure Boot Signer 2020: 00b55eb3b9'
[    1.471663] zswap: loaded using pool lzo/zbud
[    1.472900] AppArmor: AppArmor sha1 policy hashing enabled
[    1.474618] rtc_cmos 00:05: setting system clock to 2021-02-04 15:30:52 UTC (1612452652)
[    1.478687] Freeing unused kernel image memory: 1596K
[    1.494763] Write protecting the kernel read-only data: 16384k
[    1.496729] Freeing unused kernel image memory: 2028K
[    1.498135] Freeing unused kernel image memory: 776K
[    1.508019] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    1.509548] x86/mm: Checking user space page tables
[    1.519218] x86/mm: Checked W+X mappings: passed, no W+X pages found.
[    1.520740] Run /init as init process
Loading, please wait...
Starting version 241
[    1.586102] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
[    1.591811] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
[    1.593775] virtio-pci 0000:00:06.0: virtio_pci: leaving for legacy driver
[    1.595629] virtio-pci 0000:00:09.0: can't derive routing for PCI INT A
[    1.597196] virtio-pci 0000:00:09.0: PCI INT A: no GSI
[    1.598557] virtio-pci 0000:00:09.0: virtio_pci: leaving for legacy driver
[    1.604139] cryptd: max_cpu_qlen set to 1000
[    1.605803] virtio-pci 0000:00:1e.0: can't derive routing for PCI INT A
[    1.607392] virtio-pci 0000:00:1e.0: PCI INT A: no GSI
[    1.608733] virtio-pci 0000:00:1e.0: virtio_pci: leaving for legacy driver
[    1.617048] AVX2 version of gcm_enc/dec engaged.
[    1.618154] AES CTR mode by8 optimization enabled
[    1.627859] virtio_blk virtio0: [vda] 41943039 512-byte logical blocks (21.5 GB/20.0 GiB)
[    1.637992] virtio_net virtio3 enp0s9: renamed from eth0
[    2.151015] tsc: Refined TSC clocksource calibration: 2098.460 MHz
[    2.152513] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x1e3f81de2df, max_idle_ns: 440795250122 ns
[    2.154942] clocksource: Switched to clocksource tsc
[    2.345984] input: PS/2 Generic Mouse as /devices/platform/i8042/serio1/input/input2
(and that's it - it keeps on hanging forever now)


For comparison, here's the relevant section of the booting 4.9 kernel:
Code: 
[    1.360776] Freeing unused kernel memory: 1428K
[    1.361896] Write protecting the kernel read-only data: 12288k
[    1.363812] Freeing unused kernel memory: 1884K
[    1.367605] Freeing unused kernel memory: 1220K
[    1.373654] x86/mm: Checked W+X mappings: passed, no W+X pages found.
Loading, please wait...
starting version 232
[    1.399936] random: systemd-udevd: uninitialized urandom read (16 bytes read)
[    1.401764] random: systemd-udevd: uninitialized urandom read (16 bytes read)
[    1.402982] random: udevadm: uninitialized urandom read (16 bytes read)
[    1.423009] virtio-pci 0000:00:03.0: virtio_pci: leaving for legacy driver
[    1.428178] virtio-pci 0000:00:05.0: virtio_pci: leaving for legacy driver
[    1.430311] virtio-pci 0000:00:06.0: virtio_pci: leaving for legacy driver
[    1.432168] virtio-pci 0000:00:09.0: can't derive routing for PCI INT A
[    1.433835] virtio-pci 0000:00:09.0: PCI INT A: no GSI
[    1.435238] virtio-pci 0000:00:09.0: virtio_pci: leaving for legacy driver
[    1.437400] virtio-pci 0000:00:1e.0: can't derive routing for PCI INT A
[    1.439093] virtio-pci 0000:00:1e.0: PCI INT A: no GSI
[    1.440507] virtio-pci 0000:00:1e.0: virtio_pci: leaving for legacy driver
[    1.446593] AVX2 version of gcm_enc/dec engaged.
[    1.447698] AES CTR mode by8 optimization enabled
[    1.487352] virtio_net virtio3 enp0s9: renamed from eth0
[    1.489447]  vda: vda1 vda2
[    1.495380]  vdc: vdc1
[    1.501517] random: fast init done
Begin: Loading essential drivers ... done.
Begin: Running /scripts/init-premount ... done.
Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done.
Begin: Running /scripts/local-premount ... done.
Begin: Will now check root file system ... fsck from util-linux 2.29.2
[/sbin/fsck.ext4 (1) -- /dev/vda2] fsck.ext4 -a -C0 /dev/vda2
/dev/vda2: clean, 56208/1277952 files, 720915/5111296 blocks
done.
[    1.612623] EXT4-fs (vda2): mounted filesystem with ordered data mode. Opts: (null)
done.
Begin: Running /scripts/local-bottom ... done.
Begin: Running /scripts/init-bottom ... done.
[    1.768421] systemd[1]: Inserted module 'autofs4'
[    1.775918] ip_tables: (C) 2000-2006 Netfilter Core Team
(and keeps on booting)
 
4.14 was the version which introduced Spectre/Meltdown mitigations.
The mitigations were introduced step-by-step in the sub versions 4.14.xx, 4.15 and onward that followed.

Some 4.14 versions had very serious issues that broke things, as the introduction of spectre/meltdown mitigations was done in an extreme hurry, sometimes versions following each other in a few hours.

No idea whether bhyve can deal with things like microcode updating and all that stuff that was introduced or made activated by default in Linux these days.
 
Are you saying that running a Linux kernel > 4.14 is impossible under bhyve?

Here's someone who has managed to installed Debian 10 (which uses kernel 4.19 by default):

SOLVED - Quick guide to install Debian 10 in bhyve with the serial console

Abstract To have a lighter Linux virtual machine, you can enable the serial console and then remove the VNC device after the operating system installation with VNC. But do you know that you can directly run the Debian 10 installer on the serial console without VNC ? It is faster because you will...
www.truenas.com www.truenas.com
 
Utumno said:
Are you saying that running a Linux kernel > 4.14 is impossible under bhyve?
Click to expand...
No, I just said what came into my mind, as there recently another guy had problems getting bhyve run post-Spectre/Meltdown Windows 10, which were solved by some command line options.

I think there might be some bhyve options that need to be set correctly so that these late OSes, which might fiddle with the CPU microcode and other things not seen before Spectre/Meltdown, can be run.
 
I'm running several Debian VMs with bhyve; None kernel update so far made a problem, any worked. The actual 4.19 from Stable works - but there's no Debian around anymore with a 5.6 kernel; But as Testing now uses 5.10 I can say: If there was a 5.6 in Testing, it worked.

So there isn't a general problem; It must be related to your hardware and/or installation.
 
Thanks Snurg & jmos.

So it's looking like this is some misconfiguration with my bhyve.
I've tried binary search; so far I've established that kernel 4.14.219 (the latest of the 4.14 line) boots with no problems, whereas 4.16.9 stops booting at exactly the same place already given in the log in the first post. Now I am trying 4.15.13.

I've tried passing special parameters to kernel 4.16.9 to switch off Spectre & Meltdown mitigations as described in

unix.stackexchange.com

Disable Spectre and Meltdown mitigations

Can I disable Spectre and Meltdown mitigation features in Ubuntu 18.04LTS? I want to test how much more performance I gain when I disable these two features in Linux, and if the performance is big...
unix.stackexchange.com unix.stackexchange.com

( nospectre_v1 nospectre_v2 spectre_v2_user=off spec_store_bypass_disable=off l1tf=off mds=off tsx_async_abort=off nx_huge_pages=off nopti )

unfortunately that doesn't seem to make any difference - 4.16.9 with those mitigations switched off still stops booting at exactly the same point.
 
Just set this up this weekend, did nothing special:
Code: 
dice@debian:~$ uname -a
Linux debian 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64 GNU/Linux
dice@debian:~$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
Seems to work just fine.
VM configururation:
Code: 
loader="uefi"
graphics="YES"
cpu="2"
memory="4096M"
network0_type="virtio-net"
network0_switch="public"
disk0_type="virtio-blk"
disk0_name="disk0"
disk0_dev="sparse-zvol"
grub_run_dir="/grub"
uuid="<uuid>"
network0_mac="<mac address>"
The exact commands from the logs:
Code: 
Feb 08 02:22:34: restarting
Feb 08 02:22:34:  [bhyve options: -c 2 -m 4096M -Hwl bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -U <uuid> -u]
Feb 08 02:22:34:  [bhyve devices: -s 0,hostbridge -s 31,lpc -s 4:0,virtio-blk,/dev/zvol/stor10k/DATA/vm/debian/disk0 -s 5:0,virtio-net,tap13,mac=<mac address> -s 6:0,fbuf,tcp=0.0.0.0:5900]
Feb 08 02:22:34:  [bhyve console: -l com1,/dev/nmdm-debian.1A]
Feb 08 02:22:34:  [bhyve iso device: -s 3:0,ahci-cd,/vm/.config/null.iso]
Feb 08 02:22:34: starting bhyve (run 2)
 
You must log in or register to reply here.
Back
Top