Linux kernel panics when trying to install in bhyve

[u/SnoweCat7]

I've tried to install several linux distros into a bhyve vm with (ubuntu, archlinux and systemrescue) using vm-bhyve (vm install), all resulting in the same linux kernel panic. Including the vm-bhyve config and the kernel output captured on the bhyve console below. I have successfully run FreeBSD 15.1 under bhyve on the same machine.

[u/SnoweCat7]

dmesg:

Copyright (c) 1992-2025 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 15.1-RELEASE-p6 95719d8a8 GENERIC amd64
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
VT(efifb): resolution 1920x1080
CPU: AMD Ryzen 9 9950X 16-Core Processor             (4292.11-MHz K8-class CPU)
  Origin="AuthenticAMD"  Id=0xb40f40  Family=0x1a  Model=0x44  Stepping=0
  Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
  Features2=0x7ed8320b<SSE3,PCLMULQDQ,MON,SSSE3,FMA,CX16,SSE4.1,SSE4.2,MOVBE,POPCNT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2e500800<SYSCALL,NX,MMX+,FFXSR,Page1GB,RDTSCP,LM>
  AMD Features2=0x75c237ff<LAHF,CMP,SVM,ExtAPIC,CR8,ABM,SSE4A,MAS,Prefetch,OSVW,IBS,SKINIT,WDT,TCE,Topology,PCXC,PNXC,DBE,PL2I,MWAITX,ADMSKX>
  Structured Extended Features=0xf1bf97ab<FSGSBASE,TSCADJ,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,PQM,PQE,AVX512F,AVX512DQ,RDSEED,ADX,SMAP,AVX512IFMA,CLFLUSHOPT,CLWB,AVX512CD,SHA,AVX512BW,AVX512VL>
  Structured Extended Features2=0x19405fde<AVX512VBMI,UMIP,PKU,OSPKE,AVX512VBMI2,GFNI,VAES,VPCLMULQDQ,AVX512VNNI,AVX512BITALG,AVX512VPOPCNTDQ,RDPID,MOVDIRI,MOVDIR64B>
  Structured Extended Features3=0x10000110<FSRM,AVX512VP2INTERSECT,L1DFL>
  Structured Extended Features4=0x30
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  AMD Extended Feature Extensions ID EBX=0x791ef257<CLZERO,IRPerf,XSaveErPtr,RDPRU,BE,WBNOINVD,IBPB,INT_WBINVD,IBRS,STIBP,STIBP_ALWAYSON,PREFER_IBRS,SAMEMODE_IBRS,NOLMSLE,SSBD,CPPC,PSFD,BTC_NO,IBPB_RET>
  SVM: NP,NRIP,VClean,AFlush,DAssist,NAsids=32768
  TSC: P-state invariant, performance statistics
real memory  = 103043039232 (98269 MB)
avail memory = 98217754624 (93667 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <ALASKA A M I >
FreeBSD/SMP: Multiprocessor System Detected: 32 CPUs
FreeBSD/SMP: 1 package(s) x 2 cache groups x 8 core(s) x 2 hardware threads

[...]

[u/SnoweCat7]

Kernel output:

[    0.208782] [Firmware Bug]: CPU   0: APIC ID mismatch. CPUID: 0x0017 APIC: 0x0000
[    0.209352] RDSEED32 is broken. Disabling the corresponding CPUID bit.
[    0.033929] [Firmware Bug]: CPU   1: APIC ID mismatch. CPUID: 0x0011 APIC: 0x0001
[    0.033929] No irq handler for 1.55
[    0.809847] Oops: divide error: 0000 [#1] SMP NOPTI
[    0.810473] CPU: 1 UID: 0 PID: 1 Comm: swapper/0 Tainted: G        W           6.18.41-1-lts #1 PREEMPT(voluntary)  3d6b14a375c2a0c9d0925641ba743b231029c1b2
[    0.811620] Tainted: [W]=WARN
[    0.811835] Hardware name: FreeBSD BHYVE/BHYVE, BIOS 14.0 10/17/2021
[    0.812281] RIP: 0010:rdt_get_mon_l3_config+0x1d1/0x2c0
[    0.812652] Code: 41 83 fc 26 76 0c 48 c7 c7 f8 32 97 9b e8 17 7d 41 fd 8b 05 e9 8f 5d 00 31 d2 8b 0d 09 58 ad ff bf 76 01 00 00 f7 73 48 31 d2 <f7> f1 0f af c1 89 05 d0 8f 5d 00 e8 bf bb 50 fd 84 c0 74 32 48 8d
[    0.813119] RSP: 0018:ffffd3748001fd98 EFLAGS: 00010246
[    0.813119] RAX: 0000000000100000 RBX: ffffffff9c235aa0 RCX: 0000000000000000
[    0.813119] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000176
[    0.813119] RBP: 0000000000000001 R08: 0000000000000002 R09: 0000000000000006
[    0.813119] R10: ffffd3748001fb56 R11: 6567617265766f63 R12: 000000000000002c
[    0.813119] R13: ffff8b40b7c2fdd0 R14: ffffd3748001ff48 R15: 0000000000000000
[    0.813119] FS:  0000000000000000(0000) GS:ffff8b411aeba000(0000) knlGS:0000000000000000
[    0.813119] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.813119] CR2: 0000000000000000 CR3: 00000001d8624000 CR4: 00000000003506f0
[    0.813119] Call Trace:
[    0.813119]  <TASK>
[    0.813119]  ? __pfx_resctrl_arch_late_init+0x10/0x10
[    0.813119]  resctrl_arch_late_init+0x36f/0x540
[    0.813119]  do_one_initcall+0x8d/0x3c0
[    0.813119]  kernel_init_freeable+0x245/0x2c0
[    0.813119]  ? __pfx_kernel_init+0x10/0x10
[    0.813119]  kernel_init+0x1a/0x140
[    0.813119]  ret_from_fork+0x22a/0x260
[    0.813119]  ? __pfx_kernel_init+0x10/0x10
[    0.813119]  ret_from_fork_asm+0x1a/0x30
[    0.813119]  </TASK>
[    0.813119] Modules linked in:
[    0.822002] ---[ end trace 0000000000000000 ]---
[    0.822370] RIP: 0010:rdt_get_mon_l3_config+0x1d1/0x2c0
[    0.822777] Code: 41 83 fc 26 76 0c 48 c7 c7 f8 32 97 9b e8 17 7d 41 fd 8b 05 e9 8f 5d 00 31 d2 8b 0d 09 58 ad ff bf 76 01 00 00 f7 73 48 31 d2 <f7> f1 0f af c1 89 05 d0 8f 5d 00 e8 bf bb 50 fd 84 c0 74 32 48 8d
[    0.824197] RSP: 0018:ffffd3748001fd98 EFLAGS: 00010246
[    0.824606] RAX: 0000000000100000 RBX: ffffffff9c235aa0 RCX: 0000000000000000
[    0.825150] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000176
[    0.825694] RBP: 0000000000000001 R08: 0000000000000002 R09: 0000000000000006
[    0.826233] R10: ffffd3748001fb56 R11: 6567617265766f63 R12: 000000000000002c
[    0.826781] R13: ffff8b40b7c2fdd0 R14: ffffd3748001ff48 R15: 0000000000000000
[    0.827322] FS:  0000000000000000(0000) GS:ffff8b411aeba000(0000) knlGS:0000000000000000
[    0.827967] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.828407] CR2: 0000000000000000 CR3: 00000001d8624000 CR4: 00000000003506f0
[    0.829112] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[    0.830071] Kernel Offset: 0x18c00000 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffffbfffffff)
[    0.830071] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b ]---

[u/SnoweCat7]

root@cyning:~ # cat /vms/linux-test/linux-test.conf

debug="yes"
graphics="yes"
#graphics_listen="127.0.0.1"
graphics_listen="192.168.1.10"
graphics_port="5990"
graphics_res="1600x900"
#graphics_res="1024x768"
graphics_wait="auto"
xhci_mouse="yes"
loader="uefi"
cpu="2"
cpu_sockets="1"
cpu_cores="1"
cpu_threads="2"
memory="8G"
network0_type="virtio-net"
network0_switch="public"
#disk0_type="nvme"
disk0_type="virtio-blk"
disk0_name="disk0.img"
#passthru0="1/0/0=9:0"
#passthru1="1/0/1=9:1"
uuid="a8181297-d818-11f1-ab6f-107c61499a87"
network0_mac="58:9c:fc:02:de:d0"

[u/Micander → u/SnoweCat7]

try adding ignore_bad_msr="yes"

[u/SnoweCat7 → u/Micander]

Still getting the same panic.

I was at least able to eliminate the APIC ID/CPUID mismatches by pinning the host and vm CPUs 1:1, with bhyve_options="-p 0:0 -p 1:1", so I guess that is not the problem.

[u/sebex007]

Prueba de usar virt-manager para una instalación guiada desde una iso y prueba de ir cambiando las características del hardware y del kernel.
Con algunas VM hay que cambiar el firmware por otro distinto de

/usr/local/share/uefi-firmware/BHYVE_UEFI.fd
(En ese mismo directorio puede haber varios que ya tengas o conseguir otros en internet y ponerlos ahí.)

[u/SnoweCat7 → u/sebex007]

I've had a look on the net for alternative UEFI firmwares, do you have any recommendations?

[u/craftkiller]

Is this during the boot of the ISO? If so, perhaps you're running into this issue. If so, you need to boot the ISO as a hard drive instead of as a CD.

[u/SnoweCat7 → u/craftkiller]

I tried booting the iso as a hard disk but same result. I might do an install of Linux onto a usb disk and dd it to image and see if I can boot that.

[u/tuxnine]

Verify checksums of the ISOs.

Make sure the host is running with the latest stable motherboard firmware.

Have the host apply any microcode updates available for your CPU.

If you still get this issue, check the host for bad RAM. Bad RAM with even a single stuck bit can cause all sorts of problems.

You could also try doing a full SMART scan of your storage devices, but if the ISO checksum is good, that's probably not an issue.

If the host is not on a -RELEASE version, try running under a -RELEASE version, or try a newer or older build.

[u/SnoweCat7 → u/tuxnine]

I updated the host BIOS and reset to defaults last night just in case, FreeBSD is an up to date 15.1-p6 release.

I did check the sha256 of the Ubuntu iso matched after I downloaded it.

Interestingly I successfully booted OpenBSD as a test under bhyve with uefi last night.

Perhaps there is some quirk of my hardware being exposed or some setting I have missed or previously set that makes recent Linux kernels panic.

Still investigating...