Solved "exec format error" running certain docker image inside bhyve Linux guest, while everything is amd64

momocraft

momocraft

The error is from a Debian 12.11 guest running in FreeBSD host with vm-bhyve.

The whole configuration is built recently. Everything seemed to work until I tried to start a docker container with certain image:

Code: 
% docker run -it --rm mysql/mysql-server:latest bash
[Entrypoint] MySQL Docker Image 8.0.32-1.2.11-server
bash-4.4# exit

% docker run -it --rm mysql/mysql-server:8.0.31 bash
exec /entrypoint.sh: exec format error

While not fully sure this is FreeBSD related, I'm posting here because FB/bhyve could be a variable. The image runs in other configurations (like another Linux host of identical Debian version).

I'm aware that one common cause of such error is mismatch between binary and architecture. But I don't think this is my case because all my host / host OS / guest OS / image are amd64.

Any idea on possible causes, or what should I look into next?

The details I inspected by far:

Code: 
# Host uname
FreeBSD zecora 14.2-RELEASE FreeBSD 14.2-RELEASE releng/14.2-n269506-c8918d6c7412 GENERIC amd64

# Host 
sysctl hw.model hw.machine hw.ncpu hw.machine_arch
hw.model: Intel(R) Core(TM) i3-8100T CPU @ 3.10GHz
hw.machine: amd64
hw.ncpu: 4
hw.machine_arch: amd64

# Host dmesg
CPU: Intel(R) Core(TM) i3-8100T CPU @ 3.10GHz (3100.00-MHz K8-class CPU)
  Origin="GenuineIntel"  Id=0x906eb  Family=0x6  Model=0x9e  Stepping=11
  Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE>
  Features2=0x7ffafbbf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,SDBG,FMA,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,MOVBE,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX,F16C,RDRAND>
  AMD Features=0x2c100800<SYSCALL,NX,Page1GB,RDTSCP,LM>
  AMD Features2=0x121<LAHF,ABM,Prefetch>
  Structured Extended Features=0x29c67af<FSGSBASE,TSCADJ,SGX,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,NFPUSG,MPX,RDSEED,ADX,SMAP,CLFLUSHOPT,PROCTRACE>
  Structured Extended Features3=0x9c002600<MCUOPT,MD_CLEAR,TSXFA,IBPB,STIBP,L1DFL,SSBD>
  XSAVE Features=0xf<XSAVEOPT,XSAVEC,XINUSE,XSAVES>
  VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID
  TSC: P-state invariant, performance statistics

# Host packages
bhyve-firmware-1.0_2           Collection of Firmware for bhyve
edk2-bhyve-g202308_5           EDK2 Firmware for bhyve
grub2-bhyve-0.40_11            Grub-emu loader for bhyve
vm-bhyve-1.6.2                 Management system for bhyve virtual machines

# Host vm-bhyve conf
loader="grub"
cpu=3
memory=12G
network0_type="virtio-net"
network0_switch="public"
network0_mac="58:9c:fc:09:3f:ed"

uuid="feaa9679-43a5-11f0-9ebf-589cfc10fff2"

disk0_type="virtio-blk"
disk0_name="/dev/zvol/workfast/vm-bhyve/zecora-deb12/disk0"
disk0_dev="custom"

grub_run_partition="1"
grub_run_dir="/boot/grub"

disk1_type="ahci-cd"
disk1_name="seed.iso"
disk1_dev="file"

disk2_type="virtio-blk"
disk2_name="/dev/zvol/workfast/vm-bhyve/zecora-deb12/disk1"
disk2_dev="custom"

# Guest uname -a
Linux zecora-deb12 6.1.0-37-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.140-1 (2025-05-22) x86_64 GNU/Linux

# Guest cpuinfo
% cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 158
model name      : Intel(R) Core(TM) i3-8100T CPU @ 3.10GHz
stepping        : 11
cpu MHz         : 3100.000
cache size      : 6144 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 22
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht pbe syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid pni pclmulqdq dtes64 ds_cpl ssse3 sdbg fma cx16 xtpr pcid sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase bmi1 avx2 smep bmi2 erms invpcid rdseed smap xsaveopt arat md_clear
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit srbds mmio_stale_data retbleed gds bhi its
bogomips        : 6200.00
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

# Guest docker info | grep -i arch
Architecture: x86_64

# The images in guest
% docker inspect mysql/mysql-server:latest |grep -i arch
"Architecture": "amd64",

% docker inspect mysql/mysql-server:8.0.31 |grep -i arch
"Architecture": "amd64
 
momocraft said:
Code: 
# The images in guest
% docker inspect mysql/mysql-server:latest |grep -i arch
"Architecture": "amd64",

% docker inspect mysql/mysql-server:8.0.31 |grep -i arch
"Architecture": "amd64
Click to expand...
Maybe a copy/paste issue here on the forums, but the 8.0.31 architecture appears to be missing a closing quote.
 
I think I found the cause. Nothing I suspected was wrong. My copy of the image was somehow corrupted 🤦 .

Removing the image and downloading again fixed it.

I realized this when trying to extract the binaries to run outside a container:

Code: 
% docker save mysql/mysql-server:8.0.31 > image.tar
Error response from daemon: file integrity checksum failed for "etc/GREP_COLORS"
 
You must log in or register to reply here.
Back
Top