Installing FreeBSD 12.2 ppc64 in KVM on POWER9

bloudraak

bloudraak

I'm trying to install FreeBSD 12.0 ppc64 in KVM running on POWER9 (Talos II) with Ubuntu 20.04 as the host. Currently, it's failing with the following:

Code: 
Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...               
Kernel entry at 0x102390 ...
panic: Standard large pages (SLB[L] = 1, PTE[LP] = 0) not supported by this system. Please enable huge page backing if running under PowerKVM.
cpuid = 0
time = 1
KDB: stack backtrace:
#0 0xc000000000729964 at ??+0
#1 0xc0000000006c1254 at ??+0
#2 0xc0000000006c1320 at ??+0
#3 0xc000000000b1a764 at ??+0
#4 0xc000000000b0b2ac at ??+0
#5 0xc000000000b04a0c at ??+0
#6 0xc000000000102440 at ??+0
Uptime: 1s

I created the VM using the following command:

Code: 
virt-install \
    --name freebsd12-ppc64-01 \
    --memory 2048 \
    --virt-type=kvm \
    --disk pool=default,size=64,format=qcow2 \
    --vcpus=1 \
    --memorybacking hugepages=yes \
    --network network=br0 \
    --os-variant freebsd12.2 \
    --graphics none \
    --cdrom /var/lib/libvirt/boot/FreeBSD-12.2-RELEASE-powerpc-powerpc64-dvd1.iso \
    --debug \
    --force

And that spits out the following:

Code: 
[Tue, 07 Dec 2021 06:24:23 virt-install 44891] DEBUG (cli:208) Launched with command line: /usr/share/virt-manager/virt-install --name freebsd12-ppc64-01 --memory 2048 --virt-type=kvm --disk pool=default,size=64,format=qcow2 --vcpus=1 --memorybacking hugepages=yes --network network=br0 --os-variant freebsd12.2 --graphics none --cdrom /var/lib/libvirt/boot/FreeBSD-12.2-RELEASE-powerpc-powerpc64-dvd1.iso --debug --force
[Tue, 07 Dec 2021 06:24:23 virt-install 44891] DEBUG (virt-install:207) Distilled --network options: ['network=br0']
[Tue, 07 Dec 2021 06:24:23 virt-install 44891] DEBUG (virt-install:139) Distilled --disk options: ['pool=default,size=64,format=qcow2']
[Tue, 07 Dec 2021 06:24:23 virt-install 44891] DEBUG (cli:224) Requesting libvirt URI default
[Tue, 07 Dec 2021 06:24:23 virt-install 44891] DEBUG (cli:227) Received libvirt URI qemu:///system
[Tue, 07 Dec 2021 06:24:23 virt-install 44891] DEBUG (storage:139) Found default pool name=default target=/var/lib/libvirt/images
[Tue, 07 Dec 2021 06:24:23 virt-install 44891] DEBUG (storage:208) refreshing pool=default
[Tue, 07 Dec 2021 06:24:23 virt-install 44891] DEBUG (disk:224) Creating volume 'freebsd12-ppc64-01-4.qcow2' on pool 'default'
[Tue, 07 Dec 2021 06:24:23 virt-install 44891] DEBUG (disk:358) disk.set_vol_install: name=freebsd12-ppc64-01-4.qcow2 poolxml=
<pool type='dir'>
  <name>default</name>
  <uuid>fc40ce7b-fd1e-477d-807d-912394afc9dd</uuid>
  <capacity unit='bytes'>210304475136</capacity>
  <allocation unit='bytes'>116991582208</allocation>
  <available unit='bytes'>93312892928</available>
  <source>
  </source>
  <target>
    <path>/var/lib/libvirt/images</path>
    <permissions>
      <mode>0711</mode>
      <owner>0</owner>
      <group>0</group>
    </permissions>
  </target>
</pool>

[Tue, 07 Dec 2021 06:24:24 virt-install 44891] DEBUG (guest:463) Setting Guest osinfo name <_OsVariant name=generic>
[Tue, 07 Dec 2021 06:24:24 virt-install 44891] DEBUG (installer:398) installer.detect_distro returned=None
[Tue, 07 Dec 2021 06:24:24 virt-install 44891] DEBUG (guest:463) Setting Guest osinfo name <_OsVariant name=freebsd12.2>
[Tue, 07 Dec 2021 06:24:24 virt-install 44891] WARNING (virt-install:332) CDROM media does not print to the text console by default, so you likely will not see text install output. You might want to use --location. See the man page for examples of using --location with CDROM media
[Tue, 07 Dec 2021 06:24:24 virt-install 44891] DEBUG (virt-install:647) Guest.has_install_phase: True
[Tue, 07 Dec 2021 06:24:24 virt-install 44891] DEBUG (cli:272)
Starting install...

Starting install...
[Tue, 07 Dec 2021 06:24:24 virt-install 44891] DEBUG (storage:642) Creating storage volume 'freebsd12-ppc64-01-4.qcow2' with xml:
<volume>
  <name>freebsd12-ppc64-01-4.qcow2</name>
  <capacity>68719476736</capacity>
  <allocation>0</allocation>
  <target>
    <format type="qcow2"/>
    <features>
      <lazy_refcounts/>
    </features>
  </target>
</volume>

[Tue, 07 Dec 2021 06:24:24 virt-install 44891] DEBUG (storage:681) Using vol create flags=1
Allocating 'freebsd12-ppc64-01-4.qcow2'                                                                       |  64 GB  00:00:09     
[Tue, 07 Dec 2021 06:24:33 virt-install 44891] DEBUG (storage:687) Storage volume 'freebsd12-ppc64-01-4.qcow2' install complete.
[Tue, 07 Dec 2021 06:24:33 virt-install 44891] DEBUG (installer:441) Generated install XML:
<domain type="kvm">
  <name>freebsd12-ppc64-01</name>
  <uuid>48de4e44-cc3d-41f4-b6d7-68c54acfb12c</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://freebsd.org/freebsd/12.2"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory>2097152</memory>
  <currentMemory>2097152</currentMemory>
  <memoryBacking>
    <hugepages/>
  </memoryBacking>
  <vcpu>1</vcpu>
  <os>
    <type arch="ppc64le" machine="pseries">hvm</type>
    <boot dev="cdrom"/>
    <boot dev="hd"/>
  </os>
  <clock offset="utc"/>
  <on_reboot>destroy</on_reboot>
  <devices>
    <emulator>/usr/bin/qemu-system-ppc64le</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/var/lib/libvirt/images/freebsd12-ppc64-01-4.qcow2"/>
      <target dev="vda" bus="virtio"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/var/lib/libvirt/boot/FreeBSD-12.2-RELEASE-powerpc-powerpc64-dvd1.iso"/>
      <target dev="sda" bus="scsi"/>
      <readonly/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15"/>
    <controller type="scsi" index="0" model="virtio-scsi"/>
    <interface type="network">
      <source network="br0"/>
      <mac address="52:54:00:f1:4f:a0"/>
      <model type="virtio"/>
    </interface>
    <console type="pty"/>
    <channel type="unix">
      <source mode="bind"/>
      <target type="virtio" name="org.qemu.guest_agent.0"/>
    </channel>
  </devices>
</domain>

[Tue, 07 Dec 2021 06:24:33 virt-install 44891] DEBUG (installer:443) Generated boot XML:
<domain type="kvm">
  <name>freebsd12-ppc64-01</name>
  <uuid>48de4e44-cc3d-41f4-b6d7-68c54acfb12c</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://freebsd.org/freebsd/12.2"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory>2097152</memory>
  <currentMemory>2097152</currentMemory>
  <memoryBacking>
    <hugepages/>
  </memoryBacking>
  <vcpu>1</vcpu>
  <os>
    <type arch="ppc64le" machine="pseries">hvm</type>
    <boot dev="hd"/>
  </os>
  <clock offset="utc"/>
  <devices>
    <emulator>/usr/bin/qemu-system-ppc64le</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/var/lib/libvirt/images/freebsd12-ppc64-01-4.qcow2"/>
      <target dev="vda" bus="virtio"/>
    </disk>
    <disk type="file" device="cdrom">
      <target dev="sda" bus="scsi"/>
      <readonly/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15"/>
    <controller type="scsi" index="0" model="virtio-scsi"/>
    <interface type="network">
      <source network="br0"/>
      <mac address="52:54:00:f1:4f:a0"/>
      <model type="virtio"/>
    </interface>
    <console type="pty"/>
    <channel type="unix">
      <source mode="bind"/>
      <target type="virtio" name="org.qemu.guest_agent.0"/>
    </channel>
  </devices>
</domain>

[Tue, 07 Dec 2021 06:24:36 virt-install 44891] DEBUG (installer:496) XML fetched from libvirt object:
<domain type='kvm' id='20'>
  <name>freebsd12-ppc64-01</name>
  <uuid>48de4e44-cc3d-41f4-b6d7-68c54acfb12c</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://freebsd.org/freebsd/12.2"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit='KiB'>2097152</memory>
  <currentMemory unit='KiB'>2097152</currentMemory>
  <memoryBacking>
    <hugepages/>
  </memoryBacking>
  <vcpu placement='static'>1</vcpu>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='ppc64le' machine='pseries-focal'>hvm</type>
    <boot dev='cdrom'/>
    <boot dev='hd'/>
  </os>
  <cpu mode='custom' match='exact' check='none'>
    <model fallback='forbid'>POWER9</model>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>destroy</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-ppc64le</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/var/lib/libvirt/images/freebsd12-ppc64-01-4.qcow2' index='2'/>
      <backingStore/>
      <target dev='vda' bus='virtio'/>
      <alias name='virtio-disk0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/var/lib/libvirt/boot/FreeBSD-12.2-RELEASE-powerpc-powerpc64-dvd1.iso' index='1'/>
      <backingStore/>
      <target dev='sda' bus='scsi'/>
      <readonly/>
      <alias name='scsi0-0-0-0'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </controller>
    <controller type='scsi' index='0' model='virtio-scsi'>
      <alias name='scsi0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <model name='spapr-pci-host-bridge'/>
      <target index='0'/>
      <alias name='pci.0'/>
    </controller>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:f1:4f:a0'/>
      <source network='br0' portid='ab8126fa-bcec-450c-936e-e72dbc504ac5' bridge='br0'/>
      <target dev='vnet7'/>
      <model type='virtio'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/8'/>
      <target type='spapr-vio-serial' port='0'>
        <model name='spapr-vty'/>
      </target>
      <alias name='serial0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </serial>
    <console type='pty' tty='/dev/pts/8'>
      <source path='/dev/pts/8'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
      <address type='spapr-vio' reg='0x30000000'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-20-freebsd12-ppc64-01/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='disconnected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>

Hit [Enter] to boot immediately, or any other key for command prompt.
Booting [/boot/kernel/kernel]...               
Kernel entry at 0x102390 ...
panic: Standard large pages (SLB[L] = 1, PTE[LP] = 0) not supported by this system. Please enable huge page backing if running under PowerKVM.
cpuid = 0
time = 1
KDB: stack backtrace:
#0 0xc000000000729964 at ??+0
#1 0xc0000000006c1254 at ??+0
#2 0xc0000000006c1320 at ??+0
#3 0xc000000000b1a764 at ??+0
#4 0xc000000000b0b2ac at ??+0
#5 0xc000000000b04a0c at ??+0
#6 0xc000000000102440 at ??+0
Uptime: 1s

I followed the instructions at the Ubuntu site, and ran the following command:

Code: 
cat /proc/meminfo | grep Huge

It shows huge pages are enabled

Code: 
AnonHugePages:   2904064 kB
ShmemHugePages:        0 kB
FileHugePages:         0 kB
HugePages_Total:   16384
HugePages_Free:    15360
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:        33554432 kB

This is persistent, meaning I can reboot the host numerous times and the command produces similar output.

My motivation for installing FreeBSD 12.2 is to have a lab with most operating systems I can find so I can research, develop and test applications on several platforms. I am also running FreeBSD 13.
 
freebsd kernel reads available page sizes from openfirmware device tree @ibm,segment-page-sizes
if the key is present but it does not report 16MB pages it bombs
 
I'm a bit lost in configuring Ubuntu 20.04 on POWER9 to use a 16MB page size. The default huge page size remains 2M no matter what I do, even if I change the kernel parameters. I can't seem to register a pool with a 16M page size. The Debian docs say 16M is supported.

Am I missing something simple?
 
You must log in or register to reply here.
Back
Top