Using a 2013 Mac Pro as a freeBSD Desktop

marten

marten

Introduction​

Previous guides to installing freeBSD on a 2013 Mac Pro (Mac Pro 6,1 a.k.a the Trashcan) did not cover Xorg or sound configuration. In this guide, we will set up a GPU-accelerated desktop with sound using freeBSD 14.2 RELEASE. My 2013 Mac Pro has the dual AMD D700 video cards and the 8-core, 16-thread Intel Xeon E5-1680 v2, but it should work for other configurations. For sake of simplicity I will use the entire disk and will not cover dual or triple booting. I will cover the (I) initial setup, (II) getting Xorg to work and (III) getting sound to work.

I. Initial setup​

When booting the installer, press 3 in the boot menu and enter SET hw.pci.enable_pcie_hp="0", then type menu to return to the boot menu. This is necessary to prevent a looping error message from showing up on the command line. We can now boot the installer and follow the usual steps of installation. However, after finishing the installer choose "Yes" for final modifications. We can now enter the following lines in /boot/loader.conf

Code: 
hw.pci.enable_pcie_hp="0"
coretemp_load="YES"
asmc_load="YES"
vmm_load="YES"
pptdevs="2/0/0"

hw.pci.enable_pcie_hp prevents a looping error message at boot. coretemp_load and asmc_load allow us to read CPU and sensor data. The last two entries hide one of the D700 video cards from the system. The 2013 Mac Pro has two D700 cards, card A at pci 2:0:0 and card B at pci 6:0:0. Only card B can output video, and if you don't hide card A, the video drivers will try to use card A because it has the lower PCI bus address. An added benefit of this setup is that card A is now available for GPU-passtrough in Bhyve.

II. Setting up Xorg​

First we will install the graphics drivers. For my current setup, graphics/drm-61-kmod, the package that comes with the graphics/drm-kmod metapackage, crashes the system at boot. Perhaps using ports may yield better results, but as a quick workaround I swiched to the package graphics/drm-515-kmod while leaving the other graphics/drm-kmod packages untouched.

Code: 
# pkg install drm-kmod
# pkg del -f drm-61-kmod
# pkg install drm-515-kmod
# sysrc kld_list+=amdgpu

Now we can install Xorg. The x11-drivers/xf86-video-amdgpu driver for Xorg is not automatically installed with the metapackage so we need to select it manually.

Code: 
# pkg install xorg xf86-video-amdgpu

After a reboot to load the amdgpu module, Xorg will fail when we try using startx. To fix this, we can create a configuration file, copy it to the default X11 configuration location, and rename it "xorg.conf".

Code: 
# Xorg -configure
# cp /root/xorg.conf.new /usr/local/etc/X11/xorg.conf

After this step, Xorg usually starts without problems. If it fails with an error referring to not being able to set framebuffer mode, ensure that in xorg.conf all cards defined in the "Device" sections use the amdgpu driver, like shown below:

Code: 
Section "Device"
        Identifier  "Card2"
        Driver      "amdgpu"
        BusID       "PCI:6:0:0"
EndSection

Now we can Xorg with startx, and install a desktop environment according to the freeBSD handbook.

III. Sound​

The Mac Pro uses a Cirrus Logic chip which is automatically detected and no drivers need to be explicitly loaded. However, none of the output sockets work out of the box because the jack-detection logic is inverted. To get sound output, we have to connect our speakers to the headphone jack instead of the line-out jack. Then, we add a device hint to ensure the sound driver uses the inverted logic by adding hint.hdaa.0.config="senseinv" at the bottom of /boot/device.hints. As a final touch, we set the headphone jack as the default sound unit by adding hw.snd.default_unit=1 to /etc/sysctl.conf

Final thoughts​

With this we get a GPU-accelerated desktop with sound. Seeing that we have a CPU with many cores and threads, and an extra GPU ready for use in Bhyve, the 2013 Mac Pro has potential for use with virtual machines.

Sources​

Instructions for the initial setup and the hw.pci.enable_pcie_hp="0" fix. Also covers multi-booting.
https://anschwa.com/blog/2022/01/03/installing-freebsd-on-2013-macpro.html

An explanation why the sound doesn't work out of the box and a Linux solution to the inverted-jack-detection logic:
https://www.tumblr.com/lewisinthela...44672/ubuntu-2004-on-2013-trashcan-mac-pro-61
 
Update: I found a better way of fixing the sound. Instead of using a device hint for the inverted-jack-detection logic, we can fix the sound altogether by properly setting up the gpio pins:
In /etc/sysctl.conf set
Code: 
dev.hdaa.0.gpio_config="0=set 1=set"
hw.snd.default_unit=2
and the sound output is fixed. With this solution we can connect our speakers normally to the line out jack.
 
Thanks for this, I've now got my 12-core Trash-can up-and-running. It has dual D300s. I was wondering how your bhyve GPU passthrough setup works, as my FreeBSD host uses the HDMI port on the can itself, do you use the Thunderbolt 2/mini-DVI ports for a second screen for the guest, or does FreeBSD not support displays over the Thunderbolt 2 ports at all? I haven't tried it yet as I only have the one screen for now, but I'm very curious.
 
I had this set-up, and was running a Hyprland (Wayland) GUI successfully, but what I didn't realise was that
Code: 
vmm_load="YES"
is ESSENTIAL for the firmware to be able to output video correctly. I had changed the file to not use vmm and to load Virtualbox instead, what this meant was that both GPUs were once again available to the host, and it was picking the wrong one, so hardware-acceleration wouldn't work but you'd still get a very primitive low-resolution framebuffer, and the only message being something that implies a screen is not connected to the graphics card. The OPs post does say this, but in my naivity I didn't think it was vmm ring-fencing the other GPU that allowed video to work correctly, so I didn't put two and two together.
 
taiwan740 said:
Thanks for this, I've now got my 12-core Trash-can up-and-running. It has dual D300s. I was wondering how your bhyve GPU passthrough setup works, as my FreeBSD host uses the HDMI port on the can itself, do you use the Thunderbolt 2/mini-DVI ports for a second screen for the guest, or does FreeBSD not support displays over the Thunderbolt 2 ports at all? I haven't tried it yet as I only have the one screen for now, but I'm very curious.
Click to expand...
As card A cannot output video, there's no option to use a second screen. I did experiment with getting GPU acceleration working for card A in bhybe but in the end I did not manage to get it to work, probably due to the special nature of the card, being an apple card.
 
I have a Mac Pro 2013, which I currently use for vm's and as a poudriere builder machine.
I haven't played around with gpu passthrough though, do you mind sharing your Bhyve configuration for this?
 
it-people To be clear, I never got the GPU passtrough fully working, and I'm fairly sure this is because it's Apple hardware and things are a bit different than in a normal PC.

You will need to use a program called Darwindumper under Mac OS to obtain ROM files for the video cards. I've attached the ROMs for the D700 cards below, which are the only ones I have. I've tried using both the A and the B rom file, but I found the best results with the B file.

I used vm-bhyve, with the following configuration for a Linux guest:
Code: 
loader="uefi"
cpu=4
memory=8G
network0_type="virtio-net"
network0_switch="public"
network0_device="tap0"
disk0_type="nvme"
disk0_name="disk0.img"
disk0_size="80G"
graphics="yes"
graphics_res="1280x720"
passthru0="13/0/0=6:0"
passthru1="2/0/1=2:1"
xhci_mouse="yes"
sound="yes"
sound_play="/dev/dsp2"
bhyve_options="-s 2:0,passthru,2/0/0,rom=/vm/.rom/D700B.rom"
wired_memory="yes"
 
You must log in or register to reply here.
Back
Top