Sun 501-6522-07

I have three Sun 501-6522-07 cards in a Dell Poweredge 2850 and they are recognized and configurable inside FreeBSD 10.3 however no traffic is sent out or received by the NICs. I have checked /boot/kernel and if_cas.ko wasn't there. Am I correct to assume that file needs to be loaded into the kernel as the next step in troubleshooting or is there some other solution that I don't know about?
Well I suppose I should back up here... I am having the issues of the interfaces not working correctly in just about every version of FreeBSD other than FreeBSD 9.2. It would seem that earlier versions have trouble recognizing all 12 of the interfaces (usually only 3 but traffic can flow through them) and later versions recognize all 12 but no traffic is allowed in or out of them. Ultimately my goal is to be able to have all 12 interfaces work on 10.3 so that I could run Pfsense on top of that. Any ideas?
Did you try on pfSense LiveCD first and get it to work there? Are these the PCI-X cards that work in a PCI slot? Title is vague. I think i have 2 of these running in pfSense if the same.
Did you try on pfSense LiveCD first and get it to work there? Are these the PCI-X cards that work in a PCI slot? Title is vague. I think i have 2 of these running in pfSense if the same.

I have the latest Pfsense build installed on the box right now and my FreeBSD testing was done with a live USB drive on the same box. The only way I could get the cards to work was when I booted it on a FreeBSD 9.2 live USB. They are PCI-X cards in PCI-X Slots.
For the $23 or so on eBay, I'd ditch that Sun QGE card. The X4444A was a dog when it was new. That was one of Sun's first gigabit cards
pfSense forum

I think this was good advice. That said they do work. I have a pair running long time now.
Is your riser only PCI-X?

And you have pfSense currently working with recent version? Going to try bhyve?
Yeah the riser is just 3 PCI-X Slots. I do not mind going though the effort to get the current cards working. Since they work under 9.2, my thoughts were that full support was since removed in 10.3 or something of that nature. Could I possibly compile a custom kernel or is loading additional files an option? There honestly has to be some way to make them work.
I would try and do an FreeBSD -Current install as well. The Live Memstick mode might have the bios in a funky APIC state. I would do some FreeBSD disk installs.

Also I would try One and Two card setups to see if some other issue like interrupt handling..
So what you're saying is install the various versions of FreeBSD (9.2 and 10.3) completely on the box and verify that it still works on 9.2 and doesn't work on 10.3? Is there anyway to get the pfsense packages on a working installed copy of FreeBSD 9.2?
The only way that I could even find if_cas was to download the latest .ko file from the FreeBSD website and try to manually load it however it says that it is already mounted. After placing it in the /boot/kernel directory, adding your second statement to /boot/loader.conf, and running dmesg after boot, I get the same statement that if_cas is already loaded despite nothing actually working.
So would I do the same thing in that case? Grab both the latest cas and miibus .ko files from the FreeBSD website and then try to manually load them with kldload?
It sounds like cas is baked in, you are double loading it. No need to add anything to loader.conf.
If kldload if_cas came back with -module already loaded- then that was the end of that effort.

I would investigate MII bus as it is the glue for this device.
Infact if you are showing CAS interfaces in ifconfig then all the cas module loading was really useless.
After doing a full installation of straight FreeBSD 10.3 from a DVD, the interfaces were usable with no problems. I will be comparing the two's dmesg outputs soon. I don't remember whether I installed Pfsense from a USB or not but could that be the issue if the kernels are different between the two?
Dmesg outputs look identical between the Pfsense install and the FreeBSD 10.3 install with the only difference being that the interfaces actually work on FreeBSD 10.3.
Kernels from the FreeBSD installer are the same across all install media.
The difference being the architecture with AMD64 and x86 Generic kernels.

So in the end you got most of what you wanted. FreeBSD stable with all 3 cards working, Correct?
From here you want to run pfSense virtually on top of bhyve or what?

If your just trying to work around the issue checkout OPNsense and see if they have the same issue.