Solved Intel E810 - Enable VFs

T

tosaikiran

Hi,

I am Sai Kiran. I have Intel-E810, connected to my system via one of the PCI slots. In pciconf -lv output, I see ice0 and ice1. I would like to enable Virtual Functions on this Network Adapter. When I use "iovctl -C -f /etc/iovctl-ice0.conf", I get the error message saying, /dev/iov/ice0: No such file or directory. When I see in that folder, I notice that ice0 is actually missing, only ixl0, ixl1 are present. I have installed both ICE and IAVF drivers. I am not sure what the issue is. Could someone help me out here.

Thanks in advance for your help and support.

Regards,
Sai Kiran.
 
tosaikiran said:
/dev/iov/ice0: No such file or directory.
Click to expand...
Create /etc/iov/ and place your VF conf files in there.
"iovctl -C -f /etc/iov/ice0.conf"
Reboot and check /dev/iov/

forums.freebsd.org

Enabling SR-IOV on intel driver

This is a new server I just installed. It has around 10 bhyve vm:s on it right now. I use latest Intel ix driver from intel-ix-kmod-3.3.6_1 package... As the network performance went down on the VMs when leaving KVM om Linux due to not optimal bridge code in FreeBSD I want to use SR-IOV on the...
forums.freebsd.org forums.freebsd.org
 
Hi,

I have tried that. After reboot, there's no /dev/iov/ice0 folder. The same error "/dev/iov/ice0.conf: No such file or directory" exists. Also, during kernel booting, I am able to see messages related to ICE. One of the line says "ICE driver already present".
 
SirDice said:
It's /etc/iov/ice0.conf, not /dev/iov/ice0.conf.
Click to expand...
Hi,

Thanks for your reply.

Yes, the ice0.conf file is present in /etc/iov folder itself. Here is what I am doing.

$ touch /etc/iov/ice0.conf
$ iovctl -C -f /etc/iov/ice0.conf
iovctl: Could not open device '/dev/iov/ice0' : No such file or directory.

$ pciconf -lvc
In the output of the above command, I see that, under ice0, SR-IOV is disabled. I have enabled it in BIOS. I am attaching pciconf output, related to ice0 and ice1.
 

Attachments

  • pciconf.txt.txt
    3.8 KB · Views: 115
I was getting ready to tell you maybe ice does not have SR-IOV support but your message shows it is supported.
ecap 0010[160] = SR-IOV 1 IOV disabled, Memory Space disabled, ARI disabled
0 VFs configured out of 128 supported.
Click to expand...

You must get your config files right. Start with a single interface only. Then graduate to both.
Might only have all 128 VF's on ice0 interface.
 
Last edited:
On Chelsio I could only create VF's on first interface. It has to do with t5nex0 is the nexus driver for both interfaces.
Maybe Intel will act the same. So start with first interface and try expand once working.
 
Phishfry said:
You have to tell SRIOV about your config files.
/etc/rc.conf
iovctl_files="/etc/iov/ice0.conf

What good is that? You create a config file and add nothing to it? You must define your VF/PF's.
Click to expand...
I should have posted the contents of ice0.conf. Here are the contents of ice0.conf.
Code: 
PF {
           device : "ice0"
           num_vfs : 4
}
My /etc/rc.conf contains the line iovctl_files="/etc/iov/ice0.conf". When I reboot the system, I see the following boot messages.
Code: 
"ICE driver already present"
"IAVF driver already present"
But when I see the loaded kernel drivers, using kldstat(8) command, I only see ice_ddp.ko and no .ko file related to iavf driver. I do not see any messages related to IAVF driver in dmesg. After rebooting and giving the command iovctl -C -f /etc/iov/ice0.conf, I still get the error iovctl: Could not open the device 'dev/iov/ice0': No such file or directory

I have downloaded the ICE and IAVF driver source code from Intel Website and built it on FreeBSD. Here are the steps I followed to build the drivers.
Code: 
$ cd ice-0.29.4
$ make
$ kldload ./if_ice.ko
For kldload, I get the following message.
Code: 
interface ice.1 already present in the KLD 'kernel'!
linker_load_file: ./if_ice.ko - unsupported file type
To load the ICE driver at boot time, I have placed the lines if_ice_load="YES" & ice_ddp_load="YES" in /boot/loader.conf folder.

Similarly, for the IAVF driver, I have followed these steps.
Code: 
$ cd iavf-3.0.26
$ make
$ kldload ./if_iavf.ko
For kldload, I get the similar message.
Code: 
interface iavf.1 already present in the KLD 'kernel'!
linker_load_file: ./if_iavf.ko - unsupported file type
I have placed the following line in /boot/loader.conf: if_iavf_load="YES"

I am not sure what exactly is needed to fix this issue. Could you please help me out here.

Thanks.
 
Last edited by a moderator:
There shouldn't be external driver built and installed needed.

'ice' and 'iavf' are build in GENERIC kernel. If you execute (verbose) kldstat -v | egrep 'pci/ice|pci/iavf' it will return
Code: 
        490 pci/iavf
        488 pci/ice

tosaikiran said:
$ kldload ./if_ice.ko
For kldload, I get the following message.

interface ice.1 already present in the KLD 'kernel'!
linker_load_file: ./if_ice.ko - unsupported file type
Click to expand...
This message is due the fact the build-in driver inhibits loading the external driver.

To the problem, maybe the adapter is in "Firmware Recovery Mode", therefore doesn't allow configuration. From ice driver README:
Code: 
Important Notes
===============

Firmware Recovery Mode
----------------------
A device will enter Firmware Recovery mode if it detects a problem that
requires the firmware to be reprogrammed. When a device is in Firmware Recovery
mode it will not pass traffic or allow any configuration; you can only attempt
to recover the device's firmware. Refer to the Intel(R) Ethernet Adapters and
Devices User Guide for details on Firmware Recovery Mode and how to recover
from it.

If the "Firmware Recovery Mode" isn't the case: I haven't checked the version of both drivers from base, maybe the external ones are newer and work as expected. If you want to try those external drivers you need to build a custom kernel, excluding the build-in ones from /usr/src/sys/amd64/conf/GENERIC (assuming the systems architecture is amd64):
Code: 
# PCI/PCI-X/PCIe Ethernet NICs that use iflib infrastructure
...
#device          iavf                    # Intel Adaptive Virtual Function
#device          ice                     # Intel 800 Series Physical Function
 
After reading the man page for iavf(4) I do wonder if you are doing it right.
NOTE: This iavf driver is only for Virtual Functions. For 700 series
Physical Functions, use the ixl(4) driver.
Click to expand...

So I wonder if you should be using iavf driver only and not ice+iavf.
That would also change your config file.
 
T-Daemon said:
Code: 
# PCI/PCI-X/PCIe Ethernet NICs that use iflib infrastructure
...
#device          iavf                    # Intel Adaptive Virtual Function
#device          ice                     # Intel 800 Series Physical Function
Click to expand...

Thanks for your inputs. I tried building the Kernel without iavf and ice modules. Then I have built the ice and iavf drivers from source and loaded them using kldload. When I try to create VFs using iovctl command, I get the following message.
Code: 
$ iovctl -C -f /etc/iov/ice0.conf
pci10: <unknown> at device 0.8 (no driver attached)
pci10: <unknown> at device 0.9 (no driver attached)
I am able to see the VFs, using pciconf command.

Here is the output of pciconf :
Code: 
$pciconf -lvvc pci0:137:0:8
ppt0@pci0:137:0:8: class=0xffffff rev=0xff hdr=0x7f vendor=0x8086 device=0x1889 subvendor=0x0000 subdevice=0x0000
    vendor   = 'Intel Corporation'
    device   = 'Ethernet Adaptive Virtual Function'
pciconf: list_caps: bad header type

Here are the contents of ice0.conf
Code: 
PF {
         device : "ice0"
         num_vfs: 2;
}
DEFAULT {
        passthrough : true;
}
Phishfry said:
After reading the man page for iavf(4) I do wonder if you are doing it right.


So I wonder if you should be using iavf driver only and not ice+iavf.
That would also change you config file.
Click to expand...

I have checked with the Intel Support team. They have mentioned that we need both ICE & IAVF drivers, to be able to create VFs in E810 Ethernet Controller.

However, I am not sure why there is a bad header type in the VF created.
 
The driver authors are on the manpage along with their email address. Have you tried asking them?
Most FreeBSD developers answer emails.
 
Phishfry said:
The driver authors are on the manpage along with their email address. Have you tried asking them?
Most FreeBSD developers answer emails.
Click to expand...
Could you provide the link to the man page. I am not able to find email address of authors. In the downloaded ICE driver, there is an option to install man pages. In the man page, I see an email address(freebsd@intel.com). Is this the one you are talking about?
 
tosaikiran said:
I tried building the Kernel without iavf and ice modules. Then I have built the ice and iavf drivers from source and loaded them using kldload. When I try to create VFs using iovctl command, I get the following message.
Code: 
$ iovctl -C -f /etc/iov/ice0.conf
pci10: <unknown> at device 0.8 (no driver attached)
pci10: <unknown> at device 0.9 (no driver attached)
Click to expand...
This does make me wonder if you need more...

tosaikiran said:
I am able to see the VFs, using pciconf command.
Click to expand...
Does this mean you can see the two VF's from your config file with the Intel Drivers?
 
You must log in or register to reply here.
Back
Top