How do I install a commercial driver?

mariourk

mariourk

I have a new server, with this card.

I downloaded the FreeBSD-driver and copied the amd64-version of mpslsi.ko, meant for FreeBSD 9.0, to /boot/kernel/ and added mpslsi_load="YES" to /boot/loader.conf. However, the FreeBSD release it's actually running on, is 9.1.

It seems to work, but I'm not at all confident. What is the proper way to do this?
 
There is also a mps driver available in the base. This seems to be the same driver? They have the same source files, and doing a quick diff there do seem to be differences, but it seems that the version in FreeBSD base is newer / more up to date.

In any case, source is provided in the download. So you can just compile your own version if you wish to use it.

(As a bonus hint, you can just use kldload(8) on any file, ie. # kldload mymodule.ko).
 
So you can just compile your own version if you wish to use it.
Click to expand...
If it were Linux, I'd know how to do that. But I'm faily new to FreeBSD and I have no idea what's the FreeBSD way to do that... :\

The reason for trying this driver is that the server sometimes freezes for several seconds. Or the the networkconnections, anyway. The system simply doesn't respond for a few seconds and that it returns to normal, like nothing happened. I cannot figure out why it does that. So I decided to give this official driver a try. Until now the problem hasn't occured yet.
 
Normally you can just type make, but since this particular module shadows a module with the same name in the base system it seems to be a bit more involved.

Here's what I did:

Code: 
[/home/martin/Free_BSD_Driver_P15]# rm -r /usr/src/sys/modules/mps /usr/src/sys/dev/mps/
[/home/martin/Free_BSD_Driver_P15]# tar xf mpslsi-source-15.00.00.00.tar.gz
[/home/martin/Free_BSD_Driver_P15]# mv mpslsi-source-15.00.00.00/sys/* /usr/src/sys/
[/home/martin/Free_BSD_Driver_P15]# cd /usr/src/sys/modules/mps/        
[/usr/src/sys/modules/mps]# make
[... Make output snipped ... ]
[/usr/src/sys/modules/mps]# kldload ./mpslsi.ko
[/usr/src/sys/modules/mps]# kldstat
Id Refs Address            Size     Name
 1   24 0xffffffff80200000 1323420  kernel
 2    2 0xffffffff81612000 27f8     vboxnetflt.ko
 3    2 0xffffffff81615000 87b2     netgraph.ko
 4    2 0xffffffff8161e000 30020    vboxdrv.ko
 5    1 0xffffffff8164f000 1579     ng_ether.ko
 6    1 0xffffffff81651000 3f20     vboxnetadp.ko
 7    1 0xffffffff81655000 1f445    linux.ko
 8    1 0xffffffff81675000 64a50    radeon.ko
 9    1 0xffffffff816da000 1391f    drm.ko
10    1 0xffffffff81784000 1b8a4    mpslsi.ko

You probably don't *need* to remove the directories from /usr/src/, you can also edit the Makefile or somesuch. This seemed the quicker answer :)
 
Ok, I managed to compile the driver and use that mpslsi.ko. However, during boot, I see a lot of this rolling over the screen. I see it again when I check if the driver is loaded with [cmd=]sysctl -a | grep mps[/cmd]

Code: 
(probe1020:mpslsi0:0:1021:0): CAM status: Invalid Target ID
(probe1020:mpslsi0:0:1021:0): Error 22, Unretryable error
(probe1020:mpslsi0:0:1021:0): INQUIRY. CDB: 12 0 0 0 24 0 
(probe1020:mpslsi0:0:1021:0): CAM status: Invalid Target ID
(probe1020:mpslsi0:0:1021:0): Error 22, Unretryable error
(probe1021:mpslsi0:0:1022:0): INQUIRY. CDB: 12 0 0 0 24 0 
(probe1021:mpslsi0:0:1022:0): CAM status: Invalid Target ID
(probe1021:mpslsi0:0:1022:0): Error 22, Unretryable error
(probe1021:mpslsi0:0:1022:0): INQUIRY. CDB: 12 0 0 0 24 0 
(probe1021:mpslsi0:0:1022:0): CAM status: Invalid Target ID
(probe1021:mpslsi0:0:1022:0): Error 22, Unretryable error
(probe1022:mpslsi0:0:1023:0): INQUIRY. CDB: 12 0 0 0 24 0 
(probe1022:mpslsi0:0:1023:0): CAM status: Invalid Target ID
(probe1022:mpslsi0:0:1023:0): Error 22, Unretryable error
(probe1022:mpslsi0:0:1023:0): INQUIRY. CDB: 12 0 0 0 24 0 
(probe1022:mpslsi0:0:1023:0): CAM status: Invalid Target ID
(probe1022:mpslsi0:0:1023:0): Error 22, Unretryable error

Right after these errors, I see this
Code: 
ses0 at mpslsi0 bus 0 scbus0 target 20 lun 0
da0 at mpslsi0 bus 0 scbus0 target 8 lun 0
da1 at mpslsi0 bus 0 scbus0 target 9 lun 0
da2 at mpslsi0 bus 0 scbus0 target 10 lun 0
da4 at mpslsi0 bus 0 scbus0 target 12 lun 0
da5 at mpslsi0 bus 0 scbus0 target 13 lun 0
da3 at mpslsi0 bus 0 scbus0 target 11 lun 0
da7 at mpslsi0 bus 0 scbus0 target 15 lun 0
da8 at mpslsi0 bus 0 scbus0 target 16 lun 0
da9 at mpslsi0 bus 0 scbus0 target 17 lun 0
da6 at mpslsi0 bus 0 scbus0 target 14 lun 0
da10 at mpslsi0 bus 0 scbus0 target 18 lun 0
da11 at mpslsi0 bus 0 scbus0 target 19 lun 0
dev.mpslsi.0.%desc: LSI SAS2308
dev.mpslsi.0.%driver: mpslsi
dev.mpslsi.0.%location: slot=0 function=0
dev.mpslsi.0.%pnpinfo: vendor=0x1000 device=0x0087 subvendor=0x1000 subdevice=0x3030 class=0x010700
dev.mpslsi.0.%parent: pci3
dev.mpslsi.0.debug_level: 4
dev.mpslsi.0.disable_msix: 0
dev.mpslsi.0.disable_msi: 0
dev.mpslsi.0.firmware_version: 15.00.00.00
dev.mpslsi.0.driver_version: 15.00.00.00
dev.mpslsi.0.io_cmds_active: 0
dev.mpslsi.0.io_cmds_highwater: 80
dev.mpslsi.0.chain_free: 2048
dev.mpslsi.0.chain_free_lowwater: 2030
dev.mpslsi.0.max_chains: 2048
dev.mpslsi.0.chain_alloc_fail: 0

I have no idea what these errors mean and if they are a problem. Because, everything seems to work fine.
 
mariourk said:
This server is running 9.1-RELEASE. I don't know if that makes any difference.
Click to expand...
9-STABLE is a development branch, so it gets fixes (and sometimes new bugs :)). 9.1-RELEASE hardly gets anything except important security patches. The handbook explains how to track a development branch, see for yourself whether it's worth for you to try.
 
In that case, I'll stick with 9.1-RELEASE. It seems like the smart choice for a production server.
 
mariourk said:
In that case, I'll stick with 9.1-RELEASE. It seems like the smart choice for a production server.
Click to expand...

The smart choice really depends on your hardware. We are sharing the same problem and I presented you with a solution that solved it.

My approach to the problem was:

  1. Use FreeBSD 9.1 with driver 14.
  2. Use FreeBSD 9.1 with driver 15. (From LSI)
  3. Use FreeBSD 9.1-STABLE

The server is a SuperMicro driving a 45 drive JBOD chassis enclosure. In all cases the system was tested with 11 drives on a RAIDz3 pool.
 
Actually, I tried to move to 9.1-STABLE, but that didn't work.
Code: 
freebsd-update -r 9.1-STABLE upgrade
I suppose I'm doing something horribly wrong ;)

What is the proper way to move from 9.1-RELEASE to 9.1-STABLE?
 
mariourk said:
Actually, I tried to move to 9.1-STABLE, but that didn't work.
Code: 
freebsd-update -r 9.1-STABLE upgrade
I suppose I'm doing something horribly wrong ;)

What is the proper way to move from 9.1-RELEASE to 9.1-STABLE?
Click to expand...

-STABLE is not a `release', it's a branch in svn/CVS, from this -STABLE branch new -RELEASE branches are created, and from these -RELEASE branches distribution sets, install CDs, etc. are created.

With time on the x-axis:
Code: 
------------ CURRENT/HEAD -------->
   \
    \-> 9-STABLE  -------------------------->
              \                    \
               \-> 9.0-RELEASE      \-> 9.1-RELEASE

freebsd-update only works with -RELEASE, so you'll need to install from source, this is not very hard and is described in the handbook (in particular, section 25.6 and 25.7).

Once you have the source, /usr/src/Makefile also describers the basic commands which may be useful as a quick terse reference:
Code: 
# For individuals wanting to upgrade their sources (even if only a
# delta of a few days):
#
#  1.  `cd /usr/src'       (or to the directory containing your source tree).
#  2.  `make buildworld'
#  3.  `make buildkernel KERNCONF=YOUR_KERNEL_HERE'     (default is GENERIC).
#  4.  `make installkernel KERNCONF=YOUR_KERNEL_HERE'   (default is GENERIC).
#       [steps 3. & 4. can be combined by using the "kernel" target]
#  5.  `reboot'        (in single user mode: boot -s from the loader prompt).
#  6.  `mergemaster -p'
#  7.  `make installworld'
#  8.  `make delete-old'
#  9.  `mergemaster'    (you may wish to use -i, along with -U or -F).
# 10.  `reboot'
# 11.  `make delete-old-libs' (in case no 3rd party program uses them anymore)
 
As someone who is new to FreeBSD, I think I'll stick with REALEASE for a while. Especially for the production-servers... :\

Despite having these errors, the system itself shows no problems and runs fine.

Could someone explain what these errors exactly mean? And how serious they are?
 
Could someone explain what these errors exactly mean? And how serious they are?
Click to expand...

(probe1020:mpslsi0:0:1021:0): INQUIRY. CDB: 12 0 0 0 24 0
(probe1020:mpslsi0:0:1021:0): CAM status: Invalid Target ID
(probe1020:mpslsi0:0:1021:0): Error 22, Unretryable error
Click to expand...

The SCSI INQUIRY command gets basic info from the device, in this case the command fails because the target ID 12 is considered invalid for whatever reason.

What this *exactly* means or why this is caused is difficult to say. It *may* be a symptom of a more serious problem, or it may not be.
If you want a definitive answer, then a deeper investigation into the driver is required. This is something that's outside the scope of these forums. There are a number of FreeBSD mailing lists such as freebsd-scsi@ where people with more specific knowledge of this subject hang out.
 
in this case the command fails because the target ID 12 is considered invalid for whatever reason.
Click to expand...
This server has 12 SAS disks. da0 to da11. I suppose that's the reason because ID 12 is invalid? And that it's not a big deal?
 
mariourk said:
This server has 12 SAS disks. da0 to da11. I suppose that's the reason because ID 12 is invalid? And that it's not a big deal?
Click to expand...

This *may* be the case, or it *may* be something else (such as trying to get the status of an existing disk). You can use camcontrol devlist to find out about your disks and which LUN they have.
 
That gives me this
Code: 
<SEAGATE ST33000650SS 0004>        at scbus0 target 8 lun 0 (da0,pass0)
<SEAGATE ST33000650SS 0004>        at scbus0 target 9 lun 0 (da1,pass1)
<SEAGATE ST3300657SS 0008>         at scbus0 target 10 lun 0 (da2,pass2)
<SEAGATE ST3300657SS 0008>         at scbus0 target 11 lun 0 (da3,pass3)
<SEAGATE ST33000650SS 0004>        at scbus0 target 12 lun 0 (da4,pass4)
<SEAGATE ST33000650SS 0004>        at scbus0 target 13 lun 0 (da5,pass5)
<SEAGATE ST33000650SS 0004>        at scbus0 target 14 lun 0 (da6,pass6)
<SEAGATE ST33000650SS 0004>        at scbus0 target 15 lun 0 (da7,pass7)
<SEAGATE ST33000650SS 0004>        at scbus0 target 16 lun 0 (da8,pass8)
<SEAGATE ST33000650SS 0004>        at scbus0 target 17 lun 0 (da9,pass9)
<SEAGATE ST33000650SS 0004>        at scbus0 target 18 lun 0 (da10,pass10)
<SEAGATE ST33000650SS 0004>        at scbus0 target 19 lun 0 (da11,pass11)
< 80H10323501A0 0703>              at scbus0 target 20 lun 0 (ses0,pass12)
All the disks are present. I have no idea what ses0 is.
 
@mariourk,

Please post or send me the full dmesg of the server. I strongly believe that we are facing the same bug here.

If you want to try 9.1-STABLE the procedure is pretty easy.

@Carpetsmoker,

The issue has been raised in the mailing list(s).
 
This is the output of [cmd=]dmesg[/cmd]
Code: 
[... lots of probe messages snipped, they are all the same ...]

(probe1021:mpslsi0:0:1022:0): INQUIRY. CDB: 12 0 0 0 24 0
(probe1021:mpslsi0:0:1022:0): CAM status: Invalid Target ID
(probe1021:mpslsi0:0:1022:0): Error 22, Unretryable error
(probe1021:mpslsi0:0:1022:0): INQUIRY. CDB: 12 0 0 0 24 0
(probe1021:mpslsi0:0:1022:0): CAM status: Invalid Target ID
(probe1021:mpslsi0:0:1022:0): Error 22, Unretryable error
(probe1022:mpslsi0:0:1023:0): INQUIRY. CDB: 12 0 0 0 24 0
(probe1022:mpslsi0:0:1023:0): CAM status: Invalid Target ID
(probe1022:mpslsi0:0:1023:0): Error 22, Unretryable error
(probe1022:mpslsi0:0:1023:0): INQUIRY. CDB: 12 0 0 0 24 0
(probe1022:mpslsi0:0:1023:0): CAM status: Invalid Target ID
(probe1022:mpslsi0:0:1023:0): Error 22, Unretryable error
ses0 at mpslsi0 bus 0 scbus0 target 20 lun 0
ses0: < 80H10323501A0 0703> Fixed Enclosure Services SCSI-5 device
ses0: 600.000MB/s transfers
ses0: Command Queueing enabled
ses0: SCSI-3 SES Device
ums0: <ATEN ATEN  CS-175854, class 0/0, rev 1.10/1.00, addr 3> on usbus1
da0 at mpslsi0 bus 0 scbus0 target 8 lun 0
da0: <SEAGATE ST33000650SS 0004> Fixed Direct Access SCSI-6 device
da0: 600.000MB/s transfers
da0: Command Queueing enabled
da0: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
da1 at mpslsi0 bus 0 scbus0 target 9 lun 0
da1: <SEAGATE ST33000650SS 0004> Fixed Direct Access SCSI-6 device
da1: 600.000MB/s transfers
da1: Command Queueing enabled
da1: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
da2 at mpslsi0 bus 0 scbus0 target 10 lun 0
da2: <SEAGATE ST3300657SS 0008> Fixed Direct Access SCSI-5 device
da2: 600.000MB/s transfers
da2: Command Queueing enabled
da2: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da4 at mpslsi0 bus 0 scbus0 target 12 lun 0
da4: <SEAGATE ST33000650SS 0004> Fixed Direct Access SCSI-6 device
da4: 600.000MB/s transfers
da4: Command Queueing enabled
da4: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
da5 at mpslsi0 bus 0 scbus0 target 13 lun 0
da5: <SEAGATE ST33000650SS 0004> Fixed Direct Access SCSI-6 device
da5: 600.000MB/s transfers
da5: Command Queueing enabled
da5: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
da3 at mpslsi0 bus 0 scbus0 target 11 lun 0
da3: <SEAGATE ST3300657SS 0008> Fixed Direct Access SCSI-5 device
da3: 600.000MB/s transfers
da3: Command Queueing enabled
da3: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
da7 at mpslsi0 bus 0 scbus0 target 15 lun 0
da7: <SEAGATE ST33000650SS 0004> Fixed Direct Access SCSI-6 device
da7: 600.000MB/s transfers
da7: Command Queueing enabled
da7: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
da8 at mpslsi0 bus 0 scbus0 target 16 lun 0
da8: <SEAGATE ST33000650SS 0004> Fixed Direct Access SCSI-6 device
da8: 600.000MB/s transfers
da8: Command Queueing enabled
da8: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
da9 at mpslsi0 bus 0 scbus0 target 17 lun 0
da9: <SEAGATE ST33000650SS 0004> Fixed Direct Access SCSI-6 device
da9: 600.000MB/s transfers
da9: Command Queueing enabled
da9: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
da6 at mpslsi0 bus 0 scbus0 target 14 lun 0
da6: <SEAGATE ST33000650SS 0004> Fixed Direct Access SCSI-6 device
da6: 600.000MB/s transfers
da6: Command Queueing enabled
da6: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
ums0: 5 buttons and [XYZ] coordinates ID=0
SMP: AP CPU #7 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #1 Launched!
SMP: AP CPU #4 Launched!
SMP: AP CPU #2 Launched!
SMP: AP CPU #5 Launched!
SMP: AP CPU #6 Launched!
da10 at mpslsi0 bus 0 scbus0 target 18 lun 0
da10: <SEAGATE ST33000650SS 0004> Fixed Direct Access SCSI-6 device
da10: 600.000MB/s transfers
da10: Command Queueing enabled
da10: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
da11 at mpslsi0 bus 0 scbus0 target 19 lun 0
da11: <SEAGATE ST33000650SS 0004> Fixed Direct Access SCSI-6 device
da11: 600.000MB/s transfers
da11: Command Queueing enabled
da11: 2861588MB (5860533168 512 byte sectors: 255H 63S/T 364801C)
Timecounter "TSC-low" frequency 8593937 Hz quality 1000
GEOM_MIRROR: Device mirror/gm0 launched (2/2).
Root mount waiting for: usbus0
uhub4: 5 ports with 5 removable, self powered
ugen0.4: <American Megatrends Inc.> at usbus0
ukbd1: <Keyboard Interface> on usbus0
kbd2 at ukbd1
ums1: <Mouse Interface> on usbus0
ums1: 3 buttons and [Z] coordinates ID=0
Trying to mount root from ufs:/dev/mirror/gm0a [rw]...
ZFS filesystem version 5
ZFS storage pool version 28
em0: link state changed to UP
lagg0: link state changed to UP
em1: link state changed to UP
em3: link state changed to UP
em2: link state changed to UP
 
gkontos said:
My approach to the problem was:

  1. Use FreeBSD 9.1 with driver 14.
  2. Use FreeBSD 9.1 with driver 15. (From LSI)
  3. Use FreeBSD 9.1-STABLE
Click to expand...

Just curious, since I am hitting a very similar scenario. Did you use the mps driver from 9.1 STABLE? Or did you compile LSI mpslsi drivers using 9.1 STABLE sources?

Thanks
 
boris_net said:
Just curious, since I am hitting a very similar scenario. Did you use the mps driver from 9.1 STABLE? Or did you compile LSI mpslsi drivers using 9.1 STABLE sources?

Thanks
Click to expand...

I tried both and I kept getting some weird errors with the LSI driver so I decided to use the driver form 9.1-STABLE.

As a side note, the machine is running 9.1-STABLE which was compiled around three months ago. There have been a lot of enhancements and bug fixes since then but we are currently unable to find a suitable window for another upgrade.
 
gkontos said:
I tried both and I kept getting some weird errors with the LSI driver so I decided to use the driver form 9.1-STABLE.

As a side note, the machine is running 9.1-STABLE which was compiled around 3 moths ago. There have been a lot of enhancements and bug fixes since then but we are currently unable to find a suitable window for another upgrade.
Click to expand...

The same applies to me. Let me know how it turns out if you do upgrade. I will do the same, if I happen to do the upgrade before you ;)
 
Hi all,

What kind of version of drivers do you get from 9.1-STABLE? I am still on 14.00.00.01-fbsd.

Code: 
uname -a
FreeBSD houdini 9.1-STABLE FreeBSD 9.1-STABLE #0 r251482: Fri Jun  7 10:21:16 EDT 2013     root@houdini:/usr/obj/usr/src/sys/NOYO  amd64

Code: 
sysctl -a  | grep mps | grep version
dev.mps.0.firmware_version: 15.00.00.00
dev.mps.0.driver_version: 14.00.00.01-fbsd
dev.mps.1.firmware_version: 15.00.00.00
dev.mps.1.driver_version: 14.00.00.01-fbsd
dev.mps.2.firmware_version: 15.00.00.00
dev.mps.2.driver_version: 14.00.00.01-fbsd

I have also seen a 16.00.00.00 for FreeBSD on the LSI website. Has anybody tested it?

Boris
 
You must log in or register to reply here.
Back
Top