Other sesutil only shows SATA/AHCI, not SAS

sko · Mar 8, 2023

I'm in the process of setting up a new server on a supermicro dual-node system (2028TP-DC0FR).
This system has a 24-bay backplane, of which 12 bays are connected to each node. The nodes have a riser with a SAS3008 HBA that connects to the first 8 bays, the last 4 are connected via SATA.

I'm using sesutil at other servers (e.g. with one SAS3008-8i HBA and an expander-backplane) and it shows me all bays and installed drives:

Code:

# sesutil show
ses0: <LSI SAS3x28 0501>; ID: 500304800189ee3f
Desc     Dev     Model                     Ident                Size/Status
Slot00   -       -                         -                    Not Installed
Slot01   -       -                         -                    Not Installed
Slot02   -       -                         -                    Not Installed
Slot03   da8     SanDisk DDYE1920S5xnNMRI  A03AB4A3             2T
Slot04   da9     SanDisk DDYE1920S5xnNMRI  A040672D             2T
Slot05   da10    SanDisk DDYE1920S5xnNMRI  A03AC4E2             2T
Slot06   da1     SanDisk DDYE1920S5xnNMRI  A0417FEC             OK,  Swapped
Slot07   da0     SanDisk DDYE1920S5xnNMRI  A03AC901             OK,  Swapped
Slot08   da7     SanDisk DDYE1920S5xnNMRI  A03AC724             2T
Slot09   -       -                         -                    Not Installed
Slot10   -       -                         -                    Not Installed
Slot11   -       -                         -                    Not Installed

however, on that dual node system I only get SATA/AHCI drives and none of the SAS bays show up:

Code:

# sesutil show
ses0: <AHCI SGPIO Enclosure 2.00>; ID: 3061686369656d30
Desc     Dev     Model                     Ident                Size/Status
Slot 00  -       -                         -                    Not Installed
Slot 01  -       -                         -                    Not Installed
Slot 02  -       -                         -                    Not Installed
Slot 03  -       -                         -                    Not Installed

ses1: <AHCI SGPIO Enclosure 2.00>; ID: 3061686369656d31
Desc     Dev     Model                     Ident                Size/Status
Slot 00  ada0    INTEL SSDSC2BB120G6       BTWA601106V2120CGN   120G
Slot 01  ada1    INTEL SSDSC2BB120G6       BTWA601200SA120CGN   120G
Slot 02  -       -                         -                    Not Installed
Slot 03  -       -                         -                    Not Installed
Slot 04  -       -                         -                    Not Installed
Slot 05  -       -                         -                    Not Installed

To see the SAS drives I have to revert to mprutil show devices/adapter:

Code:

# mprutil show devices
B____T    SAS Address      Handle  Parent    Device        Speed Enc  Slot  Wdt
00   02   5001173100759ff9 0009    0001      SAS Target    6.0   0001 00    1
00   03   5001173100750f11 000a    0002      SAS Target    6.0   0001 01    1

# mprutil show adapter
mpr0 Adapter:
       Board Name: LSI3008-IR
   Board Assembly: 
        Chip Name: LSISAS3008
    Chip Revision: ALL
    BIOS Revision: 18.00.00.00
Firmware Revision: 12.00.02.00
  Integrated RAID: yes
         SATA NCQ: ENABLED
 PCIe Width/Speed: x8 (8.0 GB/sec)
        IOC Speed: Full
      Temperature: 63 C

PhyNum  CtlrHandle  DevHandle  Disabled  Speed   Min    Max    Device
0       0001        0009       N         6.0     3.0    12     SAS Initiator 
1       0002        000a       N         6.0     3.0    12     SAS Initiator 
2                              N                 3.0    12     SAS Initiator 
3                              N                 3.0    12     SAS Initiator 
4                              N                 3.0    12     SAS Initiator 
5                              N                 3.0    12     SAS Initiator 
6                              N                 3.0    12     SAS Initiator 
7                              N                 3.0    12     SAS Initiator

Is there any way I can get sesutil to show _all_ bays or even only the SAS ones? mprutil is rather useless, as it only shows SAS adresses, which don't translate to anything useful. I'd rather use sesutil as on other systems, which shows device nodes and serial numbers and can directl locate drives via LED.
I absolutely don't care about SATA, so it wouldn't be a huge loss if they vanish from the sesutil-output...

cracauer@ · Mar 8, 2023

Can confirm, doesn't show SAS for me either.

sko · Mar 10, 2023

Had to locate some disks in an old server with the same problem: sesutil only shows some fictional 5-slot enclosure with the only drive in that being the CD-ROM drive...
The real, 16-bay backplane is connected to 2 SAS2008 HBAs, and it's not even sure which one the system considers to be 0 and 1.

I managed to find the disks via this workflow:

Code:

# smartctl -x /dev/da12 | grep -m2 'SAS address'
    SAS address = 0x500117310108d179
    attached SAS address = 0x500605b003554070

first line is the drives address, second one the enclosure, now search for the drive on the controllers:

Code:

# mpsutil -u0 show all | grep 500117310108d179
# mpsutil -u1 show all | grep 500117310108d179
00   12   500117310108d179 0009    0001      SAS Target    6.0   0001 03    1
# # mpsutil -u1 show devices
B____T    SAS Address      Handle  Parent    Device        Speed Enc  Slot  Wdt
00   12   500117310108d179 0009    0001      SAS Target    6.0   0001 03    1
00   00   4433221103000000 000b    0003      SATA Target   6.0   0001 00    1
00   11   5000cca0c2a3aa25 000c    0004      SAS Target    6.0   0001 02    1
00   02   5000cca255027f11 000d    0005      SAS Target    6.0   0001 01    1
00   10   5000cca0c2abe779 000e    0006      SAS Target    6.0   0001 06    1
00   08   5000cca255de1781 000f    0007      SAS Target    6.0   0001 05    1

So it's the drive in slot 3 on the second (-u1) controller. For mp[s/r]util there is no "locate" command like there is for sesutil, but there's mpsutil slot set status, which sounds promising - but it isn't documented. So by try&error (the disks weren't mounted or part of any pool, so WCGW?) I found out that "status 02" lights up the fault LED, so I used that to find the correct drive:

Code:

# mpsutil -u1 slot set status 0001 03 02

Other sesutil only shows SATA/AHCI, not SAS

sko

cracauer@

sko