ZFS Dell perch 730/830 zfs mrsas driver.

I have a doubt if these Raid controllers are valid for the use of ZFS with the mrsas(4) driver.
Yes, these two drivers are supported and listed in the manual, but would they give problems for the use of ZFS?I have seen that these two controllers can be put in non-raid mode. I understand that this could be like an HBA mode, right?

Sorry if I'm saying something crazy, I currently have a perch 310 mini in IT mode, should I want to ask this question with those two controllers. So if you put those two controllers in non-raid mode, with the mrsas controller you would have the disks identified as da(4) and it would be manipulable with cam(4).

It would be correct to use these two controllers in the way described?

Thanks guys.
 
Don't know these specific controllers but I've used other LSI based RAID controllers. As long as you can set the drives as JBOD it'll be fine.
 
I also explain it in another way. I think I was not very precise before.

When I had my Perch 310 mini with Dell firmware, it was by default here managed by the driver mfi(4). I understand why this controller was running Dell firmware in "raid mode".

And I had to configure the disks in RAID-0 so that I could control or so to speak each disk as mfidX. Ok when I changed the firmware to run under IT "HBA" mode the controller used the mpr(4) driver, so my drives showed up as daX and I could read them with SMART.

Reading the mrsas manual when I connect the PERCH 730 I can choose the mrsas driver by modifying device.hints

Ok then, assuming with the PERCH 730 it is in "raid mode" also if it is under the mrsas controller the disks will appear as daX and could be controlled with camcontrol(8). But I understand that it would not be like HBA mode.

In any case, if you used the mfi(4) controller with the PERCH 730, you would still use the mfidX devices.

So if we put the controllers in "non-raid mode" would the mpr(4) driver or another be used by default?

Would it be like applying a firmware HBA mode? I think now my question is better understood. That is my question if I could do the same with these two controllers just like my 310 mini in it mode.

Thanks.


P.S:

Is the mrsas driver intended for use with the controller in "original firmware" raid mode?

In case of changing it to "non-raid mode" would you use drivers like mpr etc... as if you had the firmware of an HBA?


P.S.2:

I found this similar topic, but the user doesn't specify which driver he is using.

TOPIC
 
No, the difference between mfi(4) and mrsas(4) isn't RAID mode. It has nothing to do with RAID or the controller. It's just a different way of accessing drives (or volumes), that's a more "under the hood" kind of thing; how the driver is integrated into the existing disk framework of FreeBSD.

And I had to configure the disks in RAID-0 so that I could control or so to speak each disk as mfidX. Ok when I changed the firmware to run under IT "HBA" mode the controller used the mpr(4) driver, so my drives showed up as daX and I could read them with SMART.
Single disk RAID0 volume is not the same as JBOD. It's a RAID volume, which means the controller has put some metadata on the disk that tells the controller the disk is part of a RAID set (even if it's just a single drive). That has nothing to do with mfi(4) vs. mrsas(4). If I configure a mfi(4) controller to set the disks as JBOD those disks would show up as mfid0, etc. If I change the driver to mrsas(4) I would get the exact same drive, the only thing difference would be it showing up as da0. It's still the same JBOD drive you're accessing. A HBA cannot read or understand that RAID0 metadata, thus it will not be able to read the RAID0 'volume' that's on that disk.
 
Ok I understand what you told me, all that would be using a firmware and some raid controllers.

Now let's forget what I said about non-raid mode. I have read on the dell page that the 730 and 830 can be switched to HBA mode.

Mode HBA

So that would be the most optimal option if we are working with ZFS, at least as far as I know. Otherwise I will use the method that you told me.

Well now knowing this, I understand that this RAID controller would behave like an LSI HBA, FreeBSD would be able to access the disks directly and be able to use SMARTCTL and ZFS would have direct access to the disks.

But what controller would you use? MFI or MRSAS or when passing it to HBA mode, would you use another type of driver as I mentioned above?

To what extent it changes the behavior of the driver at the driver level.

Thanks.
 
MFI or MRSAS or when passing it to HBA mode, would you use another type of driver as I mentioned above?
It depends on the controller. Some LSI controllers only work with mfi(4), some only with mrsas(4) and some cards will work on both drivers. I would prefer the mrsas(4) driver, it's newer and is the direction LSI/Broadcom will be going for the future. The mrsas(4) also integrates better with the FreeBSD driver framework.

Only downside of mrsas(4) is that you can't use tools like mfiutil(8), although I do believe someone is updating the tool to work with mrsas(8). In all cases you can use sysutils/megacli regardless of the driver or type of LSI card.
 
I understand, in case of using mrsas I could not configure the controller through the system, I would have to restart my server and use the dell configuration tool.

But actually I have no problem using mrsas or mfi, I'm only interested in being able to use that controller in HBA mode, so you said mfi would have an advantage over mrsas, you could manage the controller through FreeBSD.

But basically my doubts lie there, not knowing or not knowing how it will behave when I put that controller in HBA mode, in any case if it is used by mfi, the mfidX devices would have to be accessible with camcontrol? smartcl? since they would not be in a raid.

I don't want to mess up the issue anymore, maybe I'll buy a 730p and see the results.

On the other hand, since the controller is in HBA and with ZFS, would I have to remove the battery?

I ask this because in my 310mini when I updated the firmware to IT mode I removed the battery.

Thanks.
 
Is there a list anywhere of which LSI controllers will work with mfi(4) and which will work with mrsas(4), or both? I'm still running FreeBSD 10.x or 11.x on some of my servers fitted with Dell H730P controllers (LSI Megaraid SAS-3 3108 'Invader') since with these older FreeBSD versions, I can load the mrsas driver at boot time using an entry in /boot/loader.conf and use that driver. But if I upgrade any of these to 13.0 with its integrated mrsas(4) driver, the system will only use the mfi(4) driver and any attempts to load mrsas(4) manually using kldload fail. So I think I need to replace the H730P with some other controller known to work with mrsas(4).
 
I can load the mrsas driver at boot time using an entry in /boot/loader.conf and use that driver. But if I upgrade any of these to 13.0 with its integrated mrsas(4) driver, the system will only use the mfi(4) driver and any attempts to load mrsas(4) manually using kldload fail.
You need the same entry in device.hints on 13.x. It still works the same way.
Code:
     Using /boot/device.hints (as mentioned below), the user can provide a
     preference for the mrsas driver to detect a MR-Fusion card instead of the
     mfi(4) driver.

           hw.mfi.mrsas_enable="1"
 
Thanks! I didn't know device.hints also needed an entry. I assume that since device names will change from /dev/mfisyspdXX to /dev/daXX if the driver is switched to mrsas(4), any ZFS pools on these servers will need to be rebuilt?

Thanks
 
Thanks! I didn't know device.hints also needed an entry. I assume that since device names will change from /dev/mfisyspdXX to /dev/daXX if the driver is switched to mrsas(4), any ZFS pools on these servers will need to be rebuilt?

Thanks

Quite possibly, because if you built the pool with some devices that are not existing now, you will not be able to use it. On the other hand, I recommend that you look at a thread that I opened not long ago.

Path

It's about gmultipath(8) I guess if you put labels on your devices and build the pool with those labels. It will not depend on the type of controller you are using, since zfs will not use mfidX or daX to identify the vdevs in your pool, only your labels.

If I'm not mistaken that could be a solution, if I'm wrong tell me.

Thanks guys.
 
any ZFS pools on these servers will need to be rebuilt?
Don't need to. ZFS will detect the pool based on the metadata that's on the disk. It's irrelevant for ZFS what disk nomination it gets, it'll find everything automatically. It might take a little longer the first time you boot a 'switched' system but ZFS will find everything anyway (assuming the pool is complete and all disks from that pool are available).
 
But if I upgrade any of these to 13.0 with its integrated mrsas(4) driver, the system will only use the mfi(4) driver and any attempts to load mrsas(4) manually using kldload fail. So I think I need to replace the H730P with some other controller known to work with mrsas(4).
Not sure what you mean by this - I've got several servers running with the PERC 730 and FreeBSD 13.2 (from 13.0 and then 13.1). They are using the hardware RAID and UFS, so not sure if there's something different if you are putting it into HBA mode?
Code:
AVAGO MegaRAID SAS FreeBSD mrsas driver version: 07.709.04.00-fbsd
mrsas0: <AVAGO Invader SAS Controller> port 0x2000-0x20ff mem 0x91d00000-0x91d0ffff,0x91c00000-0x91cfffff at device 0.0 numa-domain 0 on pci2
This in /boot/loader.conf:
Code:
hw.mfi.mrsas_enable=1

I don't have anything for this in /boot/device.hints.
 
Don't need to. ZFS will detect the pool based on the metadata that's on the disk. It's irrelevant for ZFS what disk nomination it gets, it'll find everything automatically. It might take a little longer the first time you boot a 'switched' system but ZFS will find everything anyway (assuming the pool is complete and all disks from that pool are available).
I didn't know you could do this so I'm currently backing up a 15TB ZFS RAIDz1 pool with 8TB used on one of my systems before switching the pool to using mrsas(4). This will take some time since pool I/O is very poor at about 17MB/sec - I'll report back later on how I get on.
 
I assume that since device names will change from /dev/mfisyspdXX to /dev/daXX if the driver is switched to mrsas(4)
Just for future readers of this thread - this is definitely an issue if using the PERC hardware RAID and UFS. My comments are only about PERC hardware RAID/mrsas/UFS, not ZFS.
 
Don't need to. ZFS will detect the pool based on the metadata that's on the disk. It's irrelevant for ZFS what disk nomination it gets, it'll find everything automatically. It might take a little longer the first time you boot a 'switched' system but ZFS will find everything anyway (assuming the pool is complete and all disks from that pool are available).
Apologies for the delayed update; this is just to confirm that inserting hw.mfi.mrsas_enable="1" into both /boot/loader.conf and /boot/device.hints and rebooting resulted in a test system using the mrsas(4) driver and when the ZFS pool was imported, it automatically replaced the previous /dev/mfisyspdXX devices with device names like gptid/14f487b5-adcb-11eb-a06e-1866da9c0da4

All is now working well on the initial test server and the same changes have since been made on other servers with the same hardware and FreeBSD versions - my thanks to all for their help with this.
 
So the use of labels would not be necessary, but in case of using gmultipath this would not happen since the device would be identified with the label, since this is stored in the device, and the gmultipath driver is loaded at the beginning of the boot together with the by zfs.

Then I am given to understand that the use of labels is at the user level only?

But the gptid tags will be created automatically?

I dont have any.

Code:
glabel status
Name  Status  Components
gpt/gptboot0     N/A  ada0p1

If I identify the guid with zpool status:

Code:
~ % zpool status -g
  pool: zroot
 state: ONLINE
config:

        NAME                   STATE     READ WRITE CKSUM
        zroot                  ONLINE       0     0     0
          6407458372809817099  ONLINE       0     0     0

errors: No known data errors

Code:
gpart list ada0 | grep efimedia
efimedia: HD(1,GPT,428d44b2-dc4f-11ed-8fe1-28d2443cb776,0x28,0x400)

I assume that the device is being identified with the guid shown below?

But then zfs is able to create the label in gptid and thus identify the device?

Thanks.
 
Last edited by a moderator:
Back
Top