Solved [Solved] I/O fencing. Device not configured

V

vadiraj

Hi,

I'm trying I/O fencing on FreeBSD 9 using sg_persist command. The testing is on ZFS pools. For simplicity I'm trying on a single disk pool.

Here is what I'm doing for fencing.

Node A:
Code: 
sg_persist --out --register --param-sark=123abc /dev/da1
sg_persist --out --preempt-abort --param-sark=123abd --param-rk=123abc --prout-type=5 /dev/da1
While this preempts and aborts other node with key 123abd and reserves with TYPE=5. This works great and other node say Node B is not allowed any write on this disk.

Now when I want to take over the rights due to malfunction of Node A. On Node B I do the following.
Code: 
sg_persist --out --register --param-sark=123abd /dev/da1
sg_persist --out --preempt-abort --param-rk=123abd --param-sark=123abc --prout-type=5 /dev/da1
This works fine too.

But the problem I'm facing now is, after this ping pong, Node A loses access to disk.

Code: 
 cat /dev/da1
cat '/dev/da1': Device not configured
However sg_* commands work fine. I cannot import this pool even after releasing reservation from Node B and acquiring it from Node A.

Code: 
NodeA:~ # sg_persist --out --preempt-abort --param-sark=123abd --param-rk=123abc --prout-type=5 /dev/da1
  HP        DG146BB976        HPDC
  Peripheral device type: disk

NodeA:~ # sg_persist -i /dev/da1
>> No service action given; assume Persistent Reserve In command
>> with Read Keys service action
  HP        DG146BB976        HPDC
  Peripheral device type: disk
  PR generation=0x9, 1 registered reservation key follows:
    0x123abc

NodeA:~ # cat /dev/da1
cat: /dev/da1: Device not configured

Reboots have not been helpful either. However after couple of sysctl debug.kdb.panic=1 command seem to make /dev/da1 readable. I believe couple of reboots should also make this disk readable from Node A. But that's not right.

Could someone tell me what's going on here? Why is the Node A not able to read from /dev/da1 after this exercise. I don't see any thing printed on /var/log/messages or /var/log/devd.log.

Thanks in advance.
 
I noticed that the device gets into this if the disks were reserved by another node during bootup. To get around with this is to ensure no reservations are held during a node or a partner bootup. I'm trying to understand which layer and why this happens even when the reservations on the disk are write exclusive.

Any heads up on this will be of great help.

Thanks,
Vadiraj
 
Re: I/O fencing. Device not configured

I found out that this is a firmware issue. The newer disks do not have this issue. From our analysis we see that disk firmware prior to 2004 has this problem.

Solution: update the firmware of the disk.

Thanks,
Vadiraj
 
You must log in or register to reply here.
Back
Top