camcontrol standby weirdness

F

frijsdijk

I'm trying to spindown disks in a zpool to save energy costs. This concerns a home NAS setup. I've broken in down to a single disk.

Here I want the disk to go into standby mode after 20 minutes of idle time:

camcontrol standby ada2 -t 1800

What actually happens is the disk spins down immediately. Even if it was just accessed.

Whenever I access the zpool/disk, it will spin up. Ah well, I could live with that. But, then, when the disk is idle for > 1800sec (monitor with "zpool iostat <zpool> 60"), it will not spin down. Huh?

The same goes for da0-da7, SATA disks connected to a SAS1068E flashed as pure HBA controller.

It's FreeBSD 11-RELEASE, on a Fujitsu d3417-b board with 16GB ECC DDR4. All works like sunshine.

I've found that "camcontrol standby" works with ZFS. "camcontrol idle" will also spindown the disks (immediately as well), but this breaks the zpool (ZFS considers the disk broken/disconnected).

ada2 (local backup disk on a seperate zpool) is a 4TB WDC disk connected to the motherboard:

Code: 
pass10: <WDC WD4000FYYZ-01UL1B1 01.01K02> ATA8-ACS SATA 3.x device
pass10: 600.000MB/s transfers (SATA 3.x, UDMA6, PIO 8192bytes)

protocol              ATA/ATAPI-8 SATA 3.x
device model          WDC WD4000FYYZ-01UL1B1
firmware revision     01.01K02
serial number         WD-WCC132283575
WWN                   50014ee2b4fc5ba2
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 512, offset 0
LBA supported         268435455 sectors
LBA48 supported       7814037168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6
media RPM             7200

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes      yes
write cache                    yes      yes
flush cache                    yes      yes
overlap                        no
Tagged Command Queuing (TCQ)   no       no
Native Command Queuing (NCQ)   yes              32 tags
NCQ Queue Management           no
NCQ Streaming                  no
Receive & Send FPDMA Queued    no
SMART                          yes      yes
microcode download             yes      yes
security                       yes      no
power management               yes      yes
advanced power management      yes      yes     128/0x80
automatic acoustic management  no       no
media status notification      no       no
power-up in Standby            yes      no
write-read-verify              no       no
unload                         yes      yes
general purpose logging        yes      yes
free-fall                      no       no
Data Set Management (DSM/TRIM) no
Host Protected Area (HPA)      yes      no      7814037168/7814037168
HPA - Security                 no


.. and then we have da0..da7 (the actual 14TB raidz1 zpool), 2TB WDC disks:

Code: 
pass0: <WDC WD2000FYYZ-01UL1B0 01.01K01> ATA8-ACS SATA 3.x device
pass0: 300.000MB/s transfers, Command Queueing Enabled

protocol              ATA/ATAPI-8 SATA 3.x
device model          WDC WD2000FYYZ-01UL1B0
firmware revision     01.01K01
serial number         WD-WCC1P0104881
WWN                   50014ee25d3e644d
cylinders             16383
heads                 16
sectors/track         63
sector size           logical 512, physical 512, offset 0
LBA supported         268435455 sectors
LBA48 supported       3907029168 sectors
PIO supported         PIO4
DMA supported         WDMA2 UDMA6
media RPM             7200

Feature                      Support  Enabled   Value           Vendor
read ahead                     yes      yes
write cache                    yes      yes
flush cache                    yes      yes
overlap                        no
Tagged Command Queuing (TCQ)   no       no
Native Command Queuing (NCQ)   yes              32 tags
NCQ Queue Management           no
NCQ Streaming                  no
Receive & Send FPDMA Queued    no
SMART                          yes      yes
microcode download             yes      yes
security                       yes      no
power management               yes      yes
advanced power management      yes      yes     128/0x80
automatic acoustic management  no       no
media status notification      no       no
power-up in Standby            yes      no
write-read-verify              no       no
unload                         yes      yes
general purpose logging        yes      yes
free-fall                      no       no
Data Set Management (DSM/TRIM) no
Host Protected Area (HPA)      yes      no      3907029168/3907029168
HPA - Security                 no

What am I doing wrong?
 
frijsdijk said:
(monitor with "zpool iostat <zpool> 60"), it will not spin down. Huh?
Click to expand...
I don't think zpool iostat would create activity on the pool; if there is no r/w activity on the pool, does that mean there are no disk accesses by zfs itself?
Also I don't think zfs is will be happy with disks taking a nap.
If you really want "go to sleep" functionality I would suggest to use the automounter in combination with some scripting combining camcontrol for sleep/idle/wake and zpool import once disks are spinning or zpool export before disks are put to sleep.
 
aribi said:
I don't think zpool iostat would create activity on the pool; if there is no r/w activity on the pool, does that mean there are no disk accesses by zfs itself?
Also I don't think zfs is will be happy with disks taking a nap.
If you really want "go to sleep" functionality I would suggest to use the automounter in combination with some scripting combining camcontrol for sleep/idle/wake and zpool import once disks are spinning or zpool export before disks are put to sleep.
Click to expand...

I've found a working way to do it, but it needs some scripting.

First of all: ZFS is fine with disks that go into STANDBY (accessing a zpool with disks in STANDBY will wake them up and ZFS will wait for it). ZFS cannot handle disks that are in IDLE mode (it considers the disks lost/defect/detached and the zpool will go degraded or even unavailable).

So I log a 'zpool iostat tank 60' and if tail -20 /var/log/io.log | grep -c '0 0 0 0$' = "20" , I consider the pool to be idle for 20 minutes, and I simply camcontrol idle <disk> for all disks in the pool. Accessing the pool (via Network/Samba) will wake them up and this causes no unexpected behaviour in FreeBSD or ZFS. Only had to take special care of situations where a client on the network just woke up the disks (or a cron job), and the spindown cron coming along spinning the disks down again. But that is solved.

In a NAS with 9 disks (a backup disks and 8 in a raidz1), this makes the wattage go down from 85-90W to about 37W, so that's really worth it. Specially if you consider that during an average day (until now), these disks are actually spinned down for 60-70% of the day.
 
Thanks for the hint :)

With my old drives I've used in /etc/rc.local:

Code: 
/sbin/camcontrol standby ada2 -t 900
/bin/dd if=/dev/ada2 of=/dev/null count=1
/sbin/camcontrol standby ada3 -t 900
/bin/dd if=/dev/ada3 of=/dev/null count=1
/sbin/camcontrol standby ada6 -t 900
/bin/dd if=/dev/ada6 of=/dev/null count=1
/sbin/camcontrol standby ada7 -t 900
/bin/dd if=/dev/ada7 of=/dev/null count=1

The first command sets internal timeout for go to STANDBY mode after 15 minutes of inactivity. As I found some day the second command (dd) is required (any activity on the disk) to start timer after system start.

But my new disks did not go to STANDBY mode in this way :( I saw in smartctl: IDLE_A mode.

EDIT: As written in the posts below: it has no chance to work. The following idea needs to be developed...

After reading the above post I've made simple script runs from cron every minute:

Code: 
#!/bin/sh

r_media=`zpool iostat media 900 1 | grep media | awk '{print $4}'`
w_media=`zpool iostat media 900 1 | grep media | awk '{print $5}'`

if [ "$r_media" == "0" ] && [ "$w_media" == "0" ] ; then
  /sbin/camcontrol standby ada2
  /sbin/camcontrol standby ada3
  /sbin/camcontrol standby ada6
  /sbin/camcontrol standby ada7
fi

If there is no read and no write operations on the zpool in last 15 minutes then all disks in the pool goes to standby. Works well for me :)
 
Last edited by a moderator:
The first entry of zpool iostat reports the statistics since boot. To avoid this behaviour you should use zpool iostat -y. Otherwise the disk won't ever go to standby when there had been any activity since boot.
 
The -y switch is available since 13.0-RELEASE.

I think zpool iostat or just iostat may be used to detect disk activity and manual switch to standby mode, but the script should be written with temporary data in ramdisk to compare activity counters between reads.

I thought that my script works, but it is not possible :( So it looks like my original rc.local idea still works with new drives.
 
marcinkk said:
...but the script should be written with temporary data in ramdisk to compare activity counters between reads.
Click to expand...

A read or write now doesn't mean there will be another one soon. However, a script that differentiates between active but idle, active but busy and standby will help you find the right time(s) to switch it to standby. Just run that script several times a day for a while. This is what I use for disks that won't go back into standby:

Code: 
#!/bin/sh
hdd="ada2"
tmp_file="/tmp/iostats"

iostat -c 2 -t da -x -w 60 -z > $tmp_file
if [ "$(grep -c $hdd $tmp_file)" -eq 2 ] ; then
    echo "has i/o"
else
    echo "has no i/o"
fi

Here, iostat reports statistics 2 times, first one since boot, second one after a 60 second wait. The -z switch suppresses devices without activity in the output. Therefore a device had r/w activity when it shows up twice in the output. If you have only one disk to check then you can do without the tmp file like this:

Code: 
hdd="ada2"
if [ "$(iostat -c 2 -t da -x -w 60 -z | grep -c $hdd)" -eq 2 ] ; then
    echo "has i/o"
fi

Btw, I'm using UFS. For ZFS you might want to use 'zpool iostat'.
 
frijsdijk said:
Accessing the pool (via Network/Samba) will wake them up and this causes no unexpected behaviour in FreeBSD or ZFS.
Click to expand...
I just set up a similar pool, and I also want to spin down the disks when nothing is happening.

I haven't done systematic tests, but It seems to me that some read operations on raidz vdevs will wake up the disks in sequence, instead of all at the same time. So, in a pool with 9 disks, that could give you … let's say 1 to 2 minutes of waiting for all the disks to come online. Not exactly sure when that happens, but be aware.

My guess is that zfs naively reads some metadata and some stripe data, and "discovers" that it needs to wake up one disk after another to read the whole file/metadata, but waits for each disk to come up before moving on, instead of waking up all devices in the pool at one. (I haven't seen a configuration knob to change that behavior, either.)

Now, one more thing I noticed, which might or might not help you: camcontrol standby works fine with my Seagate drives, but it sometimes didn't seem to work when I wanted to put all drives to sleep at the exact same time. So I staggered the sleep timeouts by 10 seconds ( camcontrol standby ada0 -t 350, camcontrol standby ada1 -t 360, camcontrol standby ada2 -t 370), and that seems to have done the trick.
 
from the source code of camcontrol you can see that times (-t) under 1200 secs get a granularity of 5 secs, 21mins is a special case and anything over gets a granularity of 30mins
maybe this will be useful some time (dont assume the time on cli is the exact time when the drive will go to sleep)
 
  • Thanks
Reactions: mtu
You must log in or register to reply here.
Back
Top