Good day!
I have a system with 4x1Tb HDDs, I decided to mirror its partitions (like there) under a freshly installed FreeBSD 9.0-RC3. After I had done that I decided to benchmark read speed. Now I wonder, why read speed from four mirrored disks is not even close to multiplied of 4x?
Configuration:
[cmd=]gmirror status[/cmd]
[cmd=]gpart show[/cmd]
Benchmark:
So can anybody explain this behaviour?
I have a system with 4x1Tb HDDs, I decided to mirror its partitions (like there) under a freshly installed FreeBSD 9.0-RC3. After I had done that I decided to benchmark read speed. Now I wonder, why read speed from four mirrored disks is not even close to multiplied of 4x?
Configuration:
[cmd=]gmirror status[/cmd]
Code:
gingema# gmirror status
Name Status Components
mirror/gm-boot COMPLETE ada0p1 (ACTIVE)
ada1p1 (ACTIVE)
ada2p1 (ACTIVE)
ada3p1 (ACTIVE)
mirror/gm-rootfs COMPLETE ada0p2 (ACTIVE)
ada1p2 (ACTIVE)
ada2p2 (ACTIVE)
ada3p2 (ACTIVE)
mirror/gm-swap COMPLETE ada0p3 (ACTIVE)
ada1p3 (ACTIVE)
ada2p3 (ACTIVE)
ada3p3 (ACTIVE)
gingema#
Code:
=> 34 1953525101 ada0 GPT (931G)
34 128 1 freebsd-boot (64k)
162 83886080 2 freebsd-ufs (40G)
83886242 2097152 3 freebsd-swap (1.0G)
85983394 1867541741 - free - (890G)
=> 34 1953525101 ada1 GPT (931G)
34 128 1 freebsd-boot (64k)
162 83886080 2 freebsd-ufs (40G)
83886242 2097152 3 freebsd-swap (1.0G)
85983394 1867541741 - free - (890G)
=> 34 1953525101 ada2 GPT (931G)
34 128 1 freebsd-boot (64k)
162 83886080 2 freebsd-ufs (40G)
83886242 2097152 3 freebsd-swap (1.0G)
85983394 1867541741 - free - (890G)
=> 34 1953525101 ada3 GPT (931G)
34 128 1 freebsd-boot (64k)
162 83886080 2 freebsd-ufs (40G)
83886242 2097152 3 freebsd-swap (1.0G)
85983394 1867541741 - free - (890G)
Benchmark:
- Gmirror round-robin algorithm
# gmirror configure -b round-robin gm-rootfs
# dd if=/dev/mirror/gm-rootfs of=/dev/null bs=2M count=2000
Code:4194304000 bytes transferred in 31.368543 secs (133710514 bytes/sec)
# gstat -f 'ada.p2' -I 10s
Code:L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 1 258 257 32832 0.7 2 42 0.3 17.1| ada0p2 0 258 257 32832 0.7 2 42 0.3 17.8| ada1p2 0 258 257 32832 0.7 2 42 0.3 19.0| ada2p2 0 258 257 32832 1.1 2 42 0.5 27.1| ada3p2
- Gmirror split algorithm ( block size 2048 bytes)
# gmirror configure -b split -s 2048 gm-rootfs
# dd if=/dev/mirror/gm-rootfs of=/dev/null bs=2M count=2000
Code:4194304000 bytes transferred in 36.875156 secs (113743356 bytes/sec)
# gstat -f 'ada.p2' -I 10s
Code:L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 875 874 27961 0.5 1 38 0.3 41.1| ada0p2 1 875 874 27958 0.5 1 38 0.4 40.0| ada1p2 0 875 874 27961 0.5 1 38 0.6 42.7| ada2p2 0 875 874 27961 0.5 1 38 0.3 41.5| ada3p2
- Gmirror split algorithm (default block size 4096 bytes)
# gmirror configure -b split -s 4096 gm-rootfs
# dd if=/dev/mirror/gm-rootfs of=/dev/null bs=2M count=2000
Code:4194304000 bytes transferred in 36.932793 secs (113565849 bytes/sec)
# gstat -f 'ada.p2' -I 10s
Code:L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 886 886 28344 0.5 0 0 0.0 42.2| ada0p2 1 886 886 28341 0.5 0 0 0.0 40.8| ada1p2 0 886 886 28344 0.5 0 0 0.0 41.0| ada2p2 0 886 886 28344 0.5 0 0 0.0 43.5| ada3p2
- Gmirror split algorithm (block size 16384 bytes)
# gmirror configure -b split -s 16384 gm-rootfs
# dd if=/dev/mirror/gm-rootfs of=/dev/null bs=2M count=2000
Code:4194304000 bytes transferred in 36.476575 secs (114986235 bytes/sec)
# gstat -f 'ada.p2' -I 10s
Code:L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 888 888 28427 0.5 0 0 0.0 43.0| ada0p2 0 888 888 28427 0.5 0 0 0.0 41.5| ada1p2 0 888 888 28427 0.5 0 0 0.0 41.3| ada2p2 1 888 888 28427 0.5 0 0 0.0 41.2| ada3p2
- Gmirror load algorithm
# gmirror configure -b load gm-rootfs
# dd if=/dev/mirror/gm-rootfs of=/dev/null bs=2M count=2000
Code:4194304000 bytes transferred in 30.788696 secs (136228700 bytes/sec)
# gstat -f 'ada.p2' -I 10s
Code:L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 0 1 0 0 0.0 1 42 0.3 0.0| ada0p2 1 1058 1056 135175 0.8 1 42 0.3 87.1| ada1p2 0 1 0 0 0.0 1 42 0.3 0.0| ada2p2 0 1 0 0 0.0 1 42 0.3 0.0| ada3p2
So can anybody explain this behaviour?