gmirror performance.

fxp · Nov 17, 2008

Just a single disk:

Code:

diskinfo -ctv ad0
Transfer rates:
        outside:       102400 kbytes in   0.928828 sec =   110246 kbytes/sec
        middle:        102400 kbytes in   1.063225 sec =    96311 kbytes/sec
        inside:        102400 kbytes in   1.867590 sec =    54830 kbytes/sec

Mirror if 3 same disks:

Code:

diskinfo -ctv /dev/mirror/gm0
Transfer rates:
        outside:       102400 kbytes in   1.703981 sec =    60095 kbytes/sec
        middle:        102400 kbytes in   1.799062 sec =    56919 kbytes/sec
        inside:        102400 kbytes in   2.149057 sec =    47649 kbytes/sec

gstat:

Code:

L(q)  ops/s    r/s   kBps   ms/r    w/s   kBps   ms/w   %busy Name
    0    142    142  18038    2.1      0      0    0.0   29.7| ad0
    0      0      0      0    0.0      0      0    0.0    0.0| DEV/ad0/ad0
    0      0      0      0    0.0      0      0    0.0    0.0| acd0
    0      0      0      0    0.0      0      0    0.0    0.0| MIRROR/gm0/ad0
    0    425    425  54225    2.1      0      0    0.0   89.1| mirror/gm0
    0      0      0      0    0.0      0      0    0.0    0.0| DEV/acd0/acd0
    0      0      0      0    0.0      0      0    0.0    0.0| MBR/ad0/ad0
    0      0      0      0    0.0      0      0    0.0    0.0| ad0s1
    0      0      0      0    0.0      0      0    0.0    0.0| DEV/mirror/gm0/mirror/gm0
    0      0      0      0    0.0      0      0    0.0    0.0| VFS/ffs.mirror/gm0s1a/mirror/gm0s1a
    0      0      0      0    0.0      0      0    0.0    0.0| SWAP/swap/mirror/gm0s1b
    0      0      0      0    0.0      0      0    0.0    0.0| MBR/mirror/gm0/mirror/gm0
    0    425    425  54225    2.1      0      0    0.0   89.8| mirror/gm0s1
    0    142    142  18038    2.1      0      0    0.0   29.2| ad1
    0      0      0      0    0.0      0      0    0.0    0.0| DEV/ad0s1/ad0s1
    0      0      0      0    0.0      0      0    0.0    0.0| VFS/ffs.mirror/gm0s1d/mirror/gm0s1d
    0      0      0      0    0.0      0      0    0.0    0.0| MIRROR/gm0/ad1
    0    142    142  18148    2.0      0      0    0.0   28.8| ad2
    0      0      0      0    0.0      0      0    0.0    0.0| DEV/mirror/gm0s1/mirror/gm0s1
    0      0      0      0    0.0      0      0    0.0    0.0| BSD/mirror/gm0s1/mirror/gm0s1
    0    425    425  54225    2.1      0      0    0.0   90.8| mirror/gm0s1a

So, each disk used only for 30% and in the end we got 90%. This just isn`t right :\

gmirror list:

Code:

Geom name: gm0
State: COMPLETE
Components: 3
Balance: round-robin
Slice: 4096
Flags: NONE
GenID: 0
SyncID: 1
ID: 1642310077
Providers:
1. Name: mirror/gm0
   Mediasize: 250059349504 (233G)
   Sectorsize: 512
   Mode: r3w3e4
Consumers:
1. Name: ad0
   Mediasize: 250059350016 (233G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 938101692
2. Name: ad1
   Mediasize: 250059350016 (233G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 3435672881
3. Name: ad2
   Mediasize: 250059350016 (233G)
   Sectorsize: 512
   Mode: r1w1e1
   State: ACTIVE
   Priority: 0
   Flags: NONE
   GenID: 0
   SyncID: 1
   ID: 3884381126

SaveTheRbtz · Nov 17, 2008

did you tried something other than round-robin?

I've read something bad about load, but also there is patch for it http://www.freebsd.org/cgi/query-pr.cgi?pr=113885 you can try it.

And as i know round-robin is good for big files, i don't know how it works with lots of small files.

PS. almost forgot, there was some patch for sendfile to better cache handling and tunable readahead http://www.lexa.ru/nginx-ru/msg19497.html

gmirror performance.

fxp

SaveTheRbtz