FreeBSD Software RAID 1 issues

Understudy

Understudy

Hi,

I have a server with matching 400 Gig drives. I would like to do a software RAID 1. I am using FreeBSD 9.0

The handbook RAID 1 is not matching up with my system. So lets get some basics out of the way

Code: 
spider# uname -a
FreeBSD spider.example.com 9.0-RELEASE FreeBSD 9.0-RELEASE #0: 
Tue Jan  3 07:15:25 UTC 2012     root@obrian.cse.buffalo.edu:
/usr/obj/usr/src/sys/GENERIC  i386

I am going from the Handbook section 20.4.3.

Code: 
spider# gmirror load
spider#

No problem there.

Code: 
spider# diskinfo -v ada0 | head -n3
ada0
        512             # sectorsize
        400088457216    # mediasize in bytes (372G)

So that went fine.

Code: 
spider# geom zero load
spider# gnop create -s 400088457216 gzero
spider# gmirror label -v gm0 gzero.nop ada1
Metadata value stored on gzero.nop.
Metadata value stored on ada1.
Done.
spider# gmirror forget gm0

All good there. Now this next step is where things go awry.

Code: 
spider# gpart show ada0
=>       34  781422701  ada0  GPT  (372G)
         34        128     1  freebsd-boot  (64k)
        162  771751808     2  freebsd-ufs  (368G)
  771751970    8388608     3  freebsd-swap  (4.0G)
  780140578    1282157        - free -  (626M)

When creating the partition from the FreeBSD install disk. I went with the default setup and this how they now create the disk slices. All the directories are in section 2. That includes / /root/ /etc/ /usr/ etc.

Code: 
spider# gpart backup ada0 > table.ada0
spider# gpart backup ada0s1 > table.ada0s1
gpart: No such geom: ada0s1.

See I have no adaos1. Okay not an end of the world. I really don't need to create one. So I just drop the stuff about the ada0s1.

Code: 
spider# cat table.ada0
GPT 128
1   freebsd-boot        34       128  
2    freebsd-ufs       162 771751808  
3   freebsd-swap 771751970   8388608  
spider# gpart restore mirror/gm0 < table.ada0
spider# gpart show mirror/gm0
=>       34  781422700  mirror/gm0  GPT  (372G)
         34        128           1  freebsd-boot  (64k)
        162  771751808           2  freebsd-ufs  (368G)
  771751970    8388608           3  freebsd-swap  (4.0G)
  780140578    1282156              - free -  (626M)

So this part seems ok. I think. However now things have gone spiraling downhill.

Code: 
spider# newfs -U /dev/mirror/gm0
/dev/mirror/gm0: 381554.1MB (781422760 sectors) block size 32768, fragment size 4096
        using 516 cylinder groups of 740.00MB, 23680 blks, 47360 inodes.
        with soft updates
super-block backups (for fsck -b #) at:
 192, 1515712, 3031232, 4546752, 6062272, 7577792, 9093312, 10608832, 12124352,
 13639872, 15155392, 16670912, 18186432, 19701952, 21217472, 22732992,
 24248512, 25764032, 27279552, 28795072, 30310592, 31826112, 33341632,
 34857152, 36372672, 37888192, 39403712, 40919232, 42434752, 43950272,
 45465792, 46981312, 48496832, 50012352, 51527872, 53043392, 54558912,
 56074432, 57589952, 59105472, 60620992, 62136512, 63652032, 65167552,
 66683072, 68198592, 69714112, 71229632, 72745152, 74260672, 75776192,
 77291712, 78807232, 80322752, 81838272, 83353792, 84869312, 86384832,
 87900352, 89415872, 90931392, 92446912, 93962432, 95477952, 96993472,
 98508992, 100024512, 101540032, 103055552, 104571072, 106086592, 107602112,
 109117632, 110633152, 112148672, 113664192, 115179712, 116695232, 118210752,
 119726272, 121241792, 122757312, 124272832, 125788352, 127303872, 128819392,
 130334912, 131850432, 133365952, 134881472, 136396992, 137912512, 139428032,
 140943552, 142459072, 143974592, 145490112, 147005632, 148521152, 150036672,
 151552192, 153067712, 154583232, 156098752, 157614272, 159129792, 160645312,
 162160832, 163676352, 165191872, 166707392, 168222912, 169738432, 171253952,
 172769472, 174284992, 175800512, 177316032, 178831552, 180347072, 181862592,
 183378112, 184893632, 186409152, 187924672, 189440192, 190955712, 192471232,
 193986752, 195502272, 197017792, 198533312, 200048832, 201564352, 203079872,
 204595392, 206110912, 207626432, 209141952, 210657472, 212172992, 213688512,
 215204032, 216719552, 218235072, 219750592, 221266112, 222781632, 224297152,
 225812672, 227328192, 228843712, 230359232, 231874752, 233390272, 234905792,
 236421312, 237936832, 239452352, 240967872, 242483392, 243998912, 245514432,
 247029952, 248545472, 250060992, 251576512, 253092032, 254607552, 256123072,
 257638592, 259154112, 260669632, 262185152, 263700672, 265216192, 266731712,
 268247232, 269762752, 271278272, 272793792, 274309312, 275824832, 277340352,
 278855872, 280371392, 281886912, 283402432, 284917952, 286433472, 287948992,
 289464512, 290980032, 292495552, 294011072, 295526592, 297042112, 298557632,
 300073152, 301588672, 303104192, 304619712, 306135232, 307650752, 309166272,
 310681792, 312197312, 313712832, 315228352, 316743872, 318259392, 319774912,
 321290432, 322805952, 324321472, 325836992, 327352512, 328868032, 330383552,
 331899072, 333414592, 334930112, 336445632, 337961152, 339476672, 340992192,
 342507712, 344023232, 345538752, 347054272, 348569792, 350085312, 351600832,
 353116352, 354631872, 356147392, 357662912, 359178432, 360693952, 362209472,
 363724992, 365240512, 366756032, 368271552, 369787072, 371302592, 372818112,
 374333632, 375849152, 377364672, 378880192, 380395712, 381911232, 383426752,
 384942272, 386457792, 387973312, 389488832, 391004352, 392519872, 394035392,
 395550912, 397066432, 398581952, 400097472, 401612992, 403128512, 404644032,
 406159552, 407675072, 409190592, 410706112, 412221632, 413737152, 415252672,
 416768192, 418283712, 419799232, 421314752, 422830272, 424345792, 425861312,
 427376832, 428892352, 430407872, 431923392, 433438912, 434954432, 436469952,
 437985472, 439500992, 441016512, 442532032, 444047552, 445563072, 447078592,
 448594112, 450109632, 451625152, 453140672, 454656192, 456171712, 457687232,
 459202752, 460718272, 462233792, 463749312, 465264832, 466780352, 468295872,
 469811392, 471326912, 472842432, 474357952, 475873472, 477388992, 478904512,
 480420032, 481935552, 483451072, 484966592, 486482112, 487997632, 489513152,
 491028672, 492544192, 494059712, 495575232, 497090752, 498606272, 500121792,
 501637312, 503152832, 504668352, 506183872, 507699392, 509214912, 510730432,
 512245952, 513761472, 515276992, 516792512, 518308032, 519823552, 521339072,
 522854592, 524370112, 525885632, 527401152, 528916672, 530432192, 531947712,
 533463232, 534978752, 536494272, 538009792, 539525312, 541040832, 542556352,
 544071872, 545587392, 547102912, 548618432, 550133952, 551649472, 553164992,
 554680512, 556196032, 557711552, 559227072, 560742592, 562258112, 563773632,
 565289152, 566804672, 568320192, 569835712, 571351232, 572866752, 574382272,
 575897792, 577413312, 578928832, 580444352, 581959872, 583475392, 584990912,
 586506432, 588021952, 589537472, 591052992, 592568512, 594084032, 595599552,
 597115072, 598630592, 600146112, 601661632, 603177152, 604692672, 606208192,
 607723712, 609239232, 610754752, 612270272, 613785792, 615301312, 616816832,
 618332352, 619847872, 621363392, 622878912, 624394432, 625909952, 627425472,
 628940992, 630456512, 631972032, 633487552, 635003072, 636518592, 638034112,
 639549632, 641065152, 642580672, 644096192, 645611712, 647127232, 648642752,
 650158272, 651673792, 653189312, 654704832, 656220352, 657735872, 659251392,
 660766912, 662282432, 663797952, 665313472, 666828992, 668344512, 669860032,
 671375552, 672891072, 674406592, 675922112, 677437632, 678953152, 680468672,
 681984192, 683499712, 685015232, 686530752, 688046272, 689561792, 691077312,
 692592832, 694108352, 695623872, 697139392, 698654912, 700170432, 701685952,
 703201472, 704716992, 706232512, 707748032, 709263552, 710779072, 712294592,
 713810112, 715325632, 716841152, 718356672, 719872192, 721387712, 722903232,
 724418752, 725934272, 727449792, 728965312, 730480832, 731996352, 733511872,
 735027392, 736542912, 738058432, 739573952, 741089472, 742604992, 744120512,
 745636032, 747151552, 748667072, 750182592, 751698112, 753213632, 754729152,
 756244672, 757760192, 759275712, 760791232, 762306752, 763822272, 765337792,
 766853312, 768368832, 769884352, 771399872, 772915392, 774430912, 775946432,
 777461952, 778977472, 780492992

After the next result I stopped.

Code: 
spider# gpart bootcode -b /boot/mbr mirror/gm0
gpart: table 'mirror/gm0' is corrupt: Operation not permitted

So now I am stuck and not sure how to proceed. I can do a fresh install but if I do I would want to try and to a RAID 1 mirror with better instructions. Where do I go from here?

Sincerely,

Brendhan
 
Understudy said:
Code: 
spider# gpart show ada0
=>       34  781422701  ada0  GPT  (372G)
         34        128     1  freebsd-boot  (64k)
        162  771751808     2  freebsd-ufs  (368G)
  771751970    8388608     3  freebsd-swap  (4.0G)
  780140578    1282157        - free -  (626M)

When creating the partition from the FreeBSD install disk. I went with the default setup and this how they now create the disk slices. All the directories are in section 2. That includes / /root/ /etc/ /usr/ etc.
Click to expand...

Right. Your existing disk is GPT. Unfortunately, GPT does not coexist well with gmirror(8), and the official solution is to use MBR. So you cannot copy the existing format.


Code: 
spider# gpart bootcode -b /boot/mbr mirror/gm0
gpart: table 'mirror/gm0' is corrupt: Operation not permitted

So now I am stuck and not sure how to proceed. I can do a fresh install but if I do I would want to try and to a RAID 1 mirror with better instructions. Where do I go from here?
Click to expand...

The gmirror(8) metadata and the GPT metadata try to exist in the same place. One will be corrupted by the other. That is the reason to use MBR, which does not have metadata at the end of the disk.

Don't do a fresh install, it is not needed.

The first half of the Handbook mirror instructions show how to create a new mirror. Do that, but create a one-disk mirror with the new drive.

But before you do that, destroy the GPT format on the new disk. That's important, if you just create a new MBR, the old GPT metadata at the end of the disk will be a problem.

After the new mirror is created, use dump(8)/restore(8) to copy the old disk to it. Fix /etc/fstab, power down and disconnect the original drive, and boot from the mirror. If it works (and you have good backups), power down, reconnect the old drive, and add it to the mirror. If it doesn't work, fix the problems, likely one of the problems shown at the end of that section.
 
Hi,

Thanks for the reply.

I am going to ask for some clarification if I may.

So I want to clear the GPT of the second drive based on what you said.

Code: 
# gpart destroy -F ada1

then the next step would be

Code: 
# gmirror remove gm4 ada1

next step

Code: 
# gmirror clear ada1

So this should destroy the GPT. I will go with these steps and then see what happens. I have some more questions about the getting the mirror to work but I will take baby steps here.

Sincerely,

Brendhan
 
Well that was a not so much.

Code: 
spider# gpart destroy -F ada1
gpart: arg0 'ada1': Invalid argument

Here is some info from my dmesg before I started all this.

Code: 
ada0 at ata2 bus 0 scbus0 target 0 lun 0
ada0: <ST3400620AS 3.AAC> ATA-7 SATA 1.x device
ada0: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada0: 381554MB (781422768 512 byte sectors: 16H 63S/T 16383C)
ada0: Previously was known as ad4
ada1 at ata3 bus 0 scbus1 target 0 lun 0
ada1: <ST3400620AS 3.AAC> ATA-7 SATA 1.x device
ada1: 150.000MB/s transfers (SATA 1.x, UDMA5, PIO 8192bytes)
ada1: 381554MB (781422768 512 byte sectors: 16H 63S/T 16383C)
ada1: Previously was known as ad6

So what would be a good next step?

Sincerely,

Brendhan
 
Assuming ada1 is the new drive, that message is correct--when you try to destroy a GEOM that has already been destroyed, that's what it does.

Now, create a mirror whose only member is ada1.
 
I have been unable to accomplish the task. I have decided to start over.

If GPT is the issue and MBR solves it the new install method for 9.0 is not acceptable for doing a MBR install. I cannot get it to work.
 
You must log in or register to reply here.
Back
Top