Possible to determine disk order in raidz2?

littlesandra88

littlesandra88

Hi =)

I would like to create a raidz2 of 36 disks with 6 disks in each raidz2, where I determine which disks that should go in each raidz2.

If I do

Code: 
zpool create tank3 raidz2 da0 da1 da2 da3 da4 da5 \
                         raidz2 da6 da7 da8 da9 da10 da11 \
                         raidz2 da12 da13 da14 da15 da16 da17 \
                         raidz2 da18 da19 da20 da21 da22 da23 \
                         raidz2 da24 da25 da26 da27 da28 da29 \
                         raidz2 da30 da31 da32 da33 da34 da35

then I get that the first raidz2 is made from

# da0, da8, da4 ,da9, da2, da3

instead of

# da0, da1, da2, da3, da4, da5

as I would have expected.

Does anyone know how the order of each raidz2 can be forced?
 
Well that's interesting. Each vdev should be made up of the drives that directly follow the vdev type in the command. The order inside the vdev can't be controlled (i.e. a mirror may end up as da0,da1 or da1,da0 which doesn't really matter) but if it's mixing drives as you state, then it suggests there's a bug in the create command.

If all else fails, just create the pool with one raidz2 vdev using da0-5, then use 'zpool add' to add the other vdevs one by one. If it still ends up mixing them up then there's something really wrong somewhere.
 
Label the drives (or, create a GPT partition that's 4K-aligned and label that), then use the labels in the zpool create command.

Or, if you are bound and determined to use device nodes directly, do it in separate commands. Create the pool with 1 raidz vdev. Then "zpool add" another raidz vdev. And repeat until all vdevs are added.
 
Hello all,

And thanks a lot for all the solutions =)

I were able to reproduce it on two hosts, but after I

# zpool destroy tank3

and then the "zpool create tank3 raidz2 da0 da1 ..." I haven't been able to reproduce the problem.

Somehow the pool must have been corrupted, as afterwards

# zdb -d tank3

worked.

UPDATE

But I have now noticed that after a reboot, the disks are now randomized again in the pool,

# zdb -d tank3

fails again.
 
I am very temped to think that my hardware is corrupt.

Code: 
[root@nas3s ~]# zpool destroy tank3

[root@nas3s ~]# zpool status
no pools available

[root@nas3s ~]# zpool create tank3 raidz2 da{0,1,2,3,4,5}

Everything works as it should.

Code: 
[root@nas3s ~]# reboot

[root@nas3s ~]# zpool status
  pool: tank3
 state: ONLINE
 scan: none requested
config:

	NAME        STATE     READ WRITE CKSUM
	tank3       ONLINE       0     0     0
	  raidz2-0  ONLINE       0     0     0
	    da1     ONLINE       0     0     0
	    da4     ONLINE       0     0     0
	    da0     ONLINE       0     0     0
	    da8     ONLINE       0     0     0
	    da2     ONLINE       0     0     0
	    da5     ONLINE       0     0     0

errors: No known data errors

[root@nas3s ~]# zdb -d tank3
zdb: can't open 'tank3': Device not configured

The vdev now contains random devices.


My two NAS's which makes the exact same error on FreeBSD 9-p6 is

* Supermicro CSE-847E16-R1400LPB chassis, 36 HS bays, 1400W redundant PSU
* Supermicro H8DG6-F AMD Dual G34 mainboard
* AMD Opteron 6320, 2.8GHz 8-core, 8MB L2 cache, 6400MT

What kind of hardware failure can this be?
 
@cpu82

Is this a known bug? (Please let it be =) )

The reason I installed 9.0 and not 9.1 was because

# cd /usr/ports && make fetchindex

was broken, and the fix was to compile perl, which I think was a pretty serious bug to have in a new release. So I didn't dare to go 9.1.

If I do

# freebsd-update -r 9.1-RELEASE upgrade

would "make fetchindex" then work, or would I still have to recompile perl?
 
littlesandra88 said:
Is this a known bug? (Please let it be =) )

The reason I installed 9.0 and not 9.1 was because

# cd /usr/ports && make fetchindex

was broken, and the fix was to compile perl, which I think was a pretty serious bug to have in a new release. So I didn't dare to go 9.1.

would "make fetchindex" then work, or would I still have to recompile perl?
Click to expand...

If you use the default make index target then you need to install perl in order to build an INDEX from scratch. But if you want download a pre-built INDEX by make fetchindex, then no need perl. Indeed, you can build and install ports very happily without INDEX on your system.

littlesandra88 said:
If I do

# freebsd-update -r 9.1-RELEASE upgrade
Click to expand...
To upgrade the command to run is as follows:
# freebsd-update upgrade -r 9.1-RELEASE

To successfully update check out http://www.freebsd.org/releases/9.1R/relnotes-detailed.html#upgrade.
 
fetchindex should have worked, the problem was that the index was not being rebuilt. That was fixed a couple of weeks ago. But most FreeBSD systems, even servers, end up with Perl installed as a dependency of something else anyway.
 
wblock@ said:
fetchindex should have worked, the problem was that the index was not being rebuilt. That was fixed a couple of weeks ago. But most FreeBSD systems, even servers, end up with Perl installed as a dependency of something else anyway.
Click to expand...

wblock@

Thanks for the clarification ;)
 
@cpu82

Ahhh. Now it works =) Upgrading to 9.1 was really easy.

However when I wanted to upgrade the spare NAS, I did a really bad mistake, when I wanted to install "screen". I typed

# cd /usr/ports && make install name=screen

and it started to download and compile all kinds of things.

It ended with

Code: 
ultrix-gcc

Then type 'make <config>' (ex: 'make linux-x86')

Or, run './configure' then 'make'
See './configure --help' for details

(ignore the following error message)
gmake: *** [configs/current] Error 1
*** Error code 1

Stop in /usr/ports/graphics/dri.
*** Error code 1

Stop in /usr/ports/x11-servers/xorg-vfbserver.
*** Error code 1

Stop in /usr/ports/accessibility/accerciser.
*** Error code 1

Stop in /usr/ports/accessibility.
*** Error code 1

Stop in /usr/ports.
What just happened? Can it be reversed?

UPDATE

Complete output log

http://pastebin.com/cwr678bp
 
Where are those disks coming from ? Are they behind some FC adapter ? I highly doubt (I even bet money on it) pool won't get imported upon boot (and gets healthy) if some of the disks are missing.

Did you try to check serial # of those disks afterwards ? i.e. that with different names and order, same amount of the serial # (or WWNs) are in that pool? Built-in diskinfo -v can help, I like smartctl from sysutils/smartmontools though.
 
@cpu82

"pwd" when it stopped was "/usr/ports".

I have just updated the previous post with a complete output dump.

Code: 
Stop in /usr/ports/accessibility.
*** Error code 1

Stop in /usr/ports.
[root@nas3 /usr/ports]# pwd
/usr/ports
[root@nas3 /usr/ports]# ls   
.cvsignore	accessibility	editors		math		shells
CHANGES		arabic		emulators	misc		sysutils
COPYRIGHT	archivers	finance		multimedia	textproc
CVS		astro		french		net		ukrainian
GIDs		audio		ftp		net-im		vietnamese
INDEX-9		benchmarks	games		net-mgmt	www
KNOBS		biology		german		net-p2p		x11
LEGAL		cad		graphics	news		x11-clocks
MOVED		chinese		hebrew		palm		x11-drivers
Makefile	comms		hungarian	polish		x11-fm
Mk		converters	irc		ports-mgmt	x11-fonts
README		databases	japanese	portuguese	x11-servers
Templates	deskutils	java		print		x11-themes
Tools		devel		korean		russian		x11-toolkits
UIDs		distfiles	lang		science		x11-wm
UPDATING	dns		mail		security
[root@nas3 /usr/ports]#
 
@matoatlantis

You are absolutely right. No FC adapter =)

Going through all the serial numbers, they are all unique.

Upgrading to FreeBSD 9.1-p1 fixed the problem when I destroyed the pool, and created it again.
 
littlesandra88 said:
@matoatlantis
Going through all the serial numbers, they are all unique.
Click to expand...

Hm, so are they the same ? Same "pool" of serial numbers if you will. It might be they got discovered in different order upon boot. But as you're saying it got fixed in one of the prerelease, it might have been a bug then.
 
@cpu82

Here you go =)

Code: 
# New ports collection makefile for:    dri
# Date created:         8 Nov 2003
# Whom:                 anholt@FreeBSD.org
#
# $FreeBSD: ports/graphics/dri/Makefile,v 1.36 2011/02/25 16:52:06 miwi Exp $
#

PORTNAME=       dri
PORTVERSION=    ${MESAVERSION}
PORTEPOCH=      2
CATEGORIES=     graphics

COMMENT=        OpenGL hardware acceleration drivers for the DRI

LIB_DEPENDS=    drm:${PORTSDIR}/graphics/libdrm \
                expat.6:${PORTSDIR}/textproc/expat2
BUILD_DEPENDS=  makedepend:${PORTSDIR}/devel/makedepend

CONFLICTS=      dri-6.2.2005* dri-6.5.2006*
MAKE_JOBS_UNSAFE=       yes

USE_XORG=       glproto x11 xext xxf86vm xdamage xfixes dri2proto

EXTRA_PATCHES+= ${FILESDIR}/patch-mach64_context.h \
                ${FILESDIR}/patch-sis_context.h


do-install:
        cd ${WRKSRC}/src/mesa; ${GMAKE} install-dri

.include "${.CURDIR}/../../graphics/libGL/bsd.mesalib.mk"

.include <bsd.port.pre.mk>

.if ${ARCH} == "ia64"
BROKEN=         Does not install on ia64
.endif

.include <bsd.port.post.mk>
 
Try following (e.g. freebsd-dri if uses kernel FreeBSD) :
# cd /usr/ports/graphics/dri && make clean && [b]g[/b]make freebsd-dri && make install

Instead, freebsd-dri-x86 if uses x86-fbsd or freebsd-dri-amd64 if uses amd64-fbsd. Anyway, the version you want to install of graphics/dri is old.

Have updated the ports tree using portsnap(8), don't you?
# portsnap fetch update
 
@cpu82

Code: 
[root@nas3 /usr/ports]# cd /usr/ports/graphics/dri && make clean && make freebsd-dri && make install
===>  Cleaning for makedepend-1.0.3,1
===>  Cleaning for gmake-3.82
===>  Cleaning for glproto-1.4.12
===>  Cleaning for dri2proto-2.3
===>  Cleaning for libXxf86vm-1.1.1
===>  Cleaning for libdrm-2.4.12_1
===>  Cleaning for xf86vidmodeproto-2.3.1
===>  Cleaning for dri-7.4.4,2
make: don't know how to make freebsd-dri. Stop
[root@nas3 /usr/ports/graphics/dri]#

I don't really know what I want =) I am still on FreeBSD 9.0 on this one. I didn't dare to upgrade to 9.1 before damage control had been done on my bad typo =)

Or wouldn't you worry about it, and I should just upgrade to 9.1?

UPDATE

I did

Code: 
portsnap fetch update
portsnap extract

which I probably shouldn't have done I suppose. I see Quake among many other things now being extracted.

Can it be saved, or would it be better to reinstall 9.1 from scratch?
 
Is preferible upgrade to 9.1-RELEASE, but using old-style can cause error like these. Why don't using portmaster(8) to update your ports? Is easy and safe.

EDIT

Be save install from previous 9-RELEASE-p6, is the best way to learn: correct on the fly :e
 
@cpu82

Ok, I'll reinstall with 9.1 then. I guess I have trashed it too much to be used in production by now =)

"portmaster" looks really nice. I didn't knew about it, as I pretty much just read the FreeBSD Handbook, and I don't recall seeing it there.

From now on I will use that =)
 
littlesandra88 said:
@cpu82

Here you go =)

Code: 
# New ports collection makefile for:    dri
# Date created:         8 Nov 2003
# Whom:                 anholt@FreeBSD.org
#
# $FreeBSD: ports/graphics/dri/Makefile,v 1.36 2011/02/25 16:52:06 miwi Exp $
#

PORTNAME=       dri
PORTVERSION=    ${MESAVERSION}
PORTEPOCH=      2
CATEGORIES=     graphics

COMMENT=        OpenGL hardware acceleration drivers for the DRI

LIB_DEPENDS=    drm:${PORTSDIR}/graphics/libdrm \
                expat.6:${PORTSDIR}/textproc/expat2
BUILD_DEPENDS=  makedepend:${PORTSDIR}/devel/makedepend

CONFLICTS=      dri-6.2.2005* dri-6.5.2006*
MAKE_JOBS_UNSAFE=       yes

USE_XORG=       glproto x11 xext xxf86vm xdamage xfixes dri2proto

EXTRA_PATCHES+= ${FILESDIR}/patch-mach64_context.h \
                ${FILESDIR}/patch-sis_context.h


do-install:
        cd ${WRKSRC}/src/mesa; ${GMAKE} install-dri

.include "${.CURDIR}/../../graphics/libGL/bsd.mesalib.mk"

.include <bsd.port.pre.mk>

.if ${ARCH} == "ia64"
BROKEN=         Does not install on ia64
.endif

.include <bsd.port.post.mk>
Click to expand...

It doesn't build, apparently because it's using "default" as a make target when it needs to be something else from a list it gives, in other words, fails to detect ARCH because is not defined in Makefile.

It is appropriate for this case, open a "new thread". I will be happy to help with whatever arises, even if it is necessary, starting from zero up the end :p
 
You must log in or register to reply here.
Back
Top