ZFS Can not boot any more all block copies unavailable


I can't boot my FreeBSD any more with the error.
ZFS: i/o error - all block copies unavailable

This appears twice during boot an then nothing else happens. After a while numlock stops responding and I have to hard reset.

I googled a lot and I think I need to fix my bootcode but I can't figure out exactly how. I think I may have set it up incorrectly but I am hoping it is still fixable without starting over.

I did the following.
Installed FreeBSD 10.3 using ZFS with one vdev using two disks in a mirror configuration. This created three partitions on each disk with the zpool being allocated the third. I then added two additional disks as another vdev immediately after installation. I however did not add any partitions and just added them with

zpool add mypool mirror ada2 ada3

This worked fine and I had another disk in the pool but I think this is were I might have gone wrong.
I then tried to add the bootcode to the new disks like described in https://www.freebsd.org/doc/handbook/zfs-zpool.html but that did not work. The system booted fine however so I guessed the docs might be outdated and zpool would take care of this now but now I think that was wrong.
No partitions were created on these new disks but I could copy data over fine and zpool list -v shows them used.

After a while I added two more disks as another vdev using the same method.

Everything worked for some time with multiple reboots.
Today my system does not boot any more. I replaced the if_re.ko file in /boot/kernel because I was having issues with the RealTek onboard NIC. I guess this might have been written to one of the disks without bootcode having triggered my problems. I booted using a rescue USB and the zpool can still be imported and all the data appears to be there.

gpart list shows the two disks I initially created the zpool without but none of the other ones.
I can't use gpart bootcode for the other disks because there are no partitions so the -p option does not work.

Is there anything I can do to remedy this situation? Preferably without risking the data already in the pool because I do not have enough spare disks to backup everything unless I degrade the pool mirrors to singles and use those.

I think one solution might be to use another disk independent of the pool and install FreeBSD there and and import the zpool instead of booting from it.
Last edited by a moderator: