ZFS on Linux: Changing device IDs

When I created my zpool on FreeBSD, I went to great lengths to use volume names that matched the serial numbers on the disks. I then made sure the last 7 digits of the serial numbers were visible while the disks were in-situ, so that if one failed I could immediately tell which. This all went out the damn window when I moved the pool to Linux, because ZFS-on-Linux imported them as standard Linux disks (/dev/sd[bcd]).

That’s a pain in the arse, because the ZFS disks are in the higher numbered drive slots, so if I add or remove an extra disk and reboot the machine, Linux re-letters them, and ZFS needs help to figure it out as it seems to give up if nothing matches what’s in the cache file. The cache file, by the way, is binary in part, so editing it by hand didn’t seem like much fun. This is all not to mention that my original intention of knowing exactly which disk to pull was still not working.

For work today, we ran into a situation where we had unusually named disks on one machine, and I figured I’d test it at home, scratching my itch at the same time. What follows is a log of that.

Here’s the situation:

root@ghast:~# zpool status
  pool: zeefus
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
	still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
	the pool may no longer be accessible by software that does not support
	the features. See zpool-features(5) for details.
  scan: scrub repaired 0B in 9h34m with 0 errors on Sun Oct 13 09:58:47 2019
config:

	NAME        STATE     READ WRITE CKSUM
	zeefus      ONLINE       0     0     0
	  raidz1-0  ONLINE       0     0     0
	    sdc     ONLINE       0     0     0
	    sdd     ONLINE       0     0     0
	    sdb     ONLINE       0     0     0

errors: No known data errors

So we stop LXD and export the pool to offline it so that we can work on it:

root@ghast:~# snap stop lxd
Stopped.
root@ghast:~# zpool export zeefus

Now to work out which devices I want to use: the ones in /dev/disk/by-id/ look useful enough:

root@ghast:~# ls /dev/disk/by-id/
ata-FUJITSU_MHZ2080BH_G1_K63RT8732CD3           ata-WDC_WD40EFRX-68N32N0_WD-WCC7K5PDYKAL-part3  ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6VCNC9V-part1  wwn-0x50014ee210055975-part1  wwn-0x50014ee2bab02242
ata-FUJITSU_MHZ2080BH_G1_K63RT8732CD3-part1     ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6STX1EZ        ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6VCNC9V-part9  wwn-0x50014ee210055975-part2  wwn-0x50014ee2bab02242-part1
ata-FUJITSU_MHZ2080BH_G1_K63RT8732CD3-part2     ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6STX1EZ-part1  wwn-0x500000e04258de81                          wwn-0x50014ee210055975-part3  wwn-0x50014ee2bab02242-part2
ata-WDC_WD40EFRX-68N32N0_WD-WCC7K5PDYKAL        ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6STX1EZ-part2  wwn-0x500000e04258de81-part1                    wwn-0x50014ee26596e35a        wwn-0x50014ee2bab02242-part3
ata-WDC_WD40EFRX-68N32N0_WD-WCC7K5PDYKAL-part1  ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6STX1EZ-part3  wwn-0x500000e04258de81-part2                    wwn-0x50014ee26596e35a-part1
ata-WDC_WD40EFRX-68N32N0_WD-WCC7K5PDYKAL-part2  ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6VCNC9V        wwn-0x50014ee210055975                          wwn-0x50014ee26596e35a-part9
root@ghast:~# zpool import -d /dev/disk/by-id 
   pool: zeefus
     id: 7993881214442468791
  state: ONLINE
 status: Some supported features are not enabled on the pool.
 action: The pool can be imported using its name or numeric identifier, though
	some features will not be available without an explicit 'zpool upgrade'.
 config:

	zeefus                      ONLINE
	  raidz1-0                  ONLINE
	    wwn-0x50014ee26596e35a  ONLINE
	    wwn-0x50014ee210055975  ONLINE
	    wwn-0x50014ee2bab02242  ONLINE

Aww, fuck - it imported by the wrong ones. Those are predictable, sure (at least, to some extent), but they don’t scratch my first itch. After several attempts at working out how to coax ZFS to use the other ones, I finally just rage-quit and removed the offending symlinks temporarily:

root@ghast:~# zpool export zeefus
root@ghast:/dev# cd disk/by-id
root@ghast:/dev/disk/by-id# mkdir /root/disks
root@ghast:/dev/disk/by-id# mv wwn-0x50014ee210055975* /root/disks/
root@ghast:/dev/disk/by-id# mv wwn-0x50014ee26596e35a* /root/disks/
root@ghast:/dev/disk/by-id# mv wwn-0x50014ee2bab02242* /root/disks/
root@ghast:/dev/disk/by-id# zpool import -d /dev/disk/by-id -a
cannot mount '/zeefus': directory is not empty
root@ghast:/dev/disk/by-id# zpool status
  pool: zeefus
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
	still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
	the pool may no longer be accessible by software that does not support
	the features. See zpool-features(5) for details.
  scan: scrub canceled on Thu Oct 31 03:01:42 2019
config:

	NAME                                          STATE     READ WRITE CKSUM
	zeefus                                        ONLINE       0     0     0
	  raidz1-0                                    ONLINE       0     0     0
	    ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6VCNC9V  ONLINE       0     0     0
	    ata-WDC_WD40EFRX-68N32N0_WD-WCC7K5PDYKAL  ONLINE       0     0     0
	    ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6STX1EZ  ONLINE       0     0     0

errors: No known data errors
root@ghast:/dev/disk/by-id# mv /root/disks/* ./

A quick reboot to make sure it comes back:

root@ghast:/dev/disk/by-id# shutdown -r now
...
fwaggle@ghast:~$ zpool status
  pool: zeefus
 state: ONLINE
status: Some supported features are not enabled on the pool. The pool can
	still be used, but some features are unavailable.
action: Enable all features using 'zpool upgrade'. Once this is done,
	the pool may no longer be accessible by software that does not support
	the features. See zpool-features(5) for details.
  scan: scrub canceled on Thu Oct 31 03:01:42 2019
config:

	NAME                                          STATE     READ WRITE CKSUM
	zeefus                                        ONLINE       0     0     0
	  raidz1-0                                    ONLINE       0     0     0
	    ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6VCNC9V  ONLINE       0     0     0
	    ata-WDC_WD40EFRX-68N32N0_WD-WCC7K5PDYKAL  ONLINE       0     0     0
	    ata-WDC_WD40EFRX-68N32N0_WD-WCC7K6STX1EZ  ONLINE       0     0     0

errors: No known data errors

Hopefully I haven’t broken anything long-term!

fwaggle

Published:


Modified:


Filed under:


Location:


Navigation: Older Entry