ZFS on linux upgrade from 0.6.2 to 0.6.3 made my zpool unreadable help translate CentOS to Ubuntu commands

Question

After upgrading my ubuntu 13.10 server using sudo apt-get upgrade and rebooting, my zpool wouldn't show.

messier@ANDERSON:~$ sudo zpool status
no pools available
messier@ANDERSON:~$ sudo zpool import
zpool: ../../lib/libzfs/libzfs_import.c:356: Assertion `nvlist_lookup_uint64(zhp->zpool_config, ZPOOL_CONFIG_POOL_GUID, &theguid) == 0' failed.
messier@ANDERSON:~$

I looked on the zfs on linux github and I found a fix for CentOS which is to remove the old kernel modules and then reinstall the software to get the new kernel modules.

$ find /lib/modules/$(uname -r)/extra -name "splat.ko" -or -name "zcommon.ko" -or -name "zpios.ko" -or -name "spl.ko" -or -name "zavl.ko" -or -name "zfs.ko" -or -name "znvpair.ko" -or -name "zunicode.ko" | xargs rm -f
$ find /lib/modules/$(uname -r)/weak-updates -name "splat.ko" -or -name "zcommon.ko" -or -name "zpios.ko" -or -name "spl.ko" -or -name "zavl.ko" -or -name "zfs.ko" -or -name "znvpair.ko" -or -name "zunicode.ko" | xargs rm -f
$ yum reinstall zfs-release
$ yum reinstall $(rpm -qa | egrep "zfs|spl")

People are saying this is fixing it. I just don't know how to make this fix run on my ubuntu server. Thank you for the help!

Last edit : If you come onto this page for the zfs update bug. The easiest solution is like Scott Simon said.

# apt-get update
# apt-get dist-upgrade
# reboot

I'm sorry, I meaant on the Zfs on linux github, I also added a link to the issue stating the fix. Thank you. — Reissem, Jun 19 '14 at 02:05
Possible duplicate of http://serverfault.com/q/606215/195746 — LiveWireBT, Jun 19 '14 at 14:55
It may be duplicate on the error, but since the OS is different, I thought the solution would be different. — Reissem, Jun 19 '14 at 16:41
Had this problem after `do-release-upgrade`. The solution worked for me but only after I did : `apt-add-repository ppa:zfs-native/stable` — Jocelyn, Nov 07 '14 at 07:26

84104 · Accepted Answer · 2014-06-19T18:18:19.880

$ sudo find /lib/modules/$(uname -r)/updates -name "splat.ko" -or -name "zcommon.ko" -or -name "zpios.ko" -or -name "spl.ko" -or -name "zavl.ko" -or -name "zfs.ko" -or -name "znvpair.ko" -or -name "zunicode.ko" | xargs rm -f
$ sudo apt-get install --reinstall ubuntu-zfs
$ sudo apt-get install --reinstall $(dpkg-query --show --showformat='${binary:Package}\n' | egrep 'spl|zfs')

Ubuntu keeps all its dkms in /lib/modules/$(uname -r)/updates, rather than in /lib/modules/$(uname -r)/extra and find /lib/modules/$(uname -r)/weak-updates as CentOS does.

yum resinstall is equivalent to apt-get install --reinstall.
Ubuntu's zfs metapackage is ubuntu-zfs instead of zfs-release as it is in CentOS.
rpm -qa is equivalent to dpkg-query --show --showformat='${binary:Package}\n'

I'm also uncertain if the find command actually needs to be run, as apt-get may take care of that already, but running it won't break the process.

Someone with more knowledge of or experience with dpkg may be able to write a marginally more efficient command, but I suspect this is good enough for most purposes.

Thank you for this! I've worked out my solution from your post! If you are curious I posted it as an answer. — Reissem, Jun 20 '14 at 00:30

score 3 · Answer 2 · answered Jul 15 '14 at 03:32

3

The official bug page has this fix for Ubuntu and Debian users:

# apt-get update
# apt-get dist-upgrade
# reboot

answered Jul 15 '14 at 03:32

Scott Simon

31
1

Had this problem after `do-release-upgrade`. This worked for me only after I did : `apt-add-repository ppa:zfs-native/stable` – Jocelyn Nov 07 '14 at 07:24
distr-upgrade is the solution: `REMOVED: libzfs1 libzpool1` `installed: libzfs2 libzpool2` – basic6 Apr 30 '15 at 19:09

score 2 · Answer 3 · answered Jul 08 '14 at 21:09

I have debian-zfs installed on proxmox 3.2 (using this kernel pve-headers-2.6.32-26-pve). This is how I fixed it after the upgrade.

aptitude purge dkms debian-zfs spl-dkms zfs-dkms

I chose "Yes" to all the prompts, to remove it all. After that was done, I did some checking if there are any dkms leftovers. The below command should show an empty directory.

ls /lib/modules/2.6.32-26-pve/updates/dkms/

If you see anything in there (the above directory), you need to delete it all (unless you have other custom kernel modules).

Then it was as simple as:

apt-get install debian-zfs

Then import my pool (my pool is called tank):

zpool import tank

zfs got upset because it was not properly exported before all the drama happened.

cannot import 'tank': pool may be in use on another system

That's not a problem, I tried again with this:

zpool import -f tank

The above worked, and I was able to see my pool with this command:

zpool status
    root@proxmox-01:~# zpool status
  pool: tank
 state: ONLINE
  scan: scrub repaired 0 in 0h8m with 0 errors on Tue Jul  8 20:39:13 2014
config:

        NAME                                           STATE     READ WRITE CKSUM
        tank                                           ONLINE       0     0     0
          raidz2-0                                     ONLINE       0     0     0
            scsi-SATA_WDC_WD1001FALS-_WD-WMATV4198253  ONLINE       0     0     0
            scsi-SATA_WDC_WD1002FAEX-_WD-WCAW30398837  ONLINE       0     0     0
            scsi-SATA_WDC_WD1002FAEX-_WD-WCAW30404054  ONLINE       0     0     0
            scsi-SATA_WDC_WD1002FAEX-_WD-WCAW30406039  ONLINE       0     0     0

errors: No known data errors

I was happy with this. But proxmox was a little bit more difficult. I had to mess about with the mount point for proxmox, because it automatically creates the directory structure of "storage" that you add to proxmox.

First I check if /tank was not lovely zfs pool, using du -hs /tank. My pool has a lot of stuff in it, a lot more than 12KB.

I export (disconnect) my zfs pool zpool export tank

I removed the EMPTY /tank directory with rm -R /tank (please double check!!)

Then I import my pool (connect) zpool import tank and your zfs storage should be working, and accessible to proxmox.

This is how I got my zfs to work on proxmox in the first place:

http://www.anotherlan.com/proxmox-zfs-on-linux-local-storage

upgrading my zfs on linux from 0.6.3 to 0.6.4 was painless, do not forget to upgrade your zpool: `zpool upgrade tank` — Egidijus, Apr 23 '15 at 16:29

score 0 · Answer 4 · answered Jun 20 '14 at 00:28

User 84104's solution "worked". I did the first line of it. Then the rest didn't quite work. So I tried sudo apt-get remove ubuntu-zfs dkms and then sudo apt-get install ubuntu-zfs dkms and the same error occured. Dkms did not recognized that the zfs/0.6.3 and spl/0.6.3 was different from the one that comes with the 3.11.0-23-generic kernel. So I went and used sudo dkms remove zfs/0.6.3 --all and sudo dkms remove zfs/0.6.3 --all and then I forced the install of both modules using sudo dkms install zfs/0.6.3 --force and sudo dkms install spl/0.6.3 --force

Basically the solution is to use dkms to remove the modules and then force back the install of 0.6.3.

ZFS on linux upgrade from 0.6.2 to 0.6.3 made my zpool unreadable help translate CentOS to Ubuntu commands

4 Answers4