Bug 898913 - error when using vgscan and vgchange
error when using vgscan and vgchange
Status: RESOLVED FIXED
Classification: openSUSE
Product: openSUSE 13.1
Classification: openSUSE
Component: Basesystem
Final
x86-64 openSUSE 13.1
: P5 - None : Normal (vote)
: ---
Assigned To: Liuhua Wang
E-mail List
:
Depends on:
Blocks:
  Show dependency treegraph
 
Reported: 2014-09-29 07:02 UTC by Werner Flamme
Modified: 2015-03-09 08:05 UTC (History)
5 users (show)

See Also:
Found By: ---
Services Priority:
Business Priority:
Blocker: ---
Marketing QA Status: ---
IT Deployment: ---
bwiedemann: needinfo? (werner.flamme)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Werner Flamme 2014-09-29 07:02:27 UTC
Hi everyone,

today I can't access any volumes on lvm thanks to a new feature in libdevmapper. Or is it a bug?

# vgscan ; vgchange -ay
vgscan: relocation error: vgscan: symbol dm_make_percent, version Base not defined in file libdevmapper.so.1.03 with link time reference
vgchange: relocation error: vgchange: symbol dm_make_percent, version Base not defined in file libdevmapper.so.1.03 with link time reference

# locate libdevmapper.so.1.03
/lib/libdevmapper.so.1.03
/lib64/libdevmapper.so.1.03

# rpm -qf /lib64/libdevmapper.so.1.03
device-mapper-1.02.78-0.14.1.2.x86_64

# rpm -qi device-mapper
Name        : device-mapper
Version     : 1.02.78
Release     : 0.14.1.2
Architecture: x86_64
Install Date: Wed Nov  6 21:00:26 2013
Group       : System/Base
Size        : 3055554
License     : GPL-2.0+ and LGPL-2.1+
Signature   : RSA/SHA256, Sat Sep 28 07:17:49 2013, Key ID b88b2fd43dbdc284
Source RPM  : device-mapper-1.02.78-0.14.1.2.src.rpm
Build Date  : Sat Sep 28 07:17:02 2013
Build Host  : cloud106
Relocations : (not relocatable)
Packager    : http://bugs.opensuse.org
Vendor      : openSUSE
Summary     : Device Mapper Tools
Description :
Programs, libraries, and man pages for configuring and using the device
mapper.
                               
Authors:
--------
    Joe Thornber <thornber@sistina.com>
Distribution: openSUSE 13.1

Since the package has been built yesterday, I do not think that suddenly my disk has gone mad. I try to find an older version of the package to access my data again.

Regards, Werner
Comment 1 Werner Flamme 2014-09-29 07:11:53 UTC
# rpm -qi device-mapper
Name        : device-mapper
Version     : 1.02.90
Release     : 150.1
Architecture: x86_64
Install Date: Mon Sep 29 09:05:44 2014
Group       : System/Base
Size        : 1733869
License     : GPL-2.0 and LGPL-2.1
Signature   : DSA/SHA1, Thu Sep 25 13:33:05 2014, Key ID 2abfa143a0e46e11
Source RPM  : lvm2-2.02.111-150.1.src.rpm
Build Date  : Thu Sep 25 13:32:33 2014
Build Host  : cloud132
Relocations : (not relocatable)
Vendor      : obs://build.opensuse.org/systemsmanagement
URL         : http://www.sourceware.org/lvm2/
Summary     : Device Mapper Tools
Description :
Programs, libraries, and man pages for configuring and using the device
mapper.
Distribution: systemsmanagement / openSUSE_13.1

This package works, my data becomes accessible again, though some warnings are issued:

# vgscan ; vgchange -ay
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.
  connect() failed on local socket: No such file or directory
  Internal cluster locking initialisation failed.
  WARNING: Falling back to local file-based locking.
  Volume Groups with the clustered attribute will be inaccessible.

The following "fdisk -l" shows all of my 6 LVs.
Comment 2 Bernhard Wiedemann 2014-10-02 08:48:09 UTC
your device-mapper package is already a year old
> Build Date  : Sat Sep 28 07:17:02 2013

but I guess some other update could have broken it. Please check the end of
/var/log/zypp/history

candidates would be lvm2, glibc
Comment 3 Werner Flamme 2014-10-02 09:19:15 UTC
Bernhard,

I extracted the possibly relevant parts on /var/log/zypp/histroy into bugzilla_898913.txt (grep '2014-09' history > bugzilla_898913.txt) and grepped here. 

I found an update of glibc on 2014-09-11, this might not have caused the problems that appeared on 2014-09-29.

For lvm2, I had installs on 2014-09-12 (2.02.98-148.1 from systemsmanagement repo) and 2014-09-29 (2.02.111-150.1 from systemsmanagement repo). So the latter might be the cause of the problem.

It might be a good advice to go back to 2.02.98-0.28.18.1 from the Update repo then, I guess. OK, I did it, and it brought device-mapper-1.02.78-0.14.1.2 right along. Unfortunately, mkinitrd hangs, so I can't test it right now.

Regards, Werner
Comment 4 Liuhua Wang 2015-01-21 08:35:48 UTC
1.02.78(In reply to Werner Flamme from comment #1)
> # rpm -qi device-mapper
> Name        : device-mapper
> Version     : 1.02.90
> Release     : 150.1
> Architecture: x86_64
> Install Date: Mon Sep 29 09:05:44 2014
> Group       : System/Base
> Size        : 1733869
> License     : GPL-2.0 and LGPL-2.1
> Signature   : DSA/SHA1, Thu Sep 25 13:33:05 2014, Key ID 2abfa143a0e46e11
> Source RPM  : lvm2-2.02.111-150.1.src.rpm
> Build Date  : Thu Sep 25 13:32:33 2014
> Build Host  : cloud132
> Relocations : (not relocatable)
> Vendor      : obs://build.opensuse.org/systemsmanagement
> URL         : http://www.sourceware.org/lvm2/
> Summary     : Device Mapper Tools
> Description :
> Programs, libraries, and man pages for configuring and using the device
> mapper.
> Distribution: systemsmanagement / openSUSE_13.1
> 
> This package works, my data becomes accessible again, though some warnings
> are issued:
> 
> # vgscan ; vgchange -ay
>   connect() failed on local socket: No such file or directory
>   Internal cluster locking initialisation failed.
>   WARNING: Falling back to local file-based locking.
>   Volume Groups with the clustered attribute will be inaccessible.
>   connect() failed on local socket: No such file or directory
>   Internal cluster locking initialisation failed.
>   WARNING: Falling back to local file-based locking.
>   Volume Groups with the clustered attribute will be inaccessible.
> 
> The following "fdisk -l" shows all of my 6 LVs.

So, the problem that hangs is when using device-mapper-1.02.78-0.14.1.2.x86_64, , and 1.02.90 works well, except these warning messages, right?

The warning messages are not good, but they are harmless. It is because we were using locking_type=3 by default, which means clustered aware locking, and if you are not using clustered lvm, then will automatically fall back to file-based locking(same as locking_type=1).

In opensuse13.1:update and latest opensuse:factory, we have already change the default to locking_type=1. So there will be no such warning messages.
You can also edit /etc/lvm/lvm.conf to change locking_type to 1 and test.
Comment 5 Werner Flamme 2015-01-21 09:16:18 UTC
It has been quite a while since I opened this bug case, and in the meantime I'm using Tumbleweed on a new workstation :). And no, I do and did not use clustered disks at my workstation :) My servers run with SLES 11 SP3, I do not have that problem there. Yet ;).

# rpm -qi device-mapper
Name        : device-mapper
Version     : 1.02.92
Release     : 170.1
Architecture: x86_64
Install Date: Wed Jan 14 07:16:36 2015
[...]
Distribution: systemsmanagement / openSUSE_Factory

After connecting the USB disk:
# lang=C fdisk -l

Disk /dev/sda: 238.5 GiB, 256060514304 bytes, 500118192 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x00094dd6

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sda1            2048   4208639   4206592     2G 82 Linux swap / Solaris
/dev/sda2  *      4208640 171976703 167768064    80G 83 Linux
/dev/sda3       171976704 500117503 328140800 156.5G 83 Linux

# partprobe
# vgscan ; vgchange -ay
File descriptor 8 (pipe:[48944]) leaked on vgscan invocation. Parent PID 9788: bash
  WARNING: lvmetad is running but disabled. Restart lvmetad before enabling it!
  Reading all physical volumes.  This may take a while...
  Found volume group "Hauptgruppe" using metadata type lvm2
File descriptor 8 (pipe:[48944]) leaked on vgchange invocation. Parent PID 9788: bash
  WARNING: lvmetad is running but disabled. Restart lvmetad before enabling it!
  6 logical volume(s) in volume group "Hauptgruppe" now active

# rpm -qi lvm2
Name        : lvm2
Version     : 2.02.114
Release     : 170.1
Architecture: x86_64
Install Date: Mi 14 Jan 2015 07:16:36 CET
[...]
Distribution: systemsmanagement / openSUSE_Factory

The warning texts are gone, but the volume groups on an external disk (USB3) are still not recognized. In fact, I often need the partprobe command to have the system recognize the disk (typically /dev/sdb) at all. And I do not know what to do about the lvmetad warning.

# systemctl -a -o cat | grep lvm
dev-disk-by\x2did-lvm\[...]\x2dCZx4PI.device  loaded    active   plugged   ASMT1053
lvm2-activation-early.service  loaded    inactive dead
lvm2-activation-net.service    loaded    inactive dead
lvm2-activation.service        loaded    inactive dead
lvm2-lvmetad.service           loaded    active   running
lvm2-lvmetad.socket            loaded    active   running

I can 'systemctl restart' and 'systemctl enable' both lvm2-lvmetad thingies, and still the same thing next morning. Trying 'systemctl enable' for the inactive services results in 'Failed to execute operation: No such file or directory'.

But this may have another reason - however, all the time before I opened this item, I could connect the disk and the volumes were discovered immediately and I could mount them right away (all in /etc/fstab with noauto option). I still can't. Changing the packages to those from the distro repo did not bring back the desired functionality - in fact, I used them before I installed those packages that are shown above. The glibc package is glibc-2.20-2.1.x86_64, installed on Mon Nov 10 07:35:47 2014 (according to 'rpm -qi glibc') and comes from the 'openSUSE Factory' distribution repo.

Regards,
Werner
Comment 6 Liuhua Wang 2015-02-02 07:30:08 UTC
(In reply to Werner Flamme from comment #5)
> It has been quite a while since I opened this bug case, 
Sorry for late.

> # rpm -qi device-mapper
> Name        : device-mapper
> Version     : 1.02.92
> Release     : 170.1
> Architecture: x86_64
> Install Date: Wed Jan 14 07:16:36 2015
> [...]
> Distribution: systemsmanagement / openSUSE_Factory
> 
> After connecting the USB disk:
> # lang=C fdisk -l
> 
> Disk /dev/sda: 238.5 GiB, 256060514304 bytes, 500118192 sectors
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> Disklabel type: dos
> Disk identifier: 0x00094dd6
> 
> Device     Boot     Start       End   Sectors   Size Id Type
> /dev/sda1            2048   4208639   4206592     2G 82 Linux swap / Solaris
> /dev/sda2  *      4208640 171976703 167768064    80G 83 Linux
> /dev/sda3       171976704 500117503 328140800 156.5G 83 Linux
> 
> # partprobe
> # vgscan ; vgchange -ay
> File descriptor 8 (pipe:[48944]) leaked on vgscan invocation. Parent PID
> 9788: bash
>   WARNING: lvmetad is running but disabled. Restart lvmetad before enabling
> it!
The lvm2-lvmetad is a metadata cache daemon. You can also use traditional way to deal with metadata.
The warning messages are noisy but actually harmless. That is because you disabled lvmetad(use_lvmetad=0) but the lvmetad daemon is running. In this occasion lvmetad will not be used. Why the daemon is running, it is because lvm2 package updating will 'try-restart lvm2-lvmetad.socket', which will start lvm2-lvmetad.service, which is a feature of systemd. 
There are two ways to avoid the warning messages:
1. stop .service manually.
2. enable lvmetad by set use_lvmetad=1 in /etc/lvm/lvm.conf


> The warning texts are gone, but the volume groups on an external disk (USB3)
> are still not recognized. In fact, I often need the partprobe command to
> have the system recognize the disk (typically /dev/sdb) at all. And I do not
> know what to do about the lvmetad warning.
You mean if you not excute partprobe, lv cannot be activated?  Then please provide the logs:
1. lsblk
2. cat /etc/fstab
3. pvs
4. lvs
5. lvmdump

Thanks!
Comment 7 Werner Flamme 2015-02-27 09:19:37 UTC
(In reply to Liuhua Wang from comment #6)
> (In reply to Werner Flamme from comment #5)

> The lvm2-lvmetad is a metadata cache daemon. You can also use traditional
> way to deal with metadata.
> The warning messages are noisy but actually harmless. That is because you
> disabled lvmetad(use_lvmetad=0) but the lvmetad daemon is running. In this
> occasion lvmetad will not be used. Why the daemon is running, it is because
> lvm2 package updating will 'try-restart lvm2-lvmetad.socket', which will
> start lvm2-lvmetad.service, which is a feature of systemd. 
> There are two ways to avoid the warning messages:
> 1. stop .service manually.
> 2. enable lvmetad by set use_lvmetad=1 in /etc/lvm/lvm.conf

This did the trick! Stopping the service, editing the parameter and restarting the service brought the LVM groups to visibility :)

> > The warning texts are gone, but the volume groups on an external disk (USB3)
> > are still not recognized. In fact, I often need the partprobe command to
> > have the system recognize the disk (typically /dev/sdb) at all. And I do not
> > know what to do about the lvmetad warning.
> You mean if you not excute partprobe, lv cannot be activated?  

Yes, before I changed the parameter, fdisk sometimes ignored the disk completely, or the non-LVM partitions were recognized, but not the volumes (the volume group) in the LVM partition.

Since I set use_lvmetad=1, the old behaviour is back, and all volumes are displayed right after the USB disk is connected.

Thank you!

Regards, Werner
Comment 8 Liuhua Wang 2015-03-02 06:24:50 UTC
If I understand correctly, your problem is now solved, right?
Can we close this bug now?
Comment 9 Werner Flamme 2015-03-02 06:34:55 UTC
(In reply to Liuhua Wang from comment #8)

Yes, my problem ist solved, you can close the bug now.

Thank you for your help and patience :)

Regards,
Werner
Comment 10 Liuhua Wang 2015-03-02 08:28:17 UTC
close it.