Bug 1372272

Summary: lvmconf passing with "Failed to execute operation" systemd error
Product: Red Hat Enterprise Linux 7 Reporter: Roman Bednář <rbednar>
Component: lvm2Assignee: LVM and device-mapper development team <lvm-team>
lvm2 sub component: Default / Unclassified QA Contact: cluster-qe <cluster-qe>
Status: CLOSED ERRATA Docs Contact:
Severity: unspecified    
Priority: unspecified CC: agk, ccaulfie, cluster-maint, cmarthal, heinzm, jbrassow, msnitzer, prajnoha, prockai, rbednar, thornber, zkabelac
Version: 7.3   
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: lvm2-2.02.170-2.el7 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-01 21:47:18 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1385242    

Description Roman Bednář 2016-09-01 09:58:19 UTC
Description of problem:
lvmconf --enable-cluster/--disable-cluster gives errors even though it seems to work correctly. 
Does the same with --disable-cluster option.


How reproducible:
always

Steps to Reproduce:
[root@virt-249 ~]# systemctl is-active lvm2-lvmetad
active

[root@virt-249 ~]# lvmconfig | grep locking_type
	locking_type=1

[root@virt-249 ~]# lvmconf --enable-cluster --services --startstopservices
Warning: Stopping lvm2-lvmetad.service, but it can still be activated by:
  lvm2-lvmetad.socket
Removed symlink /etc/systemd/system/sysinit.target.wants/lvm2-lvmetad.socket.
Unit lvm2-cluster-activation.service could not be found.
Failed to execute operation: Access denied
Failed to start lvm2-cluster-activation.service: Unit not found.

[root@virt-249 ~]# echo $?
0

[root@virt-249 ~]# systemctl is-active lvm2-lvmetad
inactive

[root@virt-249 ~]# lvmconfig | grep locking_type
	locking_type=3


Actual results:
lvmconf throws errors

Expected results:
lvmconf should complete, without the last 3 error lines in 'lvmconf' output shown above

=====================
3.10.0-496.el7.x86_64

lvm2-2.02.164-4.el7    BUILT: Wed Aug 31 15:47:09 CEST 2016
lvm2-libs-2.02.164-4.el7    BUILT: Wed Aug 31 15:47:09 CEST 2016
lvm2-cluster-2.02.164-4.el7    BUILT: Wed Aug 31 15:47:09 CEST 2016
device-mapper-1.02.133-4.el7    BUILT: Wed Aug 31 15:47:09 CEST 2016
device-mapper-libs-1.02.133-4.el7    BUILT: Wed Aug 31 15:47:09 CEST 2016
device-mapper-event-1.02.133-4.el7    BUILT: Wed Aug 31 15:47:09 CEST 2016
device-mapper-event-libs-1.02.133-4.el7    BUILT: Wed Aug 31 15:47:09 CEST 2016
device-mapper-persistent-data-0.6.3-1.el7    BUILT: Fri Jul 22 12:29:13 CEST 2016
cmirror-2.02.164-4.el7    BUILT: Wed Aug 31 15:47:09 CEST 2016

Comment 1 Zdenek Kabelac 2016-09-01 10:05:31 UTC
These are 'outputs' of systemctl commands.

I'm affaid that lvm2 could only through the output to /dev/null  or let it pass.

For now I'm convinced showing it is better option here.

At least user knows he has something wrong in systemd world.

Operation 'success' result code is suggesting lvm2 has been properly modified.

Reported systemd errors are unrelated to this (though still worth to check and fix if you are admin of that box...)

Comment 2 Zdenek Kabelac 2016-09-01 10:09:02 UTC
Maybe  we could 'prefix'  these 'external' command traces with some headings.

i.e. output of systemctl would be pipe though 'awk' so each such line would start with 'systemctl: '  prefix, to make it more clear.....

Comment 4 Roman Bednář 2016-09-01 12:07:34 UTC
Update:

systemd causing the errors seems to be dependent on following packages:

corosynclib-devel.x86_64
lvm2-cluster-standalone.x86_64

Errors are gone after installing those. 

Anyway, dependencies on devel package does not seem ok. It might be a corosync issue.

Comment 5 Peter Rajnoha 2016-09-01 12:57:22 UTC
For RHEL7, officially, we depend completely on pacemaker to set up the clvmd/cmirrord to set up services. The lvm2-cluster-standalone is only for testing in RHEL7 (not for production).

Comment 6 Peter Rajnoha 2016-09-01 12:59:53 UTC
...which means that in RHEL7, we don't use "--service" with lvmconf. And, I don't remember now exactly, but I think that pacemaker uses lvmconf itself to have lvm.conf correctly set. So, in RHEL7, the lvmconf should not be even used normally by end users, all should go through pacemaker which sets up things for lvm2 to work in cluster.

Comment 7 Peter Rajnoha 2016-09-01 13:04:16 UTC
We should add a note in man lvmconf about this so it's more clear...

Comment 8 Jan Friesse 2016-09-01 13:16:06 UTC
This is NOTHING to do with corosync. If lvm2-cluster-standalone needs corosynclib-devel for some weird reason, it's something what should be fixed in lvm2-cluster-standalone package. If lvm2-lvmetad.socket needs lvm2-cluster-standalone (it looks so) it should be fixed in lvm2-lvmetad.socket service.

Comment 11 Peter Rajnoha 2017-04-04 10:02:26 UTC
(In reply to Roman Bednář from comment #0)
> [root@virt-249 ~]# lvmconf --enable-cluster --services --startstopservices
> Warning: Stopping lvm2-lvmetad.service, but it can still be activated by:
>   lvm2-lvmetad.socket
> Removed symlink /etc/systemd/system/sysinit.target.wants/lvm2-lvmetad.socket.
> Unit lvm2-cluster-activation.service could not be found.
> Failed to execute operation: Access denied
> Failed to start lvm2-cluster-activation.service: Unit not found.
> 

Well, I've double-checked and actually, we already do check for "loaded" state of lvm2-cluster-activation.service systemd unit. And if it's not loaded (also meaning not installed), then we do skip any handling of this service.

Can you still reproduce? If so, can you add "set -x" at the very beginning of /usr/sbin/lvmconf script so we I can see the debug output for this part exactly:

  if [ "$type" = "systemd" ]; then
        if [ "$action" = "activate" ]; then
            for i in $@; do
                eval $($SYSTEMCTL_BIN show $i -p LoadState)
                test  "$LoadState" = "loaded" || continue
                $SYSTEMCTL_BIN enable $i
                if [ "$START_STOP_SERVICES" = "1" ]; then
                    $SYSTEMCTL_BIN start $i
                fi


The "SYSTEMCTL_BIN enable" and "SYSTEMCTL_BIN start" should be skipped, hence we shouldn't even get into a situation where "Unit not found" is issued.

Comment 16 Roman Bednář 2017-05-18 09:00:54 UTC
Verified. lvmconf now skips lvm2-cluster-activation.service if it's not loaded/installed.


# rpm -q lvm2-cluster-standalone-2.02.171-1.el7.x86_64
package lvm2-cluster-standalone-2.02.171-1.el7.x86_64 is not installed

# lvmconfig | grep locking_type
	locking_type=1

# systemctl is-active lvm2-lvmetad
active

# lvmconf --enable-cluster --services --startstopservices
Warning: Stopping lvm2-lvmetad.service, but it can still be activated by:
  lvm2-lvmetad.socket
Removed symlink /etc/systemd/system/sysinit.target.wants/lvm2-lvmetad.socket.

# lvmconfig | grep locking_type
	locking_type=3

# systemctl is-active lvm2-lvmetad
inactive



###With lvm2-cluster-standalone testing package installed:

# rpm -q lvm2-cluster-standalone-2.02.171-1.el7.x86_64
lvm2-cluster-standalone-2.02.171-1.el7.x86_64

# ps -ef | grep lvm
root      3622     1  0 10:52 ?        00:00:00 /usr/sbin/lvmetad -f
root      4125 31994  0 10:53 pts/0    00:00:00 grep --color=auto lvm

# lvmconf --enable-cluster --services --startstopservices
Warning: Stopping lvm2-lvmetad.service, but it can still be activated by:
  lvm2-lvmetad.socket
Removed symlink /etc/systemd/system/sysinit.target.wants/lvm2-lvmetad.socket.
Created symlink from /etc/systemd/system/multi-user.target.wants/lvm2-cluster-activation.service to /usr/lib/systemd/system/lvm2-cluster-activation.service.

# ps -ef | grep lvm
root      4267     1  0 10:53 ?        00:00:00 /usr/sbin/clvmd -T30
root      4286 31994  0 10:53 pts/0    00:00:00 grep --color=auto lvm

# lvmconf --disable-cluster --services --startstopservices
Created symlink from /etc/systemd/system/sysinit.target.wants/lvm2-lvmetad.socket to /usr/lib/systemd/system/lvm2-lvmetad.socket.
Removed symlink /etc/systemd/system/multi-user.target.wants/lvm2-cluster-activation.service.


# /usr/tests/sts-rhel7.4/lvm2/bin/lvm_rpms 
3.10.0-664.el7.x86_64

lvm2-2.02.171-1.el7    BUILT: Wed May  3 14:05:13 CEST 2017
lvm2-libs-2.02.171-1.el7    BUILT: Wed May  3 14:05:13 CEST 2017
lvm2-cluster-2.02.171-1.el7    BUILT: Wed May  3 14:05:13 CEST 2017
device-mapper-1.02.140-1.el7    BUILT: Wed May  3 14:05:13 CEST 2017
device-mapper-libs-1.02.140-1.el7    BUILT: Wed May  3 14:05:13 CEST 2017
device-mapper-event-1.02.140-1.el7    BUILT: Wed May  3 14:05:13 CEST 2017
device-mapper-event-libs-1.02.140-1.el7    BUILT: Wed May  3 14:05:13 CEST 2017
device-mapper-persistent-data-0.7.0-0.1.rc6.el7    BUILT: Mon Mar 27 17:15:46 CEST 2017
cmirror-2.02.171-1.el7    BUILT: Wed May  3 14:05:13 CEST 2017

Comment 17 errata-xmlrpc 2017-08-01 21:47:18 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2222