RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 729481 - rgmanager should detect when clvmd is not running when using HA LVM w/ clvm locking
Summary: rgmanager should detect when clvmd is not running when using HA LVM w/ clvm l...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: resource-agents
Version: 6.1
Hardware: x86_64
OS: Linux
low
low
Target Milestone: rc
: ---
Assignee: Jonathan Earl Brassow
QA Contact: Cluster QE
URL:
Whiteboard:
Depends On:
Blocks: 756082
TreeView+ depends on / blocked
 
Reported: 2011-08-09 20:58 UTC by Corey Marthaler
Modified: 2012-06-20 14:38 UTC (History)
4 users (show)

Fixed In Version: resource-agents-3.9.2-11.el6
Doc Type: Bug Fix
Doc Text:
No documentation needed
Clone Of:
Environment:
Last Closed: 2012-06-20 14:38:45 UTC
Target Upstream Version:
Embargoed:

Attachments (Terms of Use)
Patch to fix problem and provide better error reporting (2.56 KB, patch)
2012-03-05 19:56 UTC, Jonathan Earl Brassow 		no flags Details | Diff
View All


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2012:0947 0 normal SHIPPED_LIVE resource-agents bug fix and enhancement update 2012-06-19 20:59:56 UTC

Description Corey Marthaler 2011-08-09 20:58:57 UTC 
Description of problem:
The preferred method for HA LVM locking is now clvm instead of volume_list tags, even though both are supported. So when using a clvm configuration, rgmanager should be able to detect when clvmd is not running. If it's not, rgmanager currently assumes it's an invalid HA config due to the fact that there's no volume_list tagging section in lvm.conf. Would is be possible to check if the VG(s) specified in the rm cluster.conf section have the cluster attribute, and then, if clvmd is not running use that as a more helpful error message.


Aug  9 15:45:29 taft-03 rgmanager[2410]: I am node #3
Aug  9 15:45:29 taft-03 rgmanager[2410]: Resource Group Manager Starting
Aug  9 15:45:29 taft-03 rgmanager[2410]: Loading Service Data
Aug  9 15:45:35 taft-03 rgmanager[2410]: Initializing Services
Aug  9 15:45:36 taft-03 rgmanager[3331]: [fs] stop: Could not match /dev/TAFT1/ha with a real device
Aug  9 15:45:36 taft-03 rgmanager[2410]: stop on fs "fs1" returned 2 (invalid argument(s))
Aug  9 15:45:36 taft-03 rgmanager[3368]: [fs] stop: Could not match /dev/TAFT3/ha with a real device
Aug  9 15:45:36 taft-03 rgmanager[2410]: stop on fs "fs3" returned 2 (invalid argument(s))
Aug  9 15:45:36 taft-03 rgmanager[3376]: [fs] stop: Could not match /dev/TAFT2/ha with a real device
Aug  9 15:45:36 taft-03 rgmanager[2410]: stop on fs "fs2" returned 2 (invalid argument(s))
Aug  9 15:45:37 taft-03 rgmanager[3471]: [lvm] HA LVM:  Improper setup detected
Aug  9 15:45:37 taft-03 rgmanager[3486]: [lvm] HA LVM:  Improper setup detected
Aug  9 15:45:37 taft-03 rgmanager[3517]: [lvm] HA LVM:  Improper setup detected
Aug  9 15:45:37 taft-03 rgmanager[3503]: [lvm] * "volume_list" not specified in lvm.conf.
Aug  9 15:45:37 taft-03 rgmanager[3606]: [lvm] * "volume_list" not specified in lvm.conf.
Aug  9 15:45:38 taft-03 rgmanager[3664]: [lvm] WARNING: An improper setup can cause data corruption!
Aug  9 15:45:38 taft-03 rgmanager[3680]: [lvm] WARNING: An improper setup can cause data corruption!
Aug  9 15:45:38 taft-03 rgmanager[3657]: [lvm] * "volume_list" not specified in lvm.conf.
Aug  9 15:45:38 taft-03 rgmanager[3732]: [lvm] WARNING: An improper setup can cause data corruption!
Aug  9 15:45:41 taft-03 rgmanager[2410]: Services Initialized
Aug  9 15:45:41 taft-03 rgmanager[2410]: State change: Local UP
Aug  9 15:45:41 taft-03 rgmanager[2410]: State change: taft-01 UP
Aug  9 15:45:41 taft-03 rgmanager[2410]: State change: taft-02 UP
Aug  9 15:45:41 taft-03 rgmanager[2410]: State change: taft-04 UP
Aug  9 15:45:41 taft-03 rgmanager[2410]: Starting stopped service service:halvm1
Aug  9 15:45:41 taft-03 rgmanager[2410]: Starting stopped service service:halvm2
Aug  9 15:45:41 taft-03 rgmanager[2410]: Starting stopped service service:halvm3
Aug  9 15:45:42 taft-03 rgmanager[3863]: [lvm] HA LVM:  Improper setup detected
Aug  9 15:45:43 taft-03 rgmanager[3891]: [lvm] HA LVM:  Improper setup detected
Aug  9 15:45:43 taft-03 rgmanager[3885]: [lvm] * "volume_list" not specified in lvm.conf.
Aug  9 15:45:43 taft-03 rgmanager[2410]: start on lvm "lvm2" returned 1 (generic error)
Aug  9 15:45:43 taft-03 rgmanager[3902]: [lvm] HA LVM:  Improper setup detected
Aug  9 15:45:43 taft-03 rgmanager[2410]: #68: Failed to start service:halvm2; return value: 1
Aug  9 15:45:43 taft-03 rgmanager[2410]: Stopping service service:halvm2
Aug  9 15:45:43 taft-03 rgmanager[3946]: [lvm] * "volume_list" not specified in lvm.conf.
Aug  9 15:45:43 taft-03 rgmanager[2410]: start on lvm "lvm1" returned 1 (generic error)
Aug  9 15:45:43 taft-03 rgmanager[2410]: #68: Failed to start service:halvm1; return value: 1
Aug  9 15:45:43 taft-03 rgmanager[3960]: [lvm] * "volume_list" not specified in lvm.conf.
Aug  9 15:45:43 taft-03 rgmanager[2410]: Stopping service service:halvm1
Aug  9 15:45:43 taft-03 rgmanager[2410]: start on lvm "lvm3" returned 1 (generic error)
Aug  9 15:45:43 taft-03 rgmanager[2410]: #68: Failed to start service:halvm3; return value: 1
Aug  9 15:45:43 taft-03 rgmanager[2410]: Stopping service service:halvm3
Aug  9 15:45:43 taft-03 rgmanager[3996]: [fs] stop: Could not match /dev/TAFT2/ha with a real device
Aug  9 15:45:43 taft-03 rgmanager[2410]: stop on fs "fs2" returned 2 (invalid argument(s))
Aug  9 15:45:43 taft-03 rgmanager[4046]: [fs] stop: Could not match /dev/TAFT1/ha with a real device
Aug  9 15:45:43 taft-03 rgmanager[2410]: stop on fs "fs1" returned 2 (invalid argument(s))
Aug  9 15:45:44 taft-03 rgmanager[4090]: [fs] stop: Could not match /dev/TAFT3/ha with a real device
Aug  9 15:45:44 taft-03 rgmanager[2410]: stop on fs "fs3" returned 2 (invalid argument(s))
Aug  9 15:45:45 taft-03 rgmanager[4149]: [lvm] HA LVM:  Improper setup detected
Aug  9 15:45:45 taft-03 rgmanager[4165]: [lvm] HA LVM:  Improper setup detected
Aug  9 15:45:45 taft-03 rgmanager[4173]: [lvm] HA LVM:  Improper setup detected
Aug  9 15:45:45 taft-03 rgmanager[4188]: [lvm] * "volume_list" not specified in lvm.conf.
Aug  9 15:45:45 taft-03 rgmanager[4231]: [lvm] * "volume_list" not specified in lvm.conf.
Aug  9 15:45:45 taft-03 rgmanager[4237]: [lvm] * "volume_list" not specified in lvm.conf.
Aug  9 15:45:45 taft-03 rgmanager[4253]: [lvm] WARNING: An improper setup can cause data corruption!
Aug  9 15:45:46 taft-03 rgmanager[4297]: [lvm] WARNING: An improper setup can cause data corruption!
Aug  9 15:45:46 taft-03 rgmanager[4308]: [lvm] WARNING: An improper setup can cause data corruption!
Aug  9 15:45:48 taft-03 rgmanager[2410]: #12: RG service:halvm1 failed to stop; intervention required
Aug  9 15:45:48 taft-03 rgmanager[2410]: Service service:halvm1 is failed
Aug  9 15:45:48 taft-03 rgmanager[2410]: #13: Service service:halvm1 failed to stop cleanly
Aug  9 15:45:48 taft-03 rgmanager[2410]: #12: RG service:halvm3 failed to stop; intervention required
Aug  9 15:45:48 taft-03 rgmanager[2410]: Service service:halvm3 is failed
Aug  9 15:45:48 taft-03 rgmanager[2410]: #13: Service service:halvm3 failed to stop cleanly
Aug  9 15:45:49 taft-03 rgmanager[2410]: #12: RG service:halvm2 failed to stop; intervention required
Aug  9 15:45:49 taft-03 rgmanager[2410]: Service service:halvm2 is failed
Aug  9 15:45:49 taft-03 rgmanager[2410]: #13: Service service:halvm2 failed to stop cleanly



Version-Release number of selected component (if applicable):
rgmanager-3.0.12.1-2.el6.x86_64
Comment 1 Jonathan Earl Brassow 2011-08-11 20:48:20 UTC 
I think it does this already...  Are you should you have a 'c' attribute on the volume group?

start)
        if ! [[ $(vgs -o attr --noheadings $OCF_RESKEY_vg_name) =~ .....c ]]; then
                ha_lvm_proper_setup_check || exit 1
        fi
Comment 2 Corey Marthaler 2011-08-12 19:06:27 UTC 
Yep. If you have clvmd running, the services work, if you don't, it complains that there's an invalid configuration.

[root@taft-01 ~]# vgs -a -o +devices
  VG        #PV #LV #SN Attr   VSize   VFree   Devices
  TAFT1       3   1   0 wz--nc 203.48g 187.48g ha_mimage_0(0),ha_mimage_1(0)
  TAFT1       3   1   0 wz--nc 203.48g 187.48g /dev/sdh2(0)
  TAFT1       3   1   0 wz--nc 203.48g 187.48g /dev/sdh1(0)
  TAFT1       3   1   0 wz--nc 203.48g 187.48g /dev/sdg2(0)
  TAFT2       3   1   0 wz--nc 203.48g 187.48g ha_mimage_0(0),ha_mimage_1(0)
  TAFT2       3   1   0 wz--nc 203.48g 187.48g /dev/sdg1(0)
  TAFT2       3   1   0 wz--nc 203.48g 187.48g /dev/sdf2(0)
  TAFT2       3   1   0 wz--nc 203.48g 187.48g /dev/sdf1(0)
  TAFT3       3   1   0 wz--nc 203.48g 187.48g ha_mimage_0(0),ha_mimage_1(0)
  TAFT3       3   1   0 wz--nc 203.48g 187.48g /dev/sde2(0)
  TAFT3       3   1   0 wz--nc 203.48g 187.48g /dev/sde1(0)
  TAFT3       3   1   0 wz--nc 203.48g 187.48g /dev/sdd2(0)
  TAFT4       3   1   0 wz--nc 203.48g 187.48g ha_mimage_0(0),ha_mimage_1(0)
  TAFT4       3   1   0 wz--nc 203.48g 187.48g /dev/sdd1(0)
  TAFT4       3   1   0 wz--nc 203.48g 187.48g /dev/sdc2(0)
  TAFT4       3   1   0 wz--nc 203.48g 187.48g /dev/sdc1(0)

[root@taft-01 ~]# service rgmanager stop
Stopping Cluster Service Manager:                          [  OK  ]

[root@taft-01 ~]# service clvmd stop
Deactivating clustered VG(s):   0 logical volume(s) in volume group "TAFT1" now active
  0 logical volume(s) in volume group "TAFT2" now active
  0 logical volume(s) in volume group "TAFT3" now active
  clvmd not running on node taft-02
  0 logical volume(s) in volume group "TAFT4" now active
  clvmd not running on node taft-02
                                                           [  OK  ]
Signaling clvmd to exit                                    [  OK  ]
clvmd terminated                                           [  OK  ]

[root@taft-01 ~]# service cman status
cluster is running.
[root@taft-01 ~]# service rgmanager start
Starting Cluster Service Manager:                          [  OK  ]

Aug 12 13:52:24 taft-01 rgmanager[10551]: I am node #1
Aug 12 13:52:24 taft-01 rgmanager[10551]: Resource Group Manager Starting
Aug 12 13:52:24 taft-01 rgmanager[10551]: Loading Service Data
Aug 12 13:52:30 taft-01 rgmanager[10551]: Initializing Services
Aug 12 13:52:31 taft-01 rgmanager[11535]: [fs] stop: Could not match /dev/TAFT1/ha with a real device
Aug 12 13:52:31 taft-01 rgmanager[10551]: stop on fs "fs1" returned 2 (invalid argument(s))
Aug 12 13:52:31 taft-01 rgmanager[11559]: [fs] stop: Could not match /dev/TAFT2/ha with a real device
Aug 12 13:52:31 taft-01 rgmanager[10551]: stop on fs "fs2" returned 2 (invalid argument(s))
Aug 12 13:52:31 taft-01 rgmanager[11601]: [fs] stop: Could not match /dev/TAFT4/ha with a real device
Aug 12 13:52:31 taft-01 rgmanager[10551]: stop on fs "fs4" returned 2 (invalid argument(s))
Aug 12 13:52:31 taft-01 rgmanager[11611]: [fs] stop: Could not match /dev/TAFT3/ha with a real device
Aug 12 13:52:31 taft-01 rgmanager[10551]: stop on fs "fs3" returned 2 (invalid argument(s))
Aug 12 13:52:32 taft-01 rgmanager[11726]: [lvm] HA LVM:  Improper setup detected
Aug 12 13:52:33 taft-01 rgmanager[11748]: [lvm] * "volume_list" not specified in lvm.conf.
Aug 12 13:52:33 taft-01 rgmanager[11762]: [lvm] HA LVM:  Improper setup detected
Aug 12 13:52:33 taft-01 rgmanager[11787]: [lvm] HA LVM:  Improper setup detected
Aug 12 13:52:33 taft-01 rgmanager[11790]: [lvm] HA LVM:  Improper setup detected
Aug 12 13:52:33 taft-01 rgmanager[11784]: [lvm] WARNING: An improper setup can cause data corruption!
Aug 12 13:52:33 taft-01 rgmanager[11829]: [lvm] * "volume_list" not specified in lvm.conf.
Aug 12 13:52:34 taft-01 rgmanager[11863]: [lvm] * "volume_list" not specified in lvm.conf.
Aug 12 13:52:34 taft-01 rgmanager[11881]: [lvm] * "volume_list" not specified in lvm.conf.
Aug 12 13:52:34 taft-01 rgmanager[11902]: [lvm] WARNING: An improper setup can cause data corruption!
Aug 12 13:52:34 taft-01 rgmanager[11941]: [lvm] WARNING: An improper setup can cause data corruption!
Aug 12 13:52:34 taft-01 rgmanager[11955]: [lvm] WARNING: An improper setup can cause data corruption!
Aug 12 13:52:39 taft-01 rgmanager[10551]: Services Initialized
Aug 12 13:52:39 taft-01 rgmanager[10551]: State change: Local UP
Aug 12 13:52:39 taft-01 rgmanager[10551]: State change: taft-02 UP
Aug 12 13:52:39 taft-01 rgmanager[10551]: State change: taft-03 UP
Aug 12 13:52:39 taft-01 rgmanager[10551]: State change: taft-04 UP
Comment 4 Jonathan Earl Brassow 2012-03-05 19:56:10 UTC 
Created attachment 567742 [details]
Patch to fix problem and provide better error reporting
Comment 5 Chris Feist 2012-03-05 22:07:06 UTC 
Pushed upstream: https://github.com/ClusterLabs/resource-agents/commit/3d0503c330812f763ed31ccfbb184585d192292e
Comment 8 Chris Feist 2012-04-30 22:01:11 UTC 
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
No documentation needed
Comment 10 errata-xmlrpc 2012-06-20 14:38:45 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2012-0947.html
Note You need to log in before you can comment on or make changes to this bug.