Bug 728365 - any change to lvm.conf results in an invalid HA configuration
Summary: any change to lvm.conf results in an invalid HA configuration
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: resource-agents
Version: 6.1
Hardware: x86_64
OS: Linux
low
low
Target Milestone: rc
: ---
Assignee: Jonathan Earl Brassow
QA Contact: Cluster QE
URL:
Whiteboard:
: 817060 (view as bug list)
Depends On:
Blocks: 773372 817060 818063
TreeView+ depends on / blocked
 
Reported: 2011-08-04 20:25 UTC by Corey Marthaler
Modified: 2013-02-21 07:51 UTC (History)
18 users (show)

Fixed In Version: resource-agents-3.9.2-14.el6
Doc Type: Bug Fix
Doc Text:
For HA LVM to work properly, the initrd which the system uses during the boot process must be in-sync with lvm.conf. A warning and an error are produced if the initrd is younger than lvm.conf. This has been reduced to simply an error message due to possible issues it can cause once the HA setup is operating.
Clone Of:
: 773372 (view as bug list)
Environment:
Last Closed: 2013-02-21 07:51:47 UTC


Attachments (Terms of Use)
3 Patches to fix the problem (1.56 KB, application/x-gzip)
2012-04-27 19:18 UTC, Jonathan Earl Brassow
no flags Details
upstream committed patch for 728365 (1.88 KB, patch)
2012-10-10 20:10 UTC, Jonathan Earl Brassow
no flags Details | Diff


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2013:0288 normal SHIPPED_LIVE resource-agents bug fix and enhancement update 2013-02-20 20:36:39 UTC

Description Corey Marthaler 2011-08-04 20:25:31 UTC
Description of problem:
This may end up being a WORKASINTENDED bug, but I think it's worth documenting. When setting up the "tag" method of HA LVM, it's required that a mkinirtd be run after the volume_list tag section is edited. A check when starting the service verifies that the initrd is newer then the lvm.conf file. The problem comes when any additional changes are made to the lvm.conf file. This causes the HA LVM service start up to start to fail. Should we really require a mkinitrd and potential reboot after each lvm.conf change? If so it should be added to the documentation.


Aug  4 13:51:34 hayes-01 rgmanager[5516]: I am node #1
Aug  4 13:51:34 hayes-01 rgmanager[5516]: Resource Group Manager Starting
Aug  4 13:51:34 hayes-01 rgmanager[5516]: Loading Service Data
Aug  4 13:51:38 hayes-01 rgmanager[5516]: Initializing Services
Aug  4 13:51:38 hayes-01 rgmanager[6347]: [fs] stop: Could not match /dev/HAYES1/ha with a real device
Aug  4 13:51:38 hayes-01 rgmanager[5516]: stop on fs "fs1" returned 2 (invalid argument(s))
Aug  4 13:51:38 hayes-01 rgmanager[6370]: [fs] stop: Could not match /dev/HAYES2/ha with a real device
Aug  4 13:51:38 hayes-01 rgmanager[5516]: stop on fs "fs2" returned 2 (invalid argument(s))
Aug  4 13:51:40 hayes-01 rgmanager[6455]: [lvm] HA LVM:  Improper setup detected
Aug  4 13:51:40 hayes-01 rgmanager[6463]: [lvm] HA LVM:  Improper setup detected
Aug  4 13:51:41 hayes-01 rgmanager[6496]: [lvm] * initrd image needs to be newer than lvm.conf
Aug  4 13:51:41 hayes-01 rgmanager[6509]: [lvm] * initrd image needs to be newer than lvm.conf
Aug  4 13:51:41 hayes-01 rgmanager[6540]: [lvm] WARNING: An improper setup can cause data corruption!
Aug  4 13:51:41 hayes-01 rgmanager[6551]: [lvm] WARNING: An improper setup can cause data corruption!


[root@hayes-03 ~]# clustat
Cluster Status for HAYES @ Thu Aug  4 15:21:40 2011
Member Status: Quorate

 Member Name   ID   Status
 ------ ----   ---- ------
 hayes-01         1 Online, rgmanager
 hayes-02         2 Online, rgmanager
 hayes-03         3 Online, Local, rgmanager

 Service Name     Owner (Last)   State
 ------- ----     ----- ------   -----
 service:halvm1   (hayes-02)     failed
 service:halvm2   (hayes-02)     failed

 

Version-Release number of selected component (if applicable):
2.6.32-174.el6.x86_64

lvm2-2.02.83-3.el6    BUILT: Fri Mar 18 09:31:10 CDT 2011
lvm2-libs-2.02.83-3.el6    BUILT: Fri Mar 18 09:31:10 CDT 2011
lvm2-cluster-2.02.83-3.el6    BUILT: Fri Mar 18 09:31:10 CDT 2011
udev-147-2.35.el6    BUILT: Wed Mar 30 07:32:05 CDT 2011
device-mapper-1.02.62-3.el6    BUILT: Fri Mar 18 09:31:10 CDT 2011
device-mapper-libs-1.02.62-3.el6    BUILT: Fri Mar 18 09:31:10 CDT 2011
device-mapper-event-1.02.62-3.el6    BUILT: Fri Mar 18 09:31:10 CDT 2011
device-mapper-event-libs-1.02.62-3.el6    BUILT: Fri Mar 18 09:31:10 CDT 2011
cmirror-2.02.83-3.el6    BUILT: Fri Mar 18 09:31:10 CDT 2011

Comment 1 Steven J. Levine 2011-08-15 17:48:00 UTC
Do we know yet whether we need to clone this as a documentation bug for RHEL 6.2 (and 5.8)?

Comment 2 RHEL Product and Program Management 2011-10-07 15:55:17 UTC
Since RHEL 6.2 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.

Comment 4 Jonathan Earl Brassow 2012-04-23 22:05:16 UTC
I'm going to make it so that the script warns the user, but doesn't error out.  It is important that the initrd be updated with lvm.conf changes.  However, making HA LVM fail because of it is overkill.

Comment 5 Jonathan Earl Brassow 2012-04-27 14:42:13 UTC
*** Bug 817060 has been marked as a duplicate of this bug. ***

Comment 6 Jonathan Earl Brassow 2012-04-27 19:18:44 UTC
Created attachment 580844 [details]
3 Patches to fix the problem

Here is a set of three patches that must be added to the upstream 'resource-agents' repository and make their way into the corresponding resource-agents package for this bug.

Comment 7 Jonathan Earl Brassow 2012-04-27 21:04:15 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
For HA LVM to work properly, the initrd which the system uses during the boot process must be in-sync with lvm.conf.  A warning and an error are produced if the initrd is younger than lvm.conf.  This has been reduced to simply an error message due to possible issues it can cause once the HA setup is operating.

Comment 9 Corey Marthaler 2012-07-26 18:55:58 UTC
Adding QA ack for 6.4.

Devel will need to provide unit testing results however before this bug can be
ultimately verified by QA.

Comment 10 Jonathan Earl Brassow 2012-10-05 19:48:36 UTC
Fix committed upstream in branch RHEL6 as follows:

commit 33b5abfb99e0de71fea3d7a39901dea576129458
Author: Jonathan Brassow <jbrassow@redhat.com>
Date:   Fri Oct 5 14:07:06 2012 -0500

    LVM Agents:  Warn instead of fail if initrd is newer than lvm.conf
    
    There are many reasons why a user may change the lvm.conf file.  They might
    change filters or default segment types, etc.  Making the HA LVM scripts
    error our every time the lvm.conf file is newer than the initrd, can cause
    serious problems.  For example, if a service is running and lvm.conf is
    changed, a subsequent fail-over would not work because the HA LVM script
    would abort on this condition.
    
    Instead of making things potentially worse, we have the scripts warn the
    user (rather than failing) whenever this condition is hit.  That way,
    the user isn't suddenly surprised by things not working.
    
    Signed-off-by: Jonathan Brassow <jbrassow@redhat.com>

Comment 11 Jonathan Earl Brassow 2012-10-05 19:52:49 UTC
Comment on attachment 580844 [details]
3 Patches to fix the problem

The patches are no longer necessary as they are now upstream

Comment 12 Jonathan Earl Brassow 2012-10-10 20:10:14 UTC
Created attachment 625132 [details]
upstream committed patch for 728365

Comment 16 errata-xmlrpc 2013-02-21 07:51:47 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-0288.html


Note You need to log in before you can comment on or make changes to this bug.