Bug 612221

Created attachment 430098 [details]
log from grant-01

Created attachment 430099 [details]
log from grant-02

Created attachment 430100 [details]
log from grant-03

This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release. It has
been denied for the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **

Reproduced this issue with the latest builds:
2.6.32-44.el6.x86_64

lvm2-2.02.69-2.el6    BUILT: Fri Jul  2 07:26:01 CDT 2010
lvm2-libs-2.02.69-2.el6    BUILT: Fri Jul  2 07:26:01 CDT 2010
lvm2-cluster-2.02.69-2.el6    BUILT: Fri Jul  2 07:26:01 CDT 2010
device-mapper-1.02.51-2.el6    BUILT: Fri Jul  2 07:26:01 CDT 2010
device-mapper-libs-1.02.51-2.el6    BUILT: Fri Jul  2 07:26:01 CDT 2010
device-mapper-event-1.02.51-2.el6    BUILT: Fri Jul  2 07:26:01 CDT 2010
device-mapper-event-libs-1.02.51-2.el6    BUILT: Fri Jul  2 07:26:01 CDT 2010
cmirror-2.02.69-2.el6    BUILT: Fri Jul  2 07:26:01 CDT 2010


---------------------------------------------
 base legs 0; convert to 1 legs;
 base log core; convert to mirrored log;
 active 0; sync=0; pvs=sufficient
---------------------------------------------
creating a base linear on grant-03
Unable to convert the log of an inactive cluster mirror
leaving mirror active for this operation!
Converting from 0 leg(s) core log; to 1 leg(s) mirrored log on grant-01

legs_up_needed: 1
legs_down_needed: 0
logs_up_needed: 2
logs_down_needed: 0

current free pvs: /dev/sdb2 /dev/sdb3 /dev/sdb5 /dev/sdb6 /dev/sdc1 /dev/sdc2 /dev/sdc3 /dev/sdc5
legs_up=/dev/sdc1
logs_up=/dev/sdb3 /dev/sdc3
legs_down=
logs_down=
total=/dev/sdc1 /dev/sdb3 /dev/sdc3

lvconvert --mirrorlog mirrored -m 1 centipede/centi_base /dev/sdc1 /dev/sdb3 /dev/sdc3
[HANG]


Jul 27 16:10:15 grant-01 qarshd[23177]: Running cmdline: lvconvert --mirrorlog mirrored -m 1 centipede/centi_base /dev/sdc1 /dev/sdb3 /dev/sdc3
Jul 27 16:10:16 grant-01 lvm[27664]: Monitoring mirror device centipede-centi_base_mlog for events.
Jul 27 16:10:16 grant-01 lvm[27664]: centipede-centi_base_mlog is now in-sync.
Jul 27 16:10:20 grant-01 lvm[27664]: No longer monitoring mirror device centipede-centi_base_mlog for events.
Jul 27 16:10:20 grant-01 cmirrord[17880]: Log entry already exists: LVM-N2o1fHSLSETgzUXUIi02ke7ZvEdDoMJfiC9uRh8ZBoe6JjJ0Q2tGAueoVWA7yC39
Jul 27 16:10:20 grant-01 cmirrord[17880]: clog_resume:  Failed to create cluster CPG
Jul 27 16:10:21 grant-01 lvm[27664]: Monitoring mirror device centipede-centi_base_mlog for events.
Jul 27 16:10:21 grant-01 lvm[27664]: Monitoring mirror device centipede-centi_base for events.

I still think this should block RHEL6.0

One command:
'lvcreate -m1 -l1 -n lv vg --mirrorlog mirrored'

Result (everytime?):
Jul 28 16:02:36 bp-01 cmirrord[1993]: Log entry already exists: LVM-sf0xvb3Q5GPl2F9LnM5M6S0SoVOTKsb2BCh1fdwl96XKKb3IMwCTDeoBk21hO2ji
Jul 28 16:02:36 bp-01 cmirrord[1993]: clog_resume:  Failed to create cluster CPG



I see that you are using 'mirrored' log in comment #5 - this seems to be the problem.  Could be that LVM is passing down the wrong UUID for the mirrored log identifier.

code is using UUID of the mirror if log type is 'core' and UUID of the log_lv if log type is 'disk|mirrored'.  Therefore, the main mirror is using the UUID of the log as the unique identifier, but so is the underlying mirrored log - this leads to conflict.

UUID conflict resolved, but now top-level mirror log refuses to get past "resume" - unclear why yet.

Mirrored log type for cluster mirrors will have to be deferred to a later release.  I will soon commit the patch that disallows this.  Reason from my patch:

Disallow mirrored logs in cluster mirrors.

The cluster log daemon (cmirrord) is not multi-threaded and
can handle only one request at a time.  When a log is stacked
on top of a mirror (which itself contains a 'core' log), it
creates a situation that cannot be solved without threading.

When the top level mirror issues a "resume", the log daemon
attempts to read from the log device to retrieve the log
state.  However, the log is a mirror which, before issuing
the read, attempts to determine the 'sync' status of the
region of the mirror which is to be read.  This sync status
request cannot be completed by the daemon because it is
blocked on a read I/O to the very mirror requesting the
sync status.

This bug should be resolved by virtue of the fact that the configuration that causes the issue is disallowed.  Another bug will have to be opened to request the feature to be enabled.  To complete this feature, threading will need to be added to the cluster log daemon - at least at a remedial enough level to process requests from different logs simultaneously.

It will still be necessary to verify that this bug does not show up in other configurations (cluster mirrors with 'core' or 'disk' log types).  Will change state from 'ASSIGNED' when patch is checked in.

checked in upstream - destined for 2.02.73

[root@hayes-02 ~]# lvcreate -m 1 --mirrorlog mirrored -L 100M -n mirror centipede
  Log type, "mirrored", is unavailable to cluster mirrors

Verified that mirrored logs are no longer allowed in the latest lvm rpms. Like mentioned in comment #12, if we are to ever support mirrored logs with cluster mirrors, this bug will need to actually be fixed.

Red Hat Enterprise Linux 6.0 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.