Bug 235092

Summary:	clvmd operations timeout under heavy load == bad for recovery scenarios
Product:	[Retired] Red Hat Cluster Suite	Reporter:	Jonathan Earl Brassow <jbrassow>
Component:	lvm2-cluster	Assignee:	Milan Broz <mbroz>
Status:	CLOSED ERRATA	QA Contact:	Cluster QE <mspqa-list>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	4	CC:	agk, ccaulfie, dwysocha, jbrassow, mbroz, prockai, pvrabec
Target Milestone:	---
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	RHBA-2007-0046	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2007-05-10 21:10:01 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jonathan Earl Brassow 2007-04-03 18:51:02 UTC

When a mirror device fails under heavy load, it can take a very long time
(minutes) for each CLVM command to process.  This can lead to clvmd time-outs
being triggered and the mirror fault handling code to abort.

The root of the problem is the need for remote nodes to have to scan the devices
when doing activates/deactivates.  Those scans get queued up behind all the
other I/O that is happening and simply take a long time.

Either we need to find a completely different way to detect stalled machines, or
we need to raise the clvmd timeout (e.g. clvmd -t 100).  I'm advocating the
later for now and the former when we have more time to investigate.

Comment 1 Jonathan Earl Brassow 2007-04-03 18:52:12 UTC

To be clear about my request:

Let's increase the clvmd timeout.

Comment 2 Jonathan Earl Brassow 2007-04-16 20:49:05 UTC

Index: LVM2/scripts/clvmd_init_rhel4
===================================================================
--- LVM2.orig/scripts/clvmd_init_rhel4
+++ LVM2/scripts/clvmd_init_rhel4
@@ -15,7 +15,7 @@ VGCHANGE="/usr/sbin/vgchange"
 VGSCAN="/usr/sbin/vgscan"
 VGDISPLAY="/usr/sbin/vgdisplay"
 VGS="/usr/sbin/vgs"
-CLVMDOPTS="-T20"
+CLVMDOPTS="-T20 -t 90"

 [ -f /etc/sysconfig/cluster ] && . /etc/sysconfig/cluster

Comment 8 Red Hat Bugzilla 2007-05-10 21:10:01 UTC

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on the solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2007-0046.html