Bug 772598

Summary:	dlm tries to start before network, and fails unrecoverably
Product:	[Fedora] Fedora	Reporter:	Ilkka Tengvall <ikke>
Component:	dlm-kernel	Assignee:	Jonathan Earl Brassow <jbrassow>
Status:	CLOSED WONTFIX	QA Contact:
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	16	CC:	pcfe
Target Milestone:	---
Target Release:	---
Hardware:	x86_64
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2013-02-13 14:55:12 UTC	Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Bug Depends On:
Bug Blocks:	494832

Description Ilkka Tengvall 2012-01-09 11:45:05 UTC

Description of problem:

I keep loosing nodes from my cluster occasionally, and they won't come back after reboot (fence) of the node. The reason is perhaps the dlm that won't start. It looks like dlm tries to start, doesn't find the network, and gives up and won't retry.


Version-Release number of selected component (if applicable):

# rpm -q kernel cman
kernel-3.1.6-1.fc16.x86_64
cman-3.1.7-1.fc16.x86_64


How reproducible:
often, not always.


Steps to Reproduce:
1. reboot the node from cluster
2. see dlm not starting
3.
  
Actual results:
dlm does not start, not even by doing service cman restart

Expected results:
dlm should start, among the rest of the cluster stuff (cmnad,lvmd)

Additional info:

in error cases this appears into the log:
-------------------------
# dmesg |grep dlm
[   68.363462] dlm: no local IP address has been set
[   68.363549] dlm: cannot start dlm lowcomms -107
[   68.372532] dlm: no local IP address has been set
[   68.372622] dlm: cannot start dlm lowcomms -107
-------------------------

in working case the dmesg looks like this:

-------------------------
[   26.270687] dlm: Using TCP for communications
[   26.287725] dlm: got connection from 4
[   26.288069] dlm: got connection from 2
-------------------------

the network is manually managed, not network manager managed. There is physical device em2 which is glued into sw bridge. The config below:

-------------------------
/etc/sysconfig/network-scripts/ifcfg-em1:IPV6INIT=no
/etc/sysconfig/network-scripts/ifcfg-em1:NM_CONTROLLED=no
/etc/sysconfig/network-scripts/ifcfg-em1:HWADDR=00:17:a4:77:c8:10
/etc/sysconfig/network-scripts/ifcfg-em1:DEVICE=em1
/etc/sysconfig/network-scripts/ifcfg-em1:ONBOOT=yes
/etc/sysconfig/network-scripts/ifcfg-em1:TYPE=Ethernet
/etc/sysconfig/network-scripts/ifcfg-em1:USERCTL=no
/etc/sysconfig/network-scripts/ifcfg-em1:BRIDGE=br_lab
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:IPV6INIT=no
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:NM_CONTROLLED=no
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:BOOTPROTO=static
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:DEVICE=br_iscsi
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:ONBOOT=yes
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:DELAY=0
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:TYPE=Bridge
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:USERCTL=no
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:PREFIX=24
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:IPADDR=10.0.0.1
/etc/sysconfig/network-scripts/ifcfg-br_iscsi:NETMASK=255.255.255.0
-------------------------


And the sysctl script for cman/dlm is not modified, it has dependency to network.

coroner is told to use that interface:
/etc/corosync/corosync.conf:            bindnetaddr: 10.0.0.0

Comment 1 Ilkka Tengvall 2012-01-09 11:48:41 UTC

ah, wrong network script, here's the correct one:

-------------------------
/etc/sysconfig/network-scripts/ifcfg-em2:DEVICE=em2
/etc/sysconfig/network-scripts/ifcfg-em2:HWADDR=00:17:A4:77:C8:12
/etc/sysconfig/network-scripts/ifcfg-em2:ONBOOT=yes
/etc/sysconfig/network-scripts/ifcfg-em2:NM_CONTROLLED=no
/etc/sysconfig/network-scripts/ifcfg-em2:TYPE=Ethernet
/etc/sysconfig/network-scripts/ifcfg-em2:IPV6INIT=no
/etc/sysconfig/network-scripts/ifcfg-em2:USERCTL=no
/etc/sysconfig/network-scripts/ifcfg-em2:BRIDGE=br_iscsi
-------------------------


, and typos in the expected results, s/cmnad,lvmd/cmand,clvmd/

Comment 2 Fedora End Of Life 2013-01-16 13:58:31 UTC

This message is a reminder that Fedora 16 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 16. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '16'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 16's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 16 is end of life. If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora, you are encouraged to click on 
"Clone This Bug" and open it against that version of Fedora.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 3 Fedora End Of Life 2013-02-13 14:55:14 UTC

Fedora 16 changed to end-of-life (EOL) status on 2013-02-12. Fedora 16 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.