Bug 213036 - msg_send: Broken pipe shortly before fence
msg_send: Broken pipe shortly before fence
Status: CLOSED NOTABUG
Product: Red Hat Cluster Suite
Classification: Red Hat
Component: rgmanager (Show other bugs)
4
All Linux
medium Severity medium
: ---
: ---
Assigned To: Lon Hohberger
Cluster QE
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2006-10-30 12:14 EST by Lenny Maiorani
Modified: 2009-04-16 16:21 EDT (History)
1 user (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2006-11-21 09:56:37 EST
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Lenny Maiorani 2006-10-30 12:14:10 EST
Description of problem:
occasionally shortly before a fence (missed heartbeats is the only known
problem) rgmanager will complain numerous times about a "Broken pipe"
 

Version-Release number of selected component (if applicable):
RHEL4U4

How reproducible:
happens sometimes

Steps to Reproduce:
1. unknown
2.
3.
  
Actual results:
"msg_send: Broken pipe" printed to stdout by rgmanager

Expected results:
no broken pipes

Additional info:
stdout on node3:
Fri Oct 20 23:04:27 2006
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe

Fri Oct 20 23:04:27 2006
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe

Fri Oct 20 23:06:55 2006
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe

Fri Oct 20 23:06:55 2006
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe

Fri Oct 20 23:06:55 2006
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe

Fri Oct 20 23:06:55 2006
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe

Fri Oct 20 23:06:55 2006
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe


stdout from rgmanager on node1:

Fri Oct 20 23:04:30 2006
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe
msg_send: Broken pipe

Fri Oct 20 23:06:54 2006
msg_send: Broken pipe

Fri Oct 20 23:06:54 2006
msg_send: Broken pipe

Fri Oct 20 23:06:54 2006
msg_send: Broken pipe

Fri Oct 20 23:06:54 2006
msg_send: Broken pipe

Fri Oct 20 23:06:54 2006
msg_send: Broken pipe

Fri Oct 20 23:06:54 2006
msg_send: Broken pipe

Fri Oct 20 23:06:54 2006
msg_send: Broken pipe



/var/log/messages on node1:
Oct 20 23:22:32 flsrv01 kernel: CMAN: removing node flsrv03 from the cluster :
Missed too many heartbeats
Oct 20 23:22:32 flsrv01 fenced: flsrv03 not a cluster member after 0 sec
post_fail_delay
Oct 20 23:22:32 flsrv01 fenced: fencing node "flsrv03"
Oct 20 23:23:33 flsrv01 fenced: fence "flsrv03" success


/var/log/messages on node2:
Oct 20 23:22:32 flsrv02 kernel: CMAN: node flsrv03 has been removed from the
cluster : Missed too many heartbeats
Oct 20 23:22:32 flsrv02 fenced: fencing deferred to flsrv01



On nodes 1 and 2 there are no messages for hours before that. Not sure if these
messages are related to the fence or not.
Comment 1 Lon Hohberger 2006-11-21 09:56:37 EST
They might be related, but the EPIPE errors are being sent to stdout, and really
are there for debugging.

If there's some sort of scheduling that is preempting rgmanager (which is
likely; it does not run in realtime), you can see those messages.  Under most
situations, rgmanager will retry the connection.  If it doesn't retry, you
should see other errors in the system logs from rgmanager.

Note You need to log in before you can comment on or make changes to this bug.