Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1081240

Summary:	Filesystem resource spamming logs with fs-090_monitor_20000:15108:stderr [ cut: write error: Broken pipe ]
Product:	Red Hat Enterprise Linux 7	Reporter:	Jaroslav Kortus <jkortus>
Component:	pacemaker	Assignee:	David Vossel <dvossel>
Status:	CLOSED ERRATA	QA Contact:	Cluster QE <mspqa-list>
Severity:	medium	Docs Contact:
Priority:	medium
Version:	7.0	CC:	abeekhof, agk, cluster-maint, dvossel, fdinitto, mnovacek
Target Milestone:	rc
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	pacemaker-1.1.12-1.el7	Doc Type:	Bug Fix
Doc Text:	Cause: Child processes responsible for running resource operations inherited the direction to ignore SIGPIPE from the parent. Consequence: The logs were spammed with error logs regarding broken pipes. Fix: Instruct the child process not to ignore SIGPIPE Result: The filesystem is not filled with log spam.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2015-03-05 09:59:57 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Jaroslav Kortus 2014-03-26 20:36:55 UTC

Description of problem:
When you have larger amount of filesystem resources on one node, the Filesystem agent starts spamming logs with:
Mar 26 21:07:18 light-03 lrmd[4001]: notice: operation_finished: fs-090_monitor_20000:15108:stderr [ cut: write error: Broken pipe ]


Version-Release number of selected component (if applicable):
pacemaker-1.1.10-29.el7.x86_64

How reproducible:
always

Steps to Reproduce:
1. create about 100 or more Filesystem resources
2. watch the logs
3.

Actual results:
"Spam" messages about broken pipe

Expected results:
no broken pipe messages

Additional info:
From what I can tell this is most likely caused by a construct on line 820:
820         if list_mounts | grep -q " $MOUNTPOINT " >/dev/null 2>&1; then

the list_mounts function runs cut on /proc/mounts input and this ends up piped to grep -q. However, grep -q terminates whenever the first match is found, which might happen before cut finishes it's write calls. This is much more likely for larger /proc/mounts files. Write call to (finished) grep's stdin results in SIGPIPE.

Comment 3 David Vossel 2014-08-05 20:14:23 UTC

There is an upstream patch posted for this issue.

https://github.com/ClusterLabs/resource-agents/pull/477

Comment 4 David Vossel 2014-08-15 17:02:11 UTC

It turns out this needed to be fixed in pacemaker. SIGPIPE is ignored by pacemaker, which in turn is inherited by the resource-agents. Now pacemaker restores the default behavior of SIGPIPE when forking resource-agents. This prevents some odd behavior by bash tools like 'cut'.

https://github.com/ClusterLabs/pacemaker/commit/e3aa0b293611b41713a343e0b4f19160fd8d6bc7

Comment 8 michal novacek 2014-12-01 13:27:32 UTC

I have verified that there are no more 'broken pipe' messages with 150 filesystem resource with pacemaker-1.1.12-13.el7.x86_64.

-----

[root@virt-072 ~]# pcs status
Cluster name: STSRHTS31212
Last updated: Mon Dec  1 14:26:02 2014
Last change: Mon Dec  1 14:04:36 2014
Stack: corosync
Current DC: virt-069 (2) - partition with quorum
Version: 1.1.12-a14efad
3 Nodes configured
159 Resources configured


Online: [ virt-063 virt-069 virt-072 ]

Full list of resources:

 fence-virt-063 (stonith:fence_xvm):    Started virt-063 
 fence-virt-069 (stonith:fence_xvm):    Started virt-069 
 fence-virt-072 (stonith:fence_xvm):    Started virt-072 
 Clone Set: dlm-clone [dlm]
     Started: [ virt-063 virt-069 virt-072 ]
 Clone Set: clvmd-clone [clvmd]
     Started: [ virt-063 virt-069 virt-072 ]
 fs1    (ocf::heartbeat:Filesystem):    Started virt-069 
 ...
 fs150  (ocf::heartbeat:Filesystem):    Started virt-072 

PCSD Status:
  virt-063: Online
  virt-069: Online
  virt-072: Online

Daemon Status:
  corosync: active/disabled
  pacemaker: active/enabled
  pcsd: active/enabled

[root@virt-072 ~]# pcs resource | grep '(ocf::heartbeat:Filesystem):' | grep Started | wc -l
150

[root@virt-072 ~]# grep 'broken pipe' /var/log/messages
[root@virt-072 ~]# echo $?
0

Comment 10 errata-xmlrpc 2015-03-05 09:59:57 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0440.html