Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 880682

Summary: pcp - pmlogger service - race condition
Product: Red Hat Developer Toolset Reporter: Tomas Dohnalek <tdohnale>
Component: pcpAssignee: Nathan Scott <nathans>
Status: CLOSED CANTFIX QA Contact:
Severity: low Docs Contact:
Priority: medium    
Version: DTS 1.1 RHEL 6CC: fche, mcermak, mfranc, mgoodwin, mnewsome, ohudlick
Target Milestone: alpha   
Target Release: 2.0   
Hardware: Unspecified   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Known Issue
Doc Text:
Cause: Recent upstream changes to pcp have resulted in race conditions in pmie and pmlogger startup scripts. Consequence: The pmlogger & pmie service scripts can fail to start correctly. Workaround (if any): Apply patch in the attachment to this bug to the /opt/rh/.../rc.d/init.d/pmie and pmlogger scripts. Result: pmie/pmlogger functionality may be limited in this release. We plan to fix this issue via a later update.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2013-01-22 15:19:42 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
patch against pmie/pmlogger rc.d scripts none

Description Tomas Dohnalek 2012-11-27 15:27:49 UTC 
Description of problem:
When restarting pmlogger service, it rarely finishes with error:

# service devtoolset-1.1-pmlogger start
Starting pmlogger ... 
/opt/rh/devtoolset-1.1/root/etc/rc.d/init.d/pmlogger: line 100: /var/tmp/pcp.5vfQsSHKo/pmcheck: No such file or directory
# service devtoolset-1.1-pmlogger status
Checking for pmlogger: stopped

This is caused by running /opt/rh/devtoolset-1.1/root/etc/rc.d/init.d/pmlogger:
134  _start_pmcheck &

After that, the script pmlogger finishes and trap is caught and folder $tmp is removed. But if _start_pmcheck starts after removing $tmp folder, mentioned error occures.


Version-Release number of selected component (if applicable):
devtoolset-1.1-pcp-3.6.9-3

How reproducible:
On some machines permanently, on others it is ok,

Steps to Reproduce:
1. Install devtoolset-1.1
2. Enable devtoolset-1.1 by running "scl enable devtoolset-1.1 bash"
3. Add command "sleep 5" to file /opt/rh/devtoolset-1.1/root/etc/rc.d/init.d/pmlogger at the beginning of procedure "_start_pmcheck" (this can hopefully simulate race condition)
4. service devtoolset-1.1-pmlogger start

Actual results:
Starting pmlogger ... 
/opt/rh/devtoolset-1.1/root/etc/rc.d/init.d/pmlogger: line 100: /var/tmp/pcp.5vfQsSHKo/pmcheck: No such file or directory
And service is stopped

Expected results:
Starting pmlogger ... 
And service running.

Additional info:

This probably affects pmie rc script as well.
Comment 3 Tomas Dohnalek 2012-11-27 16:01:12 UTC 
I was too optimistic. In case of pmie, it fails for me on all machines all the time:

# service devtoolset-1.1-pmie restart
Performance Co-Pilot starting inference engine(s) ... 
/opt/rh/devtoolset-1.1/root/etc/rc.d/init.d/pmie: line 119: /var/tmp/pcp.Y4NGtHzbD/pmie: No such file or directory
Comment 4 Nathan Scott 2012-11-27 22:21:24 UTC 
Trying to figure out why I haven't observed this before (having been testing these scripts extensively for the last few days, by chance) - are you running on predominantly single CPU machines perhaps?  (I'm running on 4 and 8 core machines only here, so perhaps that might be related)

The other issue you sent me (private) mail about is slightly different, but also related to the tmpfile fixes we've just merged from our SUSE friends - our spec files are not creating the private temporary directory correctly (they block the src 'make install' target, intentionally AIUI, and creation of this directory ends up being skipped).

I'll get fixed up rpms prepared today and discuss with Frank/Matt/everyone as to level of appropriateness for uploading them at this late stage or defering to a later release.

Thanks Tomas!

--
Nathan
Comment 5 Nathan Scott 2012-11-27 22:45:41 UTC 
Have reproduced it now, ignore my question earlier re single CPU machines:

<nathans> so. wierd.  qa/575 should be failing
<nathans> oh wait
<nathans> $sudo $PCP_RC_DIR/pmie start >/dev/null 2>&1
<nathans> yikes!
<fche> haha
<fche> over to you
<nathans> now i have two things to fix here :)
Comment 6 Nathan Scott 2012-11-28 03:42:31 UTC 
This is resolved in upstream commit 26129ab3fc91a77aefa3c2807c2b49dec4631036.  My preference is to fix it in DTS-1.1, naturally, but I understand it may well be too late at this stage.
Comment 9 Frank Ch. Eigler 2012-11-28 16:19:56 UTC 
Created attachment 653599 [details]
patch against pmie/pmlogger rc.d scripts