Bug 524836

Summary: python subprocess have errors in signal handling
Product: Red Hat Enterprise Linux 5 Reporter: Shahar Frank <sfrank>
Component: pythonAssignee: Dave Malcolm <dmalcolm>
Status: CLOSED WONTFIX QA Contact: BaseOS QE <qe-baseos-auto>
Severity: high Docs Contact:
Priority: high    
Version: 5.4CC: bazulay, jthomas, rluxenbe, sct
Target Milestone: rcKeywords: Patch
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 673536 (view as bug list) Environment:
Last Closed: 2013-03-15 14:45:30 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 502792, 673536    
Attachments:
Description Flags
nointr subprocess.py patch none

Description Shahar Frank 2009-09-22 13:04:42 UTC
Description of problem:

during the test of VDSM (RHEV) we found out that in some cases (seems to be a race between two dd threads operating on different files) the dd do not write all the requested data. We found out that the DD's stdin pipe that is used to transfer the data is prematurely closed by the python side.

We suspect that it is connected to the signal handling bugs found in python. For example:

http://bugs.python.org/issue1068268
http://bugs.python.org/issue4216

Also in a test program we wrote to try to reproduce the above issue we get:


Exception in thread Thread-1327:
Traceback (most recent call last):
  File "/usr/lib64/python2.4/threading.py", line 442, in __bootstrap
    self.run()
  File "/usr/lib64/python2.4/threading.py", line 422, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/tmp/subpt.py", line 26, in writedata
    (out, err) = p.communicate(data)
  File "/usr/lib64/python2.4/subprocess.py", line 1064, in communicate
    bytes_written = os.write(self.stdin.fileno(), input[:512])
OSError: [Errno 32] Broken pipe

Where the program is run as follows: 
while python /tmp/subpt.py 100; do sleep .1 ; done; date

Comment 3 RHEL Program Management 2009-11-06 19:28:31 UTC
This request was evaluated by Red Hat Product Management for
inclusion, but this component is not scheduled to be updated in
the current Red Hat Enterprise Linux release. If you would like
this request to be reviewed for the next minor release, ask your
support representative to set the next rhel-x.y flag to "?".

Comment 6 Jon Thomas 2011-01-20 19:31:04 UTC
*** Bug 669396 has been marked as a duplicate of this bug. ***

Comment 7 Jon Thomas 2011-01-20 19:37:33 UTC
Created attachment 474519 [details]
nointr subprocess.py patch


Here's the issue reported for 669396. 

1) condor_configd sends a condor_restart to master
2) master tells condor_configd to shutdown while it's in the popen code block
3) shutdown signal fires condor_configd's exit_signal_handler
4) the signal or signal handler causes popen code block (perhaps
obj.communicate) to have an exception resulting in a -1 return value

What all that means is the program was getting a signal during 

try:
         cmd_list = shlex.split(cmd)
         obj = Popen(cmd_list, stdout=PIPE, stderr=PIPE, env=env)
         (std_out, std_err) = obj.communicate()
         retcode = obj.returncode       
      except:
         return (-1, None, None)

This would cause an exception.

The backport of these two patches for python seems to fix the signal handling problems that were occuring:

http://svn.python.org/view/python/trunk/Lib/subprocess.py?r1=64756&r2=64755&pathrev=64756
and
http://bugs.python.org/issue1068268

Comment 8 RHEL Program Management 2011-05-31 15:45:11 UTC
This request was evaluated by Red Hat Product Management for
inclusion in the current release of Red Hat Enterprise Linux.
Because the affected component is not scheduled to be updated in the
current release, Red Hat is unfortunately unable to address this
request at this time. Red Hat invites you to ask your support
representative to propose this request, if appropriate and relevant,
in the next release of Red Hat Enterprise Linux.

Comment 14 RHEL Program Management 2013-03-15 14:45:30 UTC
Development Management has reviewed and declined this request.
You may appeal this decision by reopening this request.