Bug 1058310

Summary: tmpwatch removes python multiprocessing sockets
Product: Red Hat Enterprise Linux 6 Reporter: Chris St. Pierre <cstpierr>
Component: tmpwatchAssignee: Miloslav Trmač <mitr>
Status: CLOSED ERRATA QA Contact: Eva Mrakova <emrakova>
Severity: unspecified Docs Contact:
Priority: high    
Version: 6.6CC: emrakova, mitr, salmy
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: tmpwatch-2.9.16-6.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1058482 (view as bug list) Environment:
Last Closed: 2016-05-10 19:43:10 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1058482    
Bug Blocks: 947782, 1159825    
Attachments:
Description Flags
Script to help reproduce the issue
none
Patch for /etc/cron.d/tmpwatch that fixes the issue none

Description Chris St. Pierre 2014-01-27 13:49:35 UTC
Created attachment 856072 [details]
Script to help reproduce the issue

Description of problem:

The Python multiprocessing library (http://docs.python.org/2/library/multiprocessing.html) can open an ephemeral socket for interprocess communication based on a temporary file.  It creates these temp files in /tmp/pymp-XXXXXX.  However, the atomic temp file creation does not ever list the directory, so its atime is not updated.  However, the pymp-* directory is only generated once, when the program is started.

As a result, if a Python program runs for more than ten days (the default cleanup time for /tmp in /etc/cron.d/tmpwatch), creates sockets (e.g., with multiprocessing.connnection.Listener()), and has no open sockets when tmpwatch runs, the pymp-* directory will be removed, and the next time the Python program attempts to open a socket it will fail spectacularly.

Version-Release number of selected component (if applicable):

tmpwatch-2.9.16-4.el6.x86_64
python-2.6.6-51.el6.x86_64
python-libs-2.6.6-51.el6.x86_64

How reproducible:

Always.

Steps to Reproduce:

create-socket-and-sleep.py is attached.  Run:

./create-socket-and-sleep.py 864010 & sleep 864005; bash /etc/cron.daily/tmpwatch

This takes ten days to run, so if you want something quicker, you can run:

./create-socket-and-sleep.py 70 & sleep 65;/usr/sbin/tmpwatch -x /tmp/.X11-unix -x /tmp/.XIM-unix -x /tmp/.font-unix         -x /tmp/.ICE-unix -x /tmp/.Test-unix 1m /tmp

The tmpwatch command there is taken directly from the stock /etc/cron.daily/tmpwatch, with '240' (hours) changed to '1m' so that this can be run in a timely fashion.

Actual results:

% ./create-socket-and-sleep.py 70 & sleep 65;/usr/sbin/tmpwatch -x /tmp/.X11-unix -x /tmp/.XIM-unix -x /tmp/.font-unix         -x /tmp/.ICE-unix -x /tmp/.Test-unix 1m /tmp
[1] 7361
/tmp/pymp-oki8QE/listener-AA8c0N
Traceback (most recent call last):
  File "./create-socket-and-sleep.py", line 19, in <module>
    sys.exit(main())
  File "./create-socket-and-sleep.py", line 15, in main
    l = Listener()
  File "/usr/lib64/python2.6/multiprocessing/connection.py", line 106, in __init__
    self._listener = SocketListener(address, family, backlog)
  File "/usr/lib64/python2.6/multiprocessing/connection.py", line 227, in __init__
    self._socket.bind(address)
  File "<string>", line 1, in bind
socket.error: [Errno 2] No such file or directory
Traceback (most recent call last):
  File "/usr/lib64/python2.6/multiprocessing/util.py", line 235, in _run_finalizers
    finalizer()
  File "/usr/lib64/python2.6/multiprocessing/util.py", line 174, in __call__
    res = self._callback(*self._args, **self._kwargs)
  File "/usr/lib64/python2.6/shutil.py", line 204, in rmtree
    onerror(os.listdir, path, sys.exc_info())
  File "/usr/lib64/python2.6/shutil.py", line 202, in rmtree
    names = os.listdir(path)
OSError: [Errno 2] No such file or directory: '/tmp/pymp-oki8QE'

Expected results:

The file names of two temp files should be printed.

Additional info:

Comment 1 Chris St. Pierre 2014-01-27 13:50:35 UTC
Created attachment 856073 [details]
Patch for /etc/cron.d/tmpwatch that fixes the issue

A patch for /etc/cron.d/tmpwatch that fixes the issue is attached.

Comment 3 Miloslav Trmač 2014-01-27 21:17:10 UTC
Thanks for your report and the patch.

I'm not... thrilled... about the way multiprocessing uses the temporary directory, but we have a precedent in doing the same for bug #527425 .

Needs to be fixed for RHEL 7 first, though, to avoid a regression on update,

Comment 12 errata-xmlrpc 2016-05-10 19:43:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0754.html