Bug 1202858

Summary: [UNRELEASED] restarting testing build of squid results in deleting all files in hard-drive
Product: Red Hat Enterprise Linux 6 Reporter: Swapna Krishnan <skrishna>
Component: squidAssignee: Luboš Uhliarik <luhliari>
Status: CLOSED ERRATA QA Contact: Ondřej Pták <optak>
Severity: urgent Docs Contact:
Priority: urgent    
Version: 6.7CC: admiller, agk, apetrova, btotty, charlezdelaney, emadahmed1995, germano.massullo, ggainey, jshubin, jwildebo, kdudka, lkocman, mail, marty, mattdm, mbooth, mgrepl, optak, ovasik, pmutha, rbiba, redhat-bugzilla, rjones, rupatel, rvokal, socketpair, steve.traylen, thozza, vanhoof, vlad, vpvainio, yuhongbao_386, zaezzz
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: squid-3.1.23-5.el6 Doc Type: Bug Fix
Doc Text:
During the testing phase, it was discovered that restarting squid caused all files on the system to be deleted. Red Hat has fixed the bug before it could affect any users of squid. As a result, restarting squid does not cause any files on the system to be deleted. This bug was never released neither as part of Red Hat Enterprise Linux, nor as part of any upstream version of the squid packages. For more information about the bug, see the Knowledgebase Solution linked to in the References section.
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-22 06:28:34 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Swapna Krishnan 2015-03-17 15:05:03 UTC
###############################################################
# PLEASE NOTE:                                                #
#                                                             #
# At the time of writing, RHEL 6.7 is still pre-beta and this #
# bug was found in an *UNRELEASED* update to squid.           #
# It has been fixed before RHEL 6.7 ships.                    #
###############################################################

Description of problem:
After install of test packages for RHEL 6.7, when I try to stop squid or restart squid when it's already running, it keeps waiting to stop and then suddenly starts deleting all files on the hard-drive. It's hard to diagnose anything more at that point because a lot of files on the machine are deleted. 

<snip>
[root@tyan-gt24-04 ~]# service squid status
squid is stopped
[root@tyan-gt24-04 ~]# service squid start
Starting squid: .[  OK  ]
[root@tyan-gt24-04 ~]# 
[root@tyan-gt24-04 ~]# service squid status
squid (pid  9122) is running...
[root@tyan-gt24-04 ~]# service squid restart
Stopping squid: ...........
Stopping squid: ................[  OK  ]
rm: cannot remove `/boot': Device or resource busy
rm: cannot remove `/cgroup/cpuset/libvirt/lxc/cpuset.memory_spread_slab': Operation not permitted
rm: cannot remove `/cgroup/cpuset/libvirt/lxc/cpuset.memory_spread_page': Operation not permitted
rm: cannot remove `/cgroup/cpuset/libvirt/lxc/cpuset.memory_pressure': Operation not permitted
rm: cannot remove `/cgroup/cpuset/libvirt/lxc/cpuset.memory_migrate': Operation not permitted
.....
rm: cannot remove `/dev/shm': Device or resource busy
rm: cannot remove `/home': Device or resource busy
</snip>

and it just hung here. 

On another machine, after it started deleting files, it ended with some error messages.

<snip>
rm: cannot remove `/var/lib/nfs/rpc_pipefs/portmap': Operation not permitted
rm: cannot remove `/var/lib/nfs/rpc_pipefs/nfs': Operation not permitted
rm: cannot remove `/var/lib/nfs/rpc_pipefs/mount': Operation not permitted
rm: cannot remove `/var/lib/nfs/rpc_pipefs/lockd': Operation not permitted
/etc/init.d/squid: line 53: id: command not found
/etc/init.d/squid: line 53: [: -ne: unary operator expected
[root@sgi-xe500-01 ~]# 
[root@sgi-xe500-01 ~]# 
</snip>

Version-Release number of selected component (if applicable):
squid.x86_64 7:3.1.23-4.el6

How reproducible:
100%

Steps to Reproduce:
1. Login to RHEL 6.7 host with squid installed
2. Start squid using "service squid start"
3. Restart squid using "service squid restart"


Actual results:
All files are deleted on the machine.

Expected results:
Squid is restarted.

Additional info:

Comment 2 Pavel Šimerda (pavlix) 2015-03-18 10:20:23 UTC
Confirmed in a virtual machine.

Comment 8 superusersdo 2015-03-24 06:56:41 UTC
Hello,

There was a bug in steam for linux that also resulted in removal all all files from the root directory due to symlink nonalignment.

https://github.com/valvesoftware/steam-for-linux/issues/3671

I have not looked at any of your code but I hope that this is close to the same solution,


 cheers!

Comment 9 miglen.com 2015-03-24 08:01:55 UTC
It's reproducible, confirmed on VM. Hope it will get fixed soon so we can patch our environment.

Comment 13 Pavel Šimerda (pavlix) 2015-03-24 08:41:16 UTC
Hello,

package containing this bug has never been released.

Cheers,

Pavel

Comment 24 Radek Vokál 2015-03-24 11:10:31 UTC
Thanks Swapna and Red Hat QE for catching this issue before the package was released. Great work!

Comment 31 Adam Miller 2015-03-24 13:48:25 UTC
This bug has for one reason or another started to become popular on social media so for the sake of personal sanity I would like to post this here so that I can stop replying to posts about it around the internet.

##############################################################
##############################################################
# NOTE
######
# At the time of this writing, RHEL 6.7 is still pre-beta and
# this bug was found in an *UNRELEASED* update to squid.
##############################################################
##############################################################

I would like to echo Radek Vokal's sentiments and offer thanks to the team for finding and resolving this before it was ever released.

Thank you,
-AdamM

Comment 39 Yuhong Bao 2015-03-24 18:09:50 UTC
Also see https://bugzilla.redhat.com/show_bug.cgi?id=1102343 for where the problem was.

Comment 44 Ondřej Pták 2015-03-26 14:02:22 UTC
Verified that this bug is correctly fixed in squid-3.1.23-5.el6.
Note that this rpm is not publicly available at this time, and further testing will be performed on it.  RHEL-6.7 release is still under testing. As you can see bugs can be everywhere and that's the reason we care a lot about testing packages before releasing them, especially in case of widely used programs like squid.

Comment 46 Ondřej Pták 2015-03-30 13:28:25 UTC
Results from automated test:

squid-3.1.23-4.el6
==================
no test results were collected because of broken environment, but many messages like these can be found in log:

/usr/bin/rhts-test-runner.sh: line 197: grep: command not found  
/usr/bin/rhts-test-runner.sh: line 197: ps: command not found  
/usr/bin/rhts-test-runner.sh: line 257: awk: command not found  
/usr/bin/rhts_environment.sh: /usr/bin/rhts-report-result: /bin/bash: bad interpreter: No such file or directory  
/usr/bin/rhts-test-runner.sh: line 52: /bin/logger: No such file or directory  
/usr/bin/rhts-test-runner.sh: line 39: cp: command not found  
IOError: [Errno 2] No such file or directory: '/mnt/testarea/rhts/TESTOUT.log'  
/usr/bin/rhts-test-runner.sh: line 48: mv: command not found  
/usr/bin/rhts-test-runner.sh: line 57: rm: command not found  
/usr/bin/rhts-test-runner.sh: line 58: cat: command not found  
expr: error while loading shared libraries: libc.so.6: cannot open shared object file: No such file or directory  
/usr/bin/rhts-test-runner.sh: /usr/bin/rhts-sync-set: /usr/bin/python: bad interpreter: No such file or directory  
/usr/bin/rhts-test-runner.sh: line 64: /bin/touch: No such file or directory  
/usr/bin/rhts-test-runner.sh: line 74: /bin/logger: No such file or directory  


squid-3.1.23-5.el6
==================
:: [   INFO   ] :: Trying all initscript actions
:: [   PASS   ] :: service squid start (Expected 0, got 0)
:: [   PASS   ] :: service squid restart (Expected 0, got 0)
:: [   PASS   ] :: service squid status (Expected 0, got 0)
:: [   PASS   ] :: service squid reload (Expected 0, got 0)
:: [   PASS   ] :: service squid force-reload (Expected 0, got 0)
:: [   PASS   ] :: service squid condrestart (Expected 0, got 0)
:: [   PASS   ] :: service squid try-restart (Expected 0, got 0)
:: [   PASS   ] :: service squid probe (Expected 0, got 0)
:: [   PASS   ] :: service squid stop (Expected 0, got 0)
:: [   PASS   ] :: Waiting 30m (Expected 0, got 0)
:: [   PASS   ] :: Directory /home should exist 
:: [   PASS   ] :: Directory /etc should exist 
:: [   PASS   ] :: Directory /var should exist 
:: [   PASS   ] :: Directory /usr should exist 
:: [   PASS   ] :: Directory /sbin should exist 
:: [   PASS   ] :: Directory /bin should exist 
:: [   PASS   ] :: Directory /boot should exist

Comment 48 errata-xmlrpc 2015-07-22 06:28:34 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1314.html