Bug 1255747 - salt-minion won't start anymore after system crashes
salt-minion won't start anymore after system crashes
Status: NEW
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: Calamari (Show other bugs)
1.3.0
All Linux
unspecified Severity medium
: rc
: 1.3.4
Assigned To: Gregory Meno
ceph-qe-bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-08-21 08:44 EDT by Gerald Sternagl
Modified: 2017-12-08 12:57 EST (History)
4 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Gerald Sternagl 2015-08-21 08:44:00 EDT
Description of problem:
After a power outage of our Ceph cluster none of the Calamari graphs for OSDs were displaying anymore. We figured out that the salt-minion daemon and therefor the diamond data collector process weren't able to start anymore on these systems because a left-over timestamp file (/var/cache/salt/minion/proc/20150820122333837170) was causing /usr/bin/salt-minion to crash at start-up. Deleting this file and restarting /usr/bin/salt-minion solved the issue. The file get regularly created and deleted but in the case it can't get deleted in time due to a system crash it prevents salt-minion to start.

Version-Release number of selected component (if applicable):
RHCS 1.3


How reproducible:
stop salt-minion, create file under /var/cache/salt/minion/proc/20150820122333837170, try to restart salt-minion.

Steps to Reproduce:
1. systemctl stop salt-minion.service
2. touch /var/cache/salt/minion/proc/20150820122333837170
3. systemctl start salt-minion.service

Actual results:
salt-minion.service start fails.

Expected results:
System startup should clear all left over status files / timestamps / sockets / etc. which are not required or cause startup of service like e.g. salt-minion to fail.


Additional info:
Comment 2 Gregory Meno 2015-12-10 18:05:53 EST
Gregory is this limited to power outage? What happens during a normal shutdown?
Comment 3 Gregory Meno 2016-10-06 14:48:45 EDT
Seems to be limited to abnormal terminaton

Note You need to log in before you can comment on or make changes to this bug.