Hide Forgot
Description of problem: I'm not sure of an exact repro case just yet (although you might simply be able to reboot a server while foreman is running), however attempting to restart the foreman instance first appears to work, but later is shown to have not started. The logs tell us it cannot remove the pid file. Version-Release number of selected component (if applicable): Satellite-6.0.2-RHEL-6-20130913.1 How reproducible: Steps to Reproduce: 1. Cause an unclean stoppage of foreman 2. Attempt to restart (foreman service restart or katello-service restart) 3. Observe the [OK] for foreman processes. 4. 'service foreman status' Actual results: [root@ivanova ~]# service foreman status Foreman dead but pid file exists dead but pid file exists Observed in the thin process logs: /opt/rh/ruby193/root/usr/share/gems/gems/thin-1.3.1/lib/thin/daemonizing.rb:158:in `delete': Permission denied - tmp/pids/thin.5501.pid (Errno::EACCES) from /opt/rh/ruby193/root/usr/share/gems/gems/thin-1.3.1/lib/thin/daemonizing.rb:158:in `remove_pid_file' from /opt/rh/ruby193/root/usr/share/gems/gems/thin-1.3.1/lib/thin/daemonizing.rb:59:in `block in daemonize' Expected results: * Foreman can restart itself cleanly (and remove its pid files) * Foreman shouldn't tell us [OK] if it hasn't actually started * I'm not really sure we should be seeing pid files in multiple locations ('/var/run/foreman/pids/', '/usr/share/foreman/tmp/pids/') Additional info: Manual removal of pids _may_ solve things, but that's messy
/usr/share/foreman/tmp/pids/ is owned by root. modifying it to foreman.foreman allows foreman to remove the pids.
I can't reproduce this upstream using ruby193-rubygem-thin-1.3.1-7.el6.x86_64. It appears that the PID file should be changed while thin is running as root, so I can't see what would prevent it. Have you checked for any SELinux AVCs at the time of the permission denied message? [root@rhel6b ~]# service foreman start Starting foreman: Starting server on 0.0.0.0:1234 ... [ OK ] [root@rhel6b ~]# ll ~foreman/tmp/pids/ total 4 -rw-r--r--. 1 root root 4 Sep 19 06:59 thin.1234.pid [root@rhel6b ~]# pkill -9 thin [root@rhel6b ~]# service foreman status Foreman dead but pid file exists [root@rhel6b ~]# service foreman start Starting foreman: Starting server on 0.0.0.0:1234 ... >> Deleting stale PID file tmp/pids/thin.1234.pid [ OK ] [root@rhel6b ~]# cat ~foreman/tmp/pids/thin.1234.pid; echo 6858 [root@rhel6b ~]# ps auxww | grep [t]hin foreman 6858 90.7 12.0 442984 122924 ? Sl 06:59 0:12 thin server (0.0.0.0:1234) [root@rhel6b ~]# cat /var/log/foreman/thin.1234.log >> Writing PID to tmp/pids/thin.1234.pid >> Changing process privilege to foreman:foreman >> Thin web server (v1.3.1 codename Triple Espresso) >> Maximum connections set to 1024 >> Listening on 0.0.0.0:1234, CTRL+C to stop >> Writing PID to tmp/pids/thin.1234.pid >> Changing process privilege to foreman:foreman >> Thin web server (v1.3.1 codename Triple Espresso) >> Maximum connections set to 1024 >> Listening on 0.0.0.0:1234, CTRL+C to stop Regarding the two paths, they should be equivalent. /usr/share/foreman/tmp is defined as a symlink to /var/run/foreman in the RPM, else that's a bug. I'd like to treat this as low priority anyway, since we plan on adopting Passenger for MDP3 and this isn't an issue under normal operation.
+1 to moving this to MDP3 and just switching over to Passenger
We're using Passenger in MDP3, so httpd will be the primary service instead.