Red Hat Bugzilla – Bug 822476
Embedded haproxy stops and then cannot restart when deploying to JBoss app
Last modified: 2015-05-14 18:54:14 EDT
Description of problem:
Upon deploying code to a running scaled JBoss application, haproxy is restarted. However, haproxy fails to start after it has been stopped, leaving the front end of the application inaccessible.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Create a scalable JBoss application
2. Scale the application up to two nodes
3. Verify the default application is accessible from the front-end URL
4. Deploy a clustered WAR via git push
The deployment is successful, but the application is inaccessible from the public URL, and haproxy is no longer running.
A clean restart with haproxy alive and well.
Debugged this with Mike McGrath. It appears the stop script for haproxy is leaving a pid file on the filesystem which the start script refuses to overwrite, preventing startup. For example, using the stop script directly should produce the same bad state:
service libra stopuser cbaf254d39e7484b8eb71b90ba53f8fb
The problem may be isolated to:
Fixed with 5972dc780cd282949f591d28512ac804e86e6a20 -- issue was turning off selinux causes the system-wide haproxy daemon to show up in the list of processes for the user -- this fix makes that check more resilient.
Verified this bug with devenv_1780, and PASS.
1). Launch devenv_1780 instance, and replace app_ctl.sh with old one.
2). Create a scalable app.
3). setenforce 0
4). Use the following command to produce the bad state:
# service libra stopuser <haproxy_gear_uuid>
Actually this step is not necessary, due to the first thing of git push is to stop application.
5). Do git push to deploy new app.
6). Now the application is inaccessible from the public URL.
Replace app_ctl.sh with new one, do git push again. Now the application is accessible from the public URL.