Bug 822476

Summary:	Embedded haproxy stops and then cannot restart when deploying to JBoss app
Product:	OKD	Reporter:	Dan Mace <dmace>
Component:	Containers	Assignee:	Dan Mace <dmace>
Status:	CLOSED CURRENTRELEASE	QA Contact:	libra bugs <libra-bugs>
Severity:	low	Docs Contact:
Priority:	low
Version:	2.x	CC:	jialiu, ramr
Target Milestone:	---	Keywords:	Triaged
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	> cartridge-haproxy-0.10.2-1	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-06-08 17:59:40 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Dan Mace 2012-05-17 13:12:01 UTC

Description of problem:

Upon deploying code to a running scaled JBoss application, haproxy is restarted. However, haproxy fails to start after it has been stopped, leaving the front end of the application inaccessible.

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Create a scalable JBoss application
2. Scale the application up to two nodes
3. Verify the default application is accessible from the front-end URL
4. Deploy a clustered WAR via git push
  
Actual results:

The deployment is successful, but the application is inaccessible from the public URL, and haproxy is no longer running.

Expected results:

A clean restart with haproxy alive and well.

Additional info:

Debugged this with Mike McGrath. It appears the stop script for haproxy is leaving a pid file on the filesystem which the start script refuses to overwrite, preventing startup. For example, using the stop script directly should produce the same bad state:

service libra stopuser cbaf254d39e7484b8eb71b90ba53f8fb

The problem may be isolated to:

/usr/libexec/stickshift/cartridges/embedded/haproxy-1.4/info/bin/app_ctl.sh

Comment 1 Ram Ranganathan 2012-05-17 21:19:20 UTC

Fixed with 5972dc780cd282949f591d28512ac804e86e6a20  -- issue was turning off selinux causes the system-wide haproxy daemon to show up in the list of processes for the user -- this fix makes that check more resilient.

Comment 2 Johnny Liu 2012-05-18 11:29:21 UTC

Verified this bug with devenv_1780, and PASS.

Reproduce steps:
1). Launch devenv_1780 instance, and replace app_ctl.sh with old one.
2). Create a scalable app.
3). setenforce 0
4). Use the following command to produce the bad state:
# service libra stopuser <haproxy_gear_uuid>
**NOTE**:
Actually this step is not necessary, due to the first thing of git push is to stop application.
5). Do git push to deploy new app. 
6). Now the application is inaccessible from the public URL.

Replace app_ctl.sh with new one, do git push again. Now the application is accessible from the public URL.