Bug 857205

Summary:	JBoss clustering broken with new scalable design
Product:	OKD	Reporter:	Bill DeCoste <wdecoste>
Component:	Containers	Assignee:	Bill DeCoste <wdecoste>
Status:	CLOSED CURRENTRELEASE	QA Contact:	libra bugs <libra-bugs>
Severity:	low	Docs Contact:
Priority:	low
Version:	2.x	CC:	jhou, missedone, ramr
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:
Fixed In Version:	>= ami-5ba41732	Doc Type:	Bug Fix
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:	2012-11-06 18:47:48 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Bill DeCoste 2012-09-13 19:07:07 UTC

Description of problem:
With gear #1 having haproxy and jboss and gear #2 having jboss ... the two jboss nodes do not cluster. The connection hooks are not sending the clustering information to the other gear.


Version-Release number of selected component (if applicable):


How reproducible:
100%


Steps to Reproduce:
1. Deploy a scalable JBoss app
2.
3.
  
Actual results:
JBoss instances do not cluster. Cluster env variables do not contain the info of both gears.


Expected results:
They should cluster and env variables should contain info for both gears

Additional info:

Comment 1 Bill DeCoste 2012-09-13 22:56:27 UTC

This is a larger issue of the application on the haproxy gear not subscribing to the correct data. Likely that the application descriptor in the broker for the haproxy gear is not correct to subscribe to the application data.

This will also require a migration

Comment 2 Bill DeCoste 2012-09-13 23:03:53 UTC

rchopra: ok, here it is in code :
[4:01pm] rchopra: Connections:
[4:01pm] rchopra:   auto-scale:
[4:01pm] rchopra:     Components: [\"proxy/haproxy-1.4\", \"web/#{framework}\"]
[4:01pm] rchopra: thats all the connections
[4:01pm] rchopra: it says haproxy connects to the web/jboss
[4:01pm] nduong left the chat room. (Quit: This computer has gone to sleep)
[4:02pm] rchopra: and does not say haproxy connects to proxy/jboss
[4:02pm] kraman: ah
[4:02pm] rchopra: that was by design then, this needs tobe updated
[4:03pm] rchopra: one more line should theoretically fix it: Components: [\"proxy/haproxy-1.4\", \"proxy/#{framework}\"]

Comment 3 Bill DeCoste 2012-09-13 23:12:35 UTC

This is also breaking embedded db connectivity for the app on the haproxy gear

Comment 4 Jianwei Hou 2012-09-20 05:31:10 UTC

Verified on devenv_2201

Steps:
1. Create a scalable jbossas-7 application
2. ssh into both of app's gears
ssh 868aebead3a0486f973633c64fa60a50.rhcloud.com
ssh 66227fcfed0c4afa9d77d12cabfaaf6e.rhcloud.com
3. Run 'env |grep -i cluster' on both gears.
On gear 1:
OPENSHIFT_JBOSS_CLUSTER_PROXY_PORT=35547
OPENSHIFT_JBOSS_CLUSTER_PORT=7600
OPENSHIFT_JBOSS_CLUSTER_REMOTING=76f3b7d105-2201t.dev.rhcloud.com:35555
OPENSHIFT_JBOSS_CLUSTER=76f3b7d105-2201t.dev.rhcloud.com[35552],jbs-2201t.dev.rhcloud.com[35547]
On gear 2:
OPENSHIFT_JBOSS_CLUSTER_PROXY_PORT=35552
OPENSHIFT_JBOSS_CLUSTER_PORT=7600
OPENSHIFT_JBOSS_CLUSTER_REMOTING=jbs-2201t.dev.rhcloud.com:35550
OPENSHIFT_JBOSS_CLUSTER=jbs-2201t.dev.rhcloud.com[35547],76f3b7d105-2201t.dev.rhcloud.com[35552]
4. Scale up this app, ssh into all gears, run 'env |grep -i cluster' on all gears again.
On gear 3:
OPENSHIFT_JBOSS_CLUSTER_PROXY_PORT=35542
OPENSHIFT_JBOSS_CLUSTER_PORT=7600
OPENSHIFT_JBOSS_CLUSTER_REMOTING=jbs-2201t.dev.rhcloud.com:35550
OPENSHIFT_JBOSS_CLUSTER=jbs-2201t.dev.rhcloud.com[35547],76f3b7d105-2201t.dev.rhcloud.com[35552],3f563f4c49-2201t.dev.rhcloud.com[35542]


Result:
Cluster env variables contain the info of all gears.

Comment 5 Ram Ranganathan 2012-09-28 00:13:58 UTC

This fix is causing other issues - see bugz: https://bugzilla.redhat.com/show_bug.cgi?id=860536

Comment 6 Ram Ranganathan 2012-09-28 00:15:06 UTC

Will keep parts of this fix to ensure 860536 is fixed -- please retest this bug once 860536 is fixed.

Comment 7 Jianwei Hou 2012-09-28 09:30:14 UTC

Seems bug 860536 is not fixed, so will verify this bug after bug 860536 is fixed.

Comment 8 Jianwei Hou 2012-09-28 11:39:40 UTC

Bug 860536 is verified so also verified this bug on devenv_2250

1. Create a scalable jbossas app 
2. ssh into app, check envs
[jas-jhoudev.dev.rhcloud.com conf]\> env|grep -i cluster
OPENSHIFT_JBOSS_CLUSTER_PROXY_PORT=35577
OPENSHIFT_JBOSS_CLUSTER_PORT=7600
OPENSHIFT_JBOSS_CLUSTER_REMOTING=c5f0788380-jhoudev.dev.rhcloud.com[35585],jas-jhoudev.dev.rhcloud.com[35580]
OPENSHIFT_JBOSS_CLUSTER=c5f0788380-jhoudev.dev.rhcloud.com[35582],jas-jhoudev.dev.rhcloud.com[35577]

3. ssh into another gear, check envs
[jas-jhoudev.dev.rhcloud.com conf]\> ssh c5f0788380274f86b93d635d564d5683.rhcloud.com
........
[c5f0788380-jhoudev.dev.rhcloud.com ~]\> env|grep -i cluster
OPENSHIFT_JBOSS_CLUSTER_PROXY_PORT=35582
OPENSHIFT_JBOSS_CLUSTER_PORT=7600
OPENSHIFT_JBOSS_CLUSTER_REMOTING=jas-jhoudev.dev.rhcloud.com[35580],c5f0788380-jhoudev.dev.rhcloud.com[35585]
OPENSHIFT_JBOSS_CLUSTER=jas-jhoudev.dev.rhcloud.com[35577],c5f0788380-jhoudev.dev.rhcloud.com[35582]

Cluster env variables contain the info of all gears.