Description of problem: HAProxy doesn't load the new configuration after scaling out the role running the Openstack API services. I'm deploying the Openstack API services on a custom role and starting with a deployment with 3 nodes running this role. After the deployment is done I'm trying to add an additional node and I can see haproxy.cfg gets updates with all 4 nodes but HAProxy stats page only shows the initial 3 backends. Version-Release number of selected component (if applicable): openstack-tripleo-heat-templates-5.0.0-1.1.el7ost.noarch How reproducible: 100% Steps to Reproduce: 1. Deploy overcloud with 3 nodes running the Openstack API services. Deploy command and environment files: http://paste.openstack.org/show/587619/ 2. Increment ServiceApiCount and rerun deploy command Actual results: Update completes ok and haproxy.cfg contains the new backend node but HAProxy stats page reports only the initial 3 nodes as backends instead of 4. Expected results: The new HAProxy config gets loaded. Additional info: 'systemctl reload haproxy' is needed on all controllers so it loads the new config. This symptom looks pretty similar to the one in bug 1320379 but I need someone to confirm this. Thanks.
Created attachment 1216454 [details] haproxy.cfg Attaching the haproxy.cfg and the HAProxy stats page.
Created attachment 1216455 [details] HAProxy stats
Deploying it my local env to confirm.
Just omit last 2 comments, summarized here: Scaling out a controller node haproxy is not reloaded correctly, the new config is correct but needs to be reloaded. `systemctl reload haproxy` fix the issue after the update. Added here: https://github.com/openstack/tripleo-heat-templates/commit/843d25af04f58d033e12c2c3d619303d7fd8bb02 Removed here (in favor of solving https://bugzilla.redhat.com/show_bug.cgi?id=1321036): https://github.com/openstack/tripleo-heat-templates/commit/6e56f873148784ff34babf62a8ccc718e7d789d3 The restart should be executed in the converge step but not able to find it for a non-upgrade task after finishing the deployment, this is only about deploy again the overcloud adding 1 more node.. `systemctl reload haproxy` executed only when injecting the TLS cert and when running pacemaker_maintenance_mode.sh (called from pre_puppet_pacemaker.yaml) To summarize: Before was executed in post-deployment step now it's in a pre-deployment step...
Added an upstream patch for testing.
After testing the patch, hitting this new upstream issue when started Mistral Workflow, it fails due to a malformed template. https://bugs.launchpad.net/tripleo/+bug/1639302
Marius the upstream patch I believe will be landed soon, https://review.openstack.org/#/c/393644/ mind to check it?
Also added a new reference to this current error in Launchpad.
(In reply to Carlos Camacho from comment #9) > Marius the upstream patch I believe will be landed soon, > https://review.openstack.org/#/c/393644/ mind to check it? Looks good, I tested it on my environment and reloads the HAProxy new config.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHEA-2016-2948.html