Description of problem:
HAProxy doesn't load the new configuration after scaling out the role running the Openstack API services. I'm deploying the Openstack API services on a custom role and starting with a deployment with 3 nodes running this role. After the deployment is done I'm trying to add an additional node and I can see haproxy.cfg gets updates with all 4 nodes but HAProxy stats page only shows the initial 3 backends.
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Deploy overcloud with 3 nodes running the Openstack API services. Deploy command and environment files:
2. Increment ServiceApiCount and rerun deploy command
Update completes ok and haproxy.cfg contains the new backend node but HAProxy stats page reports only the initial 3 nodes as backends instead of 4.
The new HAProxy config gets loaded.
'systemctl reload haproxy' is needed on all controllers so it loads the new config.
This symptom looks pretty similar to the one in bug 1320379 but I need someone to confirm this. Thanks.
Created attachment 1216454 [details]
Attaching the haproxy.cfg and the HAProxy stats page.
Created attachment 1216455 [details]
Deploying it my local env to confirm.
Just omit last 2 comments, summarized here:
Scaling out a controller node haproxy is not reloaded correctly, the new config is correct but needs to be reloaded.
`systemctl reload haproxy` fix the issue after the update.
Removed here (in favor of solving https://bugzilla.redhat.com/show_bug.cgi?id=1321036):
The restart should be executed in the converge step but not able to find it for a non-upgrade task after finishing the deployment, this is only about deploy again the overcloud adding 1 more node..
`systemctl reload haproxy` executed only when injecting the TLS cert and when running pacemaker_maintenance_mode.sh (called from pre_puppet_pacemaker.yaml)
To summarize: Before was executed in post-deployment step now it's in a pre-deployment step...
Added an upstream patch for testing.
After testing the patch, hitting this new upstream issue when started Mistral Workflow, it fails due to a malformed template.
Marius the upstream patch I believe will be landed soon, https://review.openstack.org/#/c/393644/ mind to check it?
Also added a new reference to this current error in Launchpad.
(In reply to Carlos Camacho from comment #9)
> Marius the upstream patch I believe will be landed soon,
> https://review.openstack.org/#/c/393644/ mind to check it?
Looks good, I tested it on my environment and reloads the HAProxy new config.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.