Bug 1260142 - Heat fails while installing with OSPd a 2 nodes controller setup
Heat fails while installing with OSPd a 2 nodes controller setup
Status: CLOSED WORKSFORME
Product: Red Hat OpenStack
Classification: Red Hat
Component: rhosp-director (Show other bugs)
7.0 (Kilo)
Unspecified Unspecified
low Severity unspecified
: ---
: 8.0 (Liberty)
Assigned To: Hugh Brock
yeylon@redhat.com
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-09-04 10:13 EDT by Raoul Scarazzini
Modified: 2016-04-26 12:12 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-04-05 01:33:25 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Raoul Scarazzini 2015-09-04 10:13:42 EDT
Description of problem:

If you want to install an unusual setup via OSPd, having just 2 controller nodes, even if the process starts it fails.

Version-Release number of selected component (if applicable):

openstack-heat-templates-0-0.6.20150605git.el7ost.noarch

How reproducible:

1. Prepare the OSPd environment as usual;
2. Start the overcloud deploy by passing a control-scale of 2:

openstack overcloud deploy --templates --control-scale 2 --control-flavor vm --compute-scale 2 --compute-flavor baremetal  --ntp-server 10.16.255.1

3. Wait a little bit, then it fails;

Actual results:

The error is not specific, it's just a simple HEAT FAILED.

Expected results:

Success.

Additional info:

Investigating on the problem reveals that it's a matter of quorum. At some time the setup do this command:

/usr/sbin/pcs status | grep -q 'partition with quorum' > /dev/null 2>&1

(see openstack-puppet/modules/pacemaker/manifests/corosync.pp) and since the cluster is composed by just two nodes there is NO quorum.
This might take us to one of these choices:

1 - Prohibit a two controllers setup;
2 - Do not check the quorum on the cluster, setting also a property like no-quorum-policy to "ignore";
Comment 6 Andrew Beekhof 2016-03-31 20:02:07 EDT
pcs will (now?) set the relevant "two_node" options in corosync.conf, so i would expect the cluster to have quorum if both nodes can see each other.

come to think of it, even without those options the cluster should have quorum if both node can see each other.
Comment 7 Andrew Beekhof 2016-04-03 19:30:04 EDT
Raoul: did you ever save the logs from a failed installation?
I'd be quite keen to look at them
Comment 8 Raoul Scarazzini 2016-04-04 03:02:07 EDT
No, I did not take any log at the time I got the problem. Now it does not happen anymore so unfortunately I cannot give you anything.
If it will happen again I'll surely do.
Comment 9 Andrew Beekhof 2016-04-04 03:29:37 EDT
What if we try installing with older cluster and/or director versions?
Comment 10 Raoul Scarazzini 2016-04-04 05:46:39 EDT
It would be a huge job, since even if we have the openstack-heat-templates version, we don't know what puppet modules were installed and neither the status of the RHEL at that time (so the pcs version and so on).
Comment 11 Andrew Beekhof 2016-04-04 18:29:03 EDT
Sounds like we should close this then
Comment 12 Michele Baldessari 2016-04-05 01:33:25 EDT
Agreed, let's close this one out. We've been deploying this two-node scenario many times in the last few days and it always worked.

Note You need to log in before you can comment on or make changes to this bug.