Bug 1288211 - Native HA deployment cannot find leader
Native HA deployment cannot find leader
Product: OpenShift Container Platform
Classification: Red Hat
Component: Pod (Show other bugs)
Unspecified Unspecified
high Severity high
: ---
: ---
Assigned To: Andrew Butcher
Jianwei Hou
: Unconfirmed
Depends On:
  Show dependency treegraph
Reported: 2015-12-03 15:47 EST by Eric Jones
Modified: 2016-01-04 15:02 EST (History)
7 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2016-01-04 15:01:47 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)

  None (edit)
Description Eric Jones 2015-12-03 15:47:25 EST
Description of problem:
Attempted to restart atomic-openshift-master-api.service and atomix-openshift-master-controllers.service in order to confirm changes to the master-config.yaml, once the restart finished
Version-Release number of selected component (if applicable):
OSE 3.1, brand new installation as of 2015/12/2

How reproducible:
Seems to be 100%

Steps to Reproduce:
1. Edit master-config.yaml
2. Restart atomic-openshift-master-api.service and atomix-openshift-master-controllers.service 

Actual results:
the masters seem to have no "leader" for the system. The logs for the  atomic-openshift-master-controllers.service all indicate
"Attempting to acquire controller lease as master-##thisid##
 Lease openshift.io/leases/controllers owned by master-##differentid## at ...."

The owner id did not match any of the ids with which the masters are trying to use to become the leader. All three masters continue to attempt this until they either timeout or give up. 

The logs continue with
"...unable to wait for lease expiration openshift.io/leases/controllers: 401: The event in requested index is outdated and cleared (the requested history has been cleared"

Expected results:
Everything to return to normal with the changes in the master-config.yaml made.

Additional info:
3 Masters
The change to the master-config.yaml was to the identity provider. Originally setup using htpasswd but changed to LDAP.

Note You need to log in before you can comment on or make changes to this bug.