Bug 1288211 - Native HA deployment cannot find leader
Summary: Native HA deployment cannot find leader
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 3.1.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Andrew Butcher
QA Contact: Jianwei Hou
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-12-03 20:47 UTC by Eric Jones
Modified: 2019-09-12 09:27 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-01-04 20:01:47 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Eric Jones 2015-12-03 20:47:25 UTC
Description of problem:
Attempted to restart atomic-openshift-master-api.service and atomix-openshift-master-controllers.service in order to confirm changes to the master-config.yaml, once the restart finished
Version-Release number of selected component (if applicable):
OSE 3.1, brand new installation as of 2015/12/2

How reproducible:
Seems to be 100%

Steps to Reproduce:
1. Edit master-config.yaml
2. Restart atomic-openshift-master-api.service and atomix-openshift-master-controllers.service 

Actual results:
the masters seem to have no "leader" for the system. The logs for the  atomic-openshift-master-controllers.service all indicate
"Attempting to acquire controller lease as master-##thisid##
 Lease openshift.io/leases/controllers owned by master-##differentid## at ...."

The owner id did not match any of the ids with which the masters are trying to use to become the leader. All three masters continue to attempt this until they either timeout or give up. 

The logs continue with
"...unable to wait for lease expiration openshift.io/leases/controllers: 401: The event in requested index is outdated and cleared (the requested history has been cleared"

Expected results:
Everything to return to normal with the changes in the master-config.yaml made.

Additional info:
3 Masters
The change to the master-config.yaml was to the identity provider. Originally setup using htpasswd but changed to LDAP.


Note You need to log in before you can comment on or make changes to this bug.