Bugzilla (bugzilla.redhat.com) will be under maintenance for infrastructure upgrades and will not be available on July 31st between 12:30 AM - 05:30 AM UTC. We appreciate your understanding and patience. You can follow status.redhat.com for details.
Bug 1564128 - Starter-us-east-2 3.7 masters failing to come up with 3.9 predicate in config
Summary: Starter-us-east-2 3.7 masters failing to come up with 3.9 predicate in config
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: OpenShift Online
Classification: Red Hat
Component: Pod
Version: 3.x
Hardware: Unspecified
OS: Linux
unspecified
urgent
Target Milestone: ---
: ---
Assignee: Avesh Agarwal
QA Contact: DeShuai Ma
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-04-05 12:33 UTC by Paul Bergene
Modified: 2018-04-09 07:22 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-09 07:22:35 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Paul Bergene 2018-04-05 12:33:59 UTC
Description of problem:
Masters failing on starter-us-east-2.  Filing a bz after #libra-ops discussion.

Version-Release number of selected component (if applicable):
OpenShift Master: v3.7.23 (online version 3.7.2.1)


The masters on starter-us-east-2 seem to be failing to come up or are in a bad state. Looking at atomic-openshift-master-controllers logs there is the following error: 

Apr 05 11:52:30 ip-172-31-67-105.us-east-2.compute.internal atomic-openshift-master-controllers[25092]: F0405 11:52:30.145529   25092 plugins.go:150] Invalid configuration: Predicate type not found for CheckVolumeBinding

Apr 05 11:52:30 ip-172-31-67-105.us-east-2.compute.internal systemd[1]: atomic-openshift-master-controllers.service: main process exited, code=exited, status=255/n/a
Apr 05 11:52:30 ip-172-31-67-105.us-east-2.compute.internal systemd[1]: Unit atomic-openshift-master-controllers.service entered failed state.

Predicate CheckVolumeBinding seems only to be referenced in 3.9 documentation, suggesting that a 3.9 config had been applied to this cluster.  There was maintenance peformed yesterday evening preceding this issue.

Comment 1 Avesh Agarwal 2018-04-05 13:30:16 UTC
Right now I dont know the root cause, but my suggestion is to remove CheckVolumeBinding from scheduler's policy file, and it wont effect anything because CheckVolumeBinding is a default predicate and is registered by default anyway, so there is not need to put it again in the policy file where it is failing.

Also the feature VolumeScheduling is disabled in 3.9 (alpha) which is used with CheckVolumeBinding, so the predicate CheckVolumeBinding does not do anything anyway.

Comment 2 Avesh Agarwal 2018-04-05 13:31:23 UTC
I will keep looking into it meanwhile why it is happening but i know that it does not happen always.

Comment 3 Avesh Agarwal 2018-04-05 13:39:46 UTC
I did not realize that the version was 3.7, i thought it was 3.9. The predicate CheckVolumeBinding was added in 3.9 so should not affect 3.7.

Please check why the scheduler policy file has CheckVolumeBinding in 3.7?

Comment 4 Paul Bergene 2018-04-06 08:00:45 UTC
This should now be resolved.

Comment 5 Avesh Agarwal 2018-04-06 15:47:43 UTC
can we close this bz if its resolved?


Note You need to log in before you can comment on or make changes to this bug.