Bug 1577955 - Router pod fails to deploy during installation
Summary: Router pod fails to deploy during installation
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Installer
Version: 3.10.0
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
: 3.10.z
Assignee: Stephen Cuppett
QA Contact: Johnny Liu
URL:
Whiteboard:
: 1577959 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-05-14 13:55 UTC by Preetesh Sharma
Modified: 2018-11-11 16:40 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-11-11 16:39:10 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:2709 0 None None None 2018-11-11 16:40:08 UTC

Description Preetesh Sharma 2018-05-14 13:55:09 UTC
Description of problem:

when installation task "[openshift_hosted : Poll for OpenShift pod deployment success]" is executed it tries to deploy router pods in all infra region nodes and during this time it also tries to deploy router pods in master nodes, in which by default "infra" region is selected, but as master servers are set to unschedulable these task fails.

Error message in verbose -
~~~
"_raw_params": "oc get replicationcontroller router-20 --namespace default --config /etc/origin/master/admin.kubeconfig -o jsonpath='{ .metadata.annotations.openshift\\.io/deployment\\.phase }'

                "images": "openshift3/ose-${component}:${version}", 
                "name": "router", 
                "namespace": "default", 
                "ports": [
                    "80:80", 
                    "443:443"
                ], 
                "replicas": "5", 
                "selector": "region=infra", 
                "serviceaccount": "router", 
                "stats_port": 1936
~~~

How reproducible: 
Steps to Reproduce:

1. Setup multi master installation with three master and three nodes, assign "infra" region to two nodes in inventory file.
2. Installer will try to deploy router pods on 5 nodes, and will finish with below message.
~~~
Failure summary:


  1. Hosts:    redhat.example.com
     Play:     Create Hosted Resources - router
     Task:     Poll for OpenShift pod deployment success
     Message:  All items completed
~~~

Actual results:
Please include the entire output from the last TASK line through the end of output if an error is generated

Get below error message for task - "[openshift_hosted : Poll for OpenShift pod deployment success]".
~~~ 
"_raw_params": "oc get replicationcontroller router-20 --namespace default --config /etc/origin/master/admin.kubeconfig -o jsonpath='{ .metadata.annotations.openshift\\.io/deployment\\.phase }'

                "images": "openshift3/ose-${component}:${version}", 
                "name": "router", 
                "namespace": "default", 
                "ports": [
                    "80:80", 
                    "443:443"
                ], 
                "replicas": "5", 
                "selector": "region=infra", 
                "serviceaccount": "router", 
                "stats_port": 1936
~~~

Expected results: Installation should not try to deploy router pods on master servers, as they are by default set to "unschedulable" due to which router pod deployment fails.

Additional info: We can use variable "openshift_router_selector='region=primary'" as workaround in this case, router pod will only get deployed on node with primary region and installer will not try to deploy them on master nodes.


Please attach logs from ansible-playbook with the -vvv flag

Comment 1 Scott Dodson 2018-05-14 14:00:01 UTC
Can you please provide the inventory used to build this cluster? Also, please clarify the version of openshift-ansible in use as 3.11 does not exist yet.

Comment 2 Scott Dodson 2018-05-16 14:22:57 UTC
*** Bug 1577959 has been marked as a duplicate of this bug. ***

Comment 8 Scott Dodson 2018-08-02 12:40:17 UTC
Fatima,

Can you clarify the specifics of the situation you're facing with the customer.

In 3.9 masters should be schedulable. Additionally, it's up to you to label the masters as infra so if you prefer not to run these pods on those hosts simply don't mark them with region=infra.

Comment 23 errata-xmlrpc 2018-11-11 16:39:10 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:2709


Note You need to log in before you can comment on or make changes to this bug.