Bug 1539691
| Summary: | 3.9.0-0.31.0 - web console pod does not start because master is not schedulable | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Mike Fiedler <mifiedle> |
| Component: | Installer | Assignee: | Vadim Rutkovsky <vrutkovs> |
| Status: | CLOSED CURRENTRELEASE | QA Contact: | Johnny Liu <jialiu> |
| Severity: | medium | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 3.9.0 | CC: | aos-bugs, jiajliu, jokerman, mmccomas, wmeng |
| Target Milestone: | --- | ||
| Target Release: | 3.9.0 | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2018-04-13 12:17:42 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Mike Fiedler
2018-01-29 13:05:07 UTC
Need to make sure that masters are by default schedulable and I think this would be addressed. Lets add a check to ensure that if the console is deployed that masters are not openshift_schedulable=false. openshift_sanitize_inventory is likely a good place for this. (In reply to Scott Dodson from comment #3) > Lets add a check to ensure that if the console is deployed that masters are > not openshift_schedulable=false. openshift_sanitize_inventory is likely a > good place for this. Created https://github.com/openshift/openshift-ansible/pull/6984 to address this Fix is available in openshift-ansible-3.9.0-0.36.0.git.0.da68f13.el7 "Taint master nodes" task is not merged into openshift-ansible-3.9.0-0.36.0.git.0.da68f13.el7.noarch yet. After go through the code, seem like the PR would introduce some other issues. 1. service catalog would have no available node to deploy. By default, installer would label the 1st master node with "openshift-infra=apiserver", once taint is added for all masters, then service catalog daemonset would fail to deploy pod. 2. By default, installer would deploy logging fluentd via daemonset on all nodes, also including master node, once train is added for all master nodes, that means no fluentd pod is running on master nodes, logging can not collect log from there. (In reply to Johnny Liu from comment #7) > After go through the code, seem like the PR would introduce some other > issues. > > 1. service catalog would have no available node to deploy. > By default, installer would label the 1st master node with > "openshift-infra=apiserver", once taint is added for all masters, then > service catalog daemonset would fail to deploy pod. > > 2. By default, installer would deploy logging fluentd via daemonset on all > nodes, also including master node, once train is added for all master nodes, > that means no fluentd pod is running on master nodes, logging can not > collect log from there. Good points, these would be discussed. Sounds like service catalog and logging templates should add tolerations too (In reply to Johnny Liu from comment #6) > "Taint master nodes" task is not merged into > openshift-ansible-3.9.0-0.36.0.git.0.da68f13.el7.noarch yet. Correct, tainting masters is still being discussed and is out of scope of this issue. @Vadim Rutkovsky Upgrade git the issue related with schedulable master that some app pods are scheduled on master node after upgrade. I think this is not expected result. # oc get node NAME STATUS ROLES AGE VERSION qe-jliu-r-master-etcd-1 Ready master 2h v1.9.1+a0ce1bc657 qe-jliu-r-node-registry-router-1 Ready <none> 2h v1.9.1+a0ce1bc657 # oc get pod -o wide --all-namespaces |grep master default registry-console-2-hlgln 1/1 Running 0 1h 10.129.0.4 qe-jliu-r-master-etcd-1 install-test mongodb-1-psr9l 1/1 Running 0 1h 10.129.0.5 qe-jliu-r-master-etcd-1 install-test nodejs-mongodb-example-1-k56zh 1/1 Running 0 1h 10.129.0.18 qe-jliu-r-master-etcd-1 openshift-web-console webconsole-54877f6577-g7tb8 1/1 Running 0 1h 10.129.0.2 qe-jliu-r-master-etcd-1 test mysql-1-ptblc 1/1 Running 0 1h 10.129.0.19 qe-jliu-r-master-etcd-1 Not sure if the issue is in the scope of this bug, or should I track it in a new bug? (In reply to liujia from comment #9) > @Vadim Rutkovsky > Upgrade git the issue related with schedulable master that some app pods are > scheduled on master node after upgrade. I think this is not expected result. Right, that's certainly not expected > Not sure if the issue is in the scope of this bug, or should I track it in a > new bug? Lets file a new bug for this (and move this one in VERIFIED), as it gets pretty complex to track it. The new bug should be a blocker for 3.9 Verified this bug with openshift-ansible-3.9.1-1.git.0.9862628.el7.noarch, and PASS. Now master is schedulable, web console could be deployed successfully. |