Bug 1950316
| Summary: | [IPI baremetal] installation without worker randomly fails due to ingress vip | ||||||
|---|---|---|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Borja Aranda <farandac> | ||||
| Component: | Installer | Assignee: | Yossi Boaron <yboaron> | ||||
| Installer sub component: | OpenShift on Bare Metal IPI | QA Contact: | Nataf Sharabi <nsharabi> | ||||
| Status: | CLOSED INSUFFICIENT_DATA | Docs Contact: | |||||
| Severity: | high | ||||||
| Priority: | high | CC: | aguclu, beth.white, kiran, pablo.iranzo, vvoronko, yboaron | ||||
| Version: | 4.8 | Keywords: | Triaged | ||||
| Target Milestone: | --- | ||||||
| Target Release: | 4.8.0 | ||||||
| Hardware: | Unspecified | ||||||
| OS: | Unspecified | ||||||
| Whiteboard: | |||||||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |||||
| Doc Text: | Story Points: | --- | |||||
| Clone Of: | Environment: | ||||||
| Last Closed: | 2021-06-01 16:39:03 UTC | Type: | Bug | ||||
| Regression: | --- | Mount Type: | --- | ||||
| Documentation: | --- | CRM: | |||||
| Verified Versions: | Category: | --- | |||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||
| Embargoed: | |||||||
| Bug Depends On: | |||||||
| Bug Blocks: | 1831748 | ||||||
| Attachments: |
|
||||||
|
Description
Borja Aranda
2021-04-16 11:51:24 UTC
Created attachment 1772422 [details]
must-gather
Are you sure the failure in this must-gather was related to the VIP? I'm seeing the "too many requests" error in the authentication pod logs, not a problem talking to ingress. Also, looking at the ingress pods, it appears the VIP correctly transitioned when new ingress pods were started. I'm not sure what prompted the change in ingress, but I see the pods started around 11:59 and the VIP transitioned from the .21 node to the .20 node at that time, which makes sense because neither of the ingress pods were running on .21 at that point. It's hard to say what was happening before that because we don't have logs from before the ingress pods restarted, but it suggests the healthcheck and VIP placement was working correctly. The other thing I will note is that this deployment used a release from before the double VIP fix merged. It's possible that bug would manifest as the VIP being on a node without ingress. You would have to look at all three masters and see if two of them have the VIP configured. If so, then it's the same bug which has already been fixed in later releases. So to make progress on this I need to see a must-gather where the failure was caused by an issue talking to ingress, and I would also like to know if the problem still occurs on a build from this week since the double VIP fix merged last week. Any update? Is this still happening? We did fix one duplicate VIP issue since this was opened. We're currently working on another, although I don't think that would affect the ingress VIP. Keepalived for ingress VIP doesn't select randomly one of the nodes, it should pick a node that a router pod is running on ( see [1] ), so even for deployment with 0 worker nodes ingress VIP supposed to land on the right master node. As Ben mentioned we solved a bug [2] related to the VIPs management, which could cause a node without router pod to hold the VIP. Please let us know if you still having this issue [1] https://github.com/openshift/machine-config-operator/blob/master/templates/master/00-master/on-prem/files/keepalived-keepalived.yaml#L39-#L43 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1931505 Closing this for now. The needinfo request has been overdue for more than a month. Please feel free to reopen. |