Bug 1414907 - POD Readiness probe fails due to POD not answering TCP SYN packages [NEEDINFO]
Summary: POD Readiness probe fails due to POD not answering TCP SYN packages
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Networking
Version: 3.2.1
Hardware: Unspecified
OS: Unspecified
high
medium
Target Milestone: ---
: ---
Assignee: Dan Winship
QA Contact: Meng Bo
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-01-19 17:15 UTC by Miheer Salunke
Modified: 2018-12-28 06:07 UTC (History)
12 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-06-02 14:55:41 UTC
Target Upstream Version:
danw: needinfo? (misalunk)

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)

Comment 7 Dan Winship 2017-01-31 19:50:22 UTC 
The dump-flows output show that there are pods with duplicate IP addresses. And so the health check fails for this pod because the packets are getting sent to the *other* pod that has that IP address.

Probably this was caused by some sort of bug in the 3.2->3.3 upgrade; the journal output attached above doesn't go back far enough to show how we originally got into this situation.

I think the simplest fix is to make the node unschedulable, delete all of the pods on it, reboot the node, and then make it schedulable again. Assuming the problem actually was upgrade-related, it shouldn't recur.
Comment 15 Ben Bennett 2017-04-19 14:02:00 UTC 
Closing due to inactivity.  Please re-open if it is still happening, and provide the requested logs.
Note You need to log in before you can comment on or make changes to this bug.