Bug 1425205

Summary: [dev-preview] "Error syncing pod, skipping: failed to "TeardownNetwork"" warning when deploying templates
Product: OpenShift Online Reporter: Will Gordon <wgordon>
Component: NetworkingAssignee: Dan Williams <dcbw>
Status: CLOSED DUPLICATE QA Contact: Meng Bo <bmeng>
Severity: medium Docs Contact:
Priority: medium    
Version: 3.xCC: aos-bugs, eparis, jkaur, jokerman, mmccomas, wgordon
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-03-01 19:15:11 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Will Gordon 2017-02-20 20:13:15 UTC
Description of problem:
Deploying default templates in OpenShift Dev Preview, e.g., nodejs-mongo-persistent or laravel-mysql-example causing confusing error messages in Event Monitoring.

The error message is: 

Error syncing pod, skipping: failed to "SetupNetwork" for "mysql-1-deploy_wgordon-test" with SetupNetworkError: "Failed to setup network for pod \"mysql-1-deploy_wgordon-test(808a01b7-f7a7-11e6-9235-0e63b9c1c48f)\" using network plugins \"cni\": CNI request failed with status 400: 'Failed to ensure that nat chain POSTROUTING jumps to MASQUERADE: error checking rule: exit status 4: iptables: Resource temporarily unavailable.\n\n'; Skipping pod"

This seems to cause no actual degradation of service, and only leads to the assumption that our default templates are somehow broken.

Version-Release number of selected component (if applicable):
v3.4.1.2 (online version 3.4.0.13)

How reproducible:
always


Steps to Reproduce:
1. Start a new project
2. Select "nodejs-mongo-persistent" to add to your project
3. Watch the Monitoring > Events page

Actual results:
Successful deployment with error message that apparently has no bearing on the outcome

Expected results:
Always successful, or errors with a "real" error message that requires the template owner to fix

Additional info:
Occurs in both -build and -deploy pods

Comment 1 Dan Williams 2017-03-01 19:09:59 UTC
I have gotten a patch accepted upstream to iptables that will prevent this error report from iptables when checking rules.

But this patch doesn't solve the problem of contention in the kernel that is the actual cause here.  We somehow need to figure out what two iptables processes are doing stuff at the same time, and which one isn't properly waiting for the other to finish.

Are you able to reliably reproduce this issue?