the node in V(2) loses 80%+ of its logs because of spam. Please eliminate the list of actual iptables rules from being logged at V(2) so more useful messages can come through.
Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: I1026 19:03:01.694600 119323 proxier.go:1602] Rules: Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: *filter Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: :KUBE-SERVICES - [0:0] Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: -A KUBE-SERVICES -m comment --comment "trys/tc8:8443-tcp has no endpoints" -m tcp -p tcp -d 172.30.141.35/32 --dport 8443 -j REJECT Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: -A KUBE-SERVICES -m comment --comment "maew/maew:8080-tcp has no endpoints" -m tcp -p tcp -d 172.30.133.108/32 --dport 8080 -j REJECT Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: -A KUBE-SERVICES -m comment --comment "moody-backend/moody:8080-tcp has no endpoints" -m tcp -p tcp -d 172.30.196.168/32 --dport 8080 -j REJECT Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: -A KUBE-SERVICES -m comment --comment "open5/ruby-ex:8080-tcp has no endpoints" -m tcp -p tcp -d 172.30.27.112/32 --dport 8080 -j REJECT Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: -A KUBE-SERVICES -m comment --comment "hongz/mongodb:mongodb has no endpoints" -m tcp -p tcp -d 172.30.29.86/32 --dport 27017 -j REJECT Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: -A KUBE-SERVICES -m comment --comment "dsntest12/cakephp-mysql-persistent:web has no endpoints" -m tcp -p tcp -d 172.30.101.200/32 --dport 8080 -j REJECT Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: -A KUBE-SERVICES -m comment --comment "rfd/orbit:8080-tcp has no endpoints" -m tcp -p tcp -d 172.30.209.225/32 --dport 8080 -j REJECT Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: -A KUBE-SERVICES -m comment --comment "deliriy-tmp/apiserver:secure has no endpoints" -m tcp -p tcp -d 172.30.1.2/32 --dport 443 -j REJECT Oct 26 19:03:01 ip-172-31-21-249.ca-central-1.compute.internal atomic-openshift-node[119323]: -A KUBE-SERVICES -m comment --comment "611pro/pro611:8080-tcp has no endpoints" -m tcp -p tcp -d 172.30.85.236/32 --dport 8080 -j REJECT ...
PR https://github.com/openshift/origin/issues/17060
Commits pushed to master at https://github.com/openshift/origin https://github.com/openshift/origin/commit/74c77fc0e12485be8dca6dd6580891f5224ad565 Reduce node iptables logging in V(2) bug: 1506332 https://bugzilla.redhat.com/show_bug.cgi?id=1506332 https://github.com/openshift/origin/commit/92cabfbec15f9de10d5e7cf0160defc976c82293 Merge pull request #17060 from pecameron/bz1506332 Automatic merge from submit-queue. UPSTREAM: 54700: Reduce node iptables logging in V(2) bug: 1506332 https://bugzilla.redhat.com/show_bug.cgi?id=1506332
Tested on ocp v3.7.0-0.190.0 The iptables restore failure will not print the rule list. Nov 03 07:31:49 qe-bmeng-37-new-master-etcd-nfs-1.c.openshift-gce-devel.internal atomic-openshift-node[11969]: E1103 07:31:49.977839 11969 proxier.go:1601] Failed to execute iptables-restore: exit status 4 (Another app is currently holding the xtables lock. Perhaps you want to use the -w option? Nov 03 07:31:49 qe-bmeng-37-new-master-etcd-nfs-1.c.openshift-gce-devel.internal atomic-openshift-node[11969]: ) Nov 03 07:31:49 qe-bmeng-37-new-master-etcd-nfs-1.c.openshift-gce-devel.internal atomic-openshift-node[11969]: I1103 07:31:49.977869 11969 proxier.go:993] syncProxyRules took 2.360046661s Nov 03 07:31:49 qe-bmeng-37-new-master-etcd-nfs-1.c.openshift-gce-devel.internal atomic-openshift-node[11969]: I1103 07:31:49.977884 11969 bounded_frequency_runner.go:221] sync-runner: ran, next possible in 0s, periodic in 30s Nov 03 07:31:50 qe-bmeng-37-new-master-etcd-nfs-1.c.openshift-gce-devel.internal atomic-openshift-node[11969]: I1103 07:31:50.062045 11969 generic.go:183] GenericPLEG: Relisting
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2017:3188