Discussion about this error here: https://github.com/kubernetes/kubernetes/issues/84610

Moving this to 4.4 because it is low priority and I haven't reproduced it locally yet.

9 occurrences in past 48hrs, bumping to high priority
https://search.svc.ci.openshift.org/?search=failed%28.*%29Delete+Grace+Period+should+be+submitted+and+removed&maxAge=336h&context=2&type=build-log

Giuseppe, can take this one over?

there is already an upstream fix.

Backport here: https://github.com/openshift/origin/pull/25008

@Mrunal: What is the next step for this bz?

$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.5.0-0.nightly-2020-06-09-223121   True        False         132m    Cluster version is 4.5.0-0.nightly-2020-06-09-223121

$ ./_output/local/bin/linux/amd64/openshift-tests run-test "[k8s.io] [sig-node] Pods Extended [k8s.io] Delete Grace Period should be submitted and removed [Flaky] [Suite:k8s]"
I0610 19:16:36.811246   29225 test_context.go:423] Tolerating taints "node-role.kubernetes.io/master" when considering if nodes are ready
Jun 10 19:16:36.870: INFO: Waiting up to 30m0s for all (but 100) nodes to be schedulable
Jun 10 19:16:39.173: INFO: Waiting up to 10m0s for all pods (need at least 0) in namespace 'kube-system' to be running and ready
Jun 10 19:16:40.080: INFO: 0 / 0 pods in namespace 'kube-system' are running and ready (0 seconds elapsed)
Jun 10 19:16:40.080: INFO: expected 0 pod replicas in namespace 'kube-system', 0 are Running and Ready.
Jun 10 19:16:40.080: INFO: Waiting up to 5m0s for all daemonsets in namespace 'kube-system' to start
Jun 10 19:16:40.407: INFO: e2e test version: v1.18.0-rc.1
Jun 10 19:16:40.796: INFO: kube-apiserver version: v1.18.3+cc5c3e3
Jun 10 19:16:41.204: INFO: Cluster IP family: ipv4
[BeforeEach] [Top Level]
  /home/sunilc/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/framework.go:1374
[BeforeEach] [Top Level]
  /home/sunilc/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/framework.go:1374
[BeforeEach] [Top Level]
  /home/sunilc/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/test.go:58
[BeforeEach] [k8s.io] [sig-node] Pods Extended
  /home/sunilc/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:178
STEP: Creating a kubernetes client
STEP: Building a namespace api object, basename pods
Jun 10 19:16:42.258: INFO: About to run a Kube e2e test, ensuring namespace is privileged
Jun 10 19:16:46.425: INFO: No PodSecurityPolicies found; assuming PodSecurityPolicy is disabled.
STEP: Waiting for a default service account to be provisioned in namespace
[BeforeEach] [k8s.io] Delete Grace Period
  /home/sunilc/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/node/pods.go:50
[It] should be submitted and removed [Flaky] [Suite:k8s]
  /home/sunilc/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/node/pods.go:59
STEP: creating the pod
STEP: setting up selector
STEP: submitting the pod to kubernetes
STEP: verifying the pod is in kubernetes
STEP: deleting the pod gracefully
STEP: verifying the kubelet observed the termination notice
Jun 10 19:17:06.330: INFO: start=2020-06-10 19:16:59.231967197 +0530 IST m=+25.505446802, now=2020-06-10 19:17:06.330768471 +0530 IST m=+32.604248131, kubelet pod: {"metadata":{"name":"pod-submit-remove-e4ee6f98-dc78-4b21-a159-7193ab5552c6","namespace":"e2e-pods-3808","selfLink":"/api/v1/namespaces/e2e-pods-3808/pods/pod-submit-remove-e4ee6f98-dc78-4b21-a159-7193ab5552c6","uid":"b99a01f6-9ca2-484d-a428-d4c74f28b9c6","resourceVersion":"29662","creationTimestamp":"2020-06-10T13:46:47Z","deletionTimestamp":"2020-06-10T13:47:28Z","deletionGracePeriodSeconds":30,"labels":{"name":"foo","time":"743883945"},"annotations":{"k8s.v1.cni.cncf.io/network-status":"[{\n    \"name\": \"openshift-sdn\",\n    \"interface\": \"eth0\",\n    \"ips\": [\n        \"10.128.2.11\"\n    ],\n    \"default\": true,\n    \"dns\": {}\n}]","k8s.v1.cni.cncf.io/networks-status":"[{\n    \"name\": \"openshift-sdn\",\n    \"interface\": \"eth0\",\n    \"ips\": [\n        \"10.128.2.11\"\n    ],\n    \"default\": true,\n    \"dns\": {}\n}]","kubernetes.io/config.seen":"2020-06-10T13:46:47.311024752Z","kubernetes.io/config.source":"api","openshift.io/scc":"anyuid"},"managedFields":[{"manager":"openshift-tests","operation":"Update","apiVersion":"v1","time":"2020-06-10T13:46:47Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:name":{},"f:time":{}}},"f:spec":{"f:containers":{"k:{\"name\":\"agnhost\"}":{".":{},"f:args":{},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{},"f:terminationGracePeriodSeconds":{}}}}]},"spec":{"volumes":[{"name":"default-token-shsq6","secret":{"secretName":"default-token-shsq6","defaultMode":420}}],"containers":[{"name":"agnhost","image":"us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.12","args":["pause"],"resources":{},"volumeMounts":[{"name":"default-token-shsq6","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent","securityContext":{"capabilities":{"drop":["MKNOD"]}}}],"restartPolicy":"Always","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"ip-10-0-134-0.us-east-2.compute.internal","securityContext":{"seLinuxOptions":{"level":"s0:c24,c9"}},"imagePullSecrets":[{"name":"default-dockercfg-ttn7l"}],"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExecute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true},"status":{"phase":"Pending","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-06-10T13:46:47Z"},{"type":"Ready","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-10T13:47:00Z","reason":"ContainersNotReady","message":"containers with unready status: [agnhost]"},{"type":"ContainersReady","status":"False","lastProbeTime":null,"lastTransitionTime":"2020-06-10T13:47:00Z","reason":"ContainersNotReady","message":"containers with unready status: [agnhost]"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2020-06-10T13:46:47Z"}],"hostIP":"10.0.134.0","podIP":"10.128.2.11","podIPs":[{"ip":"10.128.2.11"}],"startTime":"2020-06-10T13:46:47Z","containerStatuses":[{"name":"agnhost","state":{"waiting":{"reason":"ContainerCreating"}},"lastState":{},"ready":false,"restartCount":0,"image":"us.gcr.io/k8s-artifacts-prod/e2e-test-images/agnhost:2.12","imageID":"","started":false}],"qosClass":"BestEffort"}}
Jun 10 19:17:10.722: INFO: no pod exists with the name we were looking for, assuming the termination request was observed and completed
[AfterEach] [k8s.io] [sig-node] Pods Extended
  /home/sunilc/go/src/github.com/openshift/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:179
Jun 10 19:17:11.099: INFO: Waiting up to 7m0s for all (but 100) nodes to be ready
STEP: Destroying namespace "e2e-pods-3808" for this suite.
Jun 10 19:17:11.763: INFO: Running AfterSuite actions on all nodes
Jun 10 19:17:11.768: INFO: Running AfterSuite actions on node 1

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:2409