Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 2089933

Summary: Pods do not post readiness status on termination
Product: OpenShift Container Platform Reporter: Ryan Phillips <rphillips>
Component: NodeAssignee: Ryan Phillips <rphillips>
Node sub component: Kubelet QA Contact: Sunil Choudhary <schoudha>
Status: CLOSED ERRATA Docs Contact:
Severity: high    
Priority: high CC: nagrawal, wking
Version: 4.9   
Target Milestone: ---   
Target Release: 4.11.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Cause: Code changes in 4.9 Consequence: Readiness probes were not running. Fix: Code change Result: Readiness probes now fire on pod termination.
 Story Points: ---
Clone Of: Environment:
Last Closed: 2022-08-10 11:14:04 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Ryan Phillips 2022-05-24 18:13:42 UTC 
Description of problem:
4.9 introduced a pod refactor within the kubelet. This refactor stopped all probes on termination; however, eadiness probes are extremely useful to load balancers and controllers to know when a pod is not handling traffic anymore (to allow for a graceful handoff).

The kubelet should run readiness probes on termination. The documentation states that the kubelet should, but it does not.

Upstream Issue: https://github.com/kubernetes/kubernetes/issues/102537
Related Endpoints and EndpointSlices issue: https://github.com/kubernetes/kubernetes/pull/110115
PR to Fix: https://github.com/kubernetes/kubernetes/pull/110191

Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 3 Sunil Choudhary 2022-06-17 10:55:02 UTC 
% oc get clusterversion
NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.11.0-0.nightly-2022-06-15-222801   True        False         47m     Cluster version is 4.11.0-0.nightly-2022-06-15-222801

% cat epod.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: twocontainers
  annotations:
    cpu-load-balancing.crio.io: disable
spec:
  containers:
  - name: sise
    image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:aa8d1daf3432d8dedc5c56d94aeb1f25301bce6ccd7d5406fb03a00be97374ad
    command:
      - "bin/bash"
      - "-c"
      - "sleep 10000"
  resources:
    limits:
      cpu: “500m”
      memory: "500Mi"
    requests:
      cpu: “400m”
      memory: "400Mi"


% oc create -f epod.yaml                                     
Warning: would violate PodSecurity "restricted:latest": allowPrivilegeEscalation != false (container "sise" must set securityContext.allowPrivilegeEscalation=false), unrestricted capabilities (container "sise" must set securityContext.capabilities.drop=["ALL"]), runAsNonRoot != true (pod or container "sise" must set securityContext.runAsNonRoot=true), seccompProfile (pod or container "sise" must set securityContext.seccompProfile.type to "RuntimeDefault" or "Localhost")
pod/twocontainers created

% oc get pods
NAME            READY   STATUS    RESTARTS   AGE
twocontainers   1/1     Running   0          3s

On terminal 1, I deleted pod and on terminal 2 I see the pod status changed to Terminating immediately.

Terminal 1:

% oc get pods
NAME            READY   STATUS    RESTARTS   AGE
twocontainers   1/1     Running   0          7m39s
sunilc@schoudha-mac debug1 % oc delete pod twocontainers

Terminal 2:

% oc get pods -w
NAME            READY   STATUS    RESTARTS   AGE
twocontainers   1/1     Running   0          7m32s
twocontainers   1/1     Terminating   0          7m44s
twocontainers   0/1     Terminating   0          8m15s
twocontainers   0/1     Terminating   0          8m15s
twocontainers   0/1     Terminating   0          8m15s
Comment 5 errata-xmlrpc 2022-08-10 11:14:04 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Important: OpenShift Container Platform 4.11.0 bug fix and security update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5069