Bug 1934867 - Static pod logs (and others?) are not being separated by container restarts
Summary: Static pod logs (and others?) are not being separated by container restarts
Keywords:
Status: CLOSED DUPLICATE of bug 1956898
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: Node
Version: 4.8
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
: ---
Assignee: Ryan Phillips
QA Contact: Weinan Liu
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2021-03-03 22:03 UTC by Clayton Coleman
Modified: 2023-09-15 01:02 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-06-01 16:55:48 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)

Description Clayton Coleman 2021-03-03 22:03:49 UTC
In an e2e test I noticed that MCD caused a reboot and after the reboot the new container (new start) shared the same log directory.  That is a bug - whenever Kubelet starts a new container the logs should be distinct.  That goes for static, daemonset, or regular pods, and applies to reboots, crashed containers, or deliberate container stops by an admin using process kill.  The container ID must change everytime a container is started, this is a part of Kube conformance i.e. (if pid 1 dies the value in status: containerID: cri-o://922f57399fddb33cbeeec03203fa508f00553587a8ea711a3c0bd9b879ff1057 must be different the next time).  Everytime the new process is started (which means the previous process died) we must get a new container id and a new set of logs.

This is high because it means some pods are not behaving the same as others and we are not conformant.  We need an e2e test in kube that ensures that if a container process is killed as a static pod, the next time it starts it has a different containerID and the logs are empty, and we need cri-o to behave that way.  I would expect to make this a conformance test.

The feature behavior (this is nice!) can be separate and discussed separately.

Comment 1 Ryan Phillips 2021-03-04 01:04:36 UTC
I tested this a number of times with a local-up-cluster. The rotation is working correctly (with and without the UID) in the example pod. Moving the static pod in and out of the static pod directory works correctly. The log name is based on the restartCount: 0.log, 1.log. The problem stems from crio-wipe being run and wiping the restartCount.

apiVersion: v1
kind: Pod
uid: 8673c22a-27d6-4a2e-aedd-bc791bf665cf
metadata:
  name: test-pod-1
  namespace: default
spec:
  containers:
  - name: test-hello
    image: fedora
    command:
    - /bin/sh
    - -c
    - |
      #!/bin/sh
      while true ; do
        date
        sleep 5
      done

Comment 2 Ryan Phillips 2021-03-04 02:52:59 UTC
Potential PR: https://github.com/kubernetes/kubernetes/pull/99748

Comment 7 Ryan Phillips 2021-06-01 16:55:48 UTC
Fixed and merged via a separate BZ.

*** This bug has been marked as a duplicate of bug 1956898 ***

Comment 9 Red Hat Bugzilla 2023-09-15 01:02:43 UTC
The needinfo request[s] on this closed bug have been removed as they have been unresolved for 500 days


Note You need to log in before you can comment on or make changes to this bug.