Bug 1959322
| Summary: | Cannot start container with CreateContainerError for the cnf-app-mac-operator | ||
|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Gurenko Alex <agurenko> |
| Component: | Node | Assignee: | Harshal Patil <harpatil> |
| Node sub component: | Kubelet | QA Contact: | Sunil Choudhary <schoudha> |
| Status: | CLOSED DUPLICATE | Docs Contact: | |
| Severity: | high | ||
| Priority: | unspecified | CC: | aadam, aos-bugs, ehashman, fidencio, harpatil, mcornea |
| Version: | 4.7 | ||
| Target Milestone: | --- | ||
| Target Release: | 4.8.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2021-05-17 09:45:57 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
|
Description
Gurenko Alex
2021-05-11 09:34:31 UTC
Just FYI, this might be not kata related, I'm not sure if I picked a right component, it's a regular deployment of OCP This is definitely not related to kata-containers, at all. The container failing to start is using runc and the operator used is not the `sandboxed-containers` one. Let me re-assign it to what I think that may be the right component. This is not the right component, sorry. I'm not sure that CNF is a right group either, it looks more like a general podman/runc issue?
I've been monitoring the env, it looks like oom-kill is invoked for unknown reasons:
May 11 13:59:36 worker-0-1 kernel: oom-kill:constraint=CONSTRAINT_MEMCG,nodemask=(null),cpuset=crio-ec1a1e99d5bcbea421595974f82ade54748ce554eeff196cc50e987add54d1df.scope,mems_allowed=0,oom_memcg=/kubepods.slice/kubepods-burstable.slice/kubepods-bursta>
May 11 13:59:36 worker-0-1 kernel: Memory cgroup out of memory: Killed process 104706 (runc:[2:INIT]) total-vm:729424kB, anon-rss:6144kB, file-rss:1200kB, shmem-rss:0kB, UID:0
May 11 13:59:36 worker-0-1 hyperkube[3522]: I0511 13:59:36.324442 3522 oom_watcher_linux.go:76] Got sys oom event: &{104706 runc:[2:INIT] 2021-05-11 13:59:35.14392774 +0000 UTC m=+4139.686261680 / / }
May 11 13:59:36 worker-0-1 kernel: oom_reaper: reaped process 104706 (runc:[2:INIT]), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
The host has 26G of available RAM
[kni@provisionhost-0-0 ~]$ oc describe node worker-0-1
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 1234m (16%) 300m (4%)
memory 4718Mi (15%) 158Mi (0%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-1Gi 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning SystemOOM 19m kubelet System OOM encountered, victim process: elasticsearch-o, pid: 73894
Warning SystemOOM 19m kubelet System OOM encountered, victim process: pod, pid: 73605
Warning SystemOOM 19m kubelet System OOM encountered, victim process: elasticsearch-o, pid: 74801
Warning SystemOOM 19m kubelet System OOM encountered, victim process: pod, pid: 74694
Warning SystemOOM 18m kubelet System OOM encountered, victim process: elasticsearch-o, pid: 75903
Warning SystemOOM 18m kubelet System OOM encountered, victim process: pod, pid: 75503
Warning SystemOOM 18m kubelet System OOM encountered, victim process: elasticsearch-o, pid: 77169
Warning SystemOOM 18m kubelet System OOM encountered, victim process: pod, pid: 76678
Warning SystemOOM 17m kubelet System OOM encountered, victim process: elasticsearch-o, pid: 79353
Warning SystemOOM 2m15s (x26 over 17m) kubelet (combined from similar events): System OOM encountered, victim process: runc:[2:INIT], pid: 107285
Moving this to the Node team, specifically to the Memory manager subcomponent. The subcomponent may be wrong, but I think now you're being redirected to the correct component, Alex. (In reply to Fabiano Fidêncio from comment #6) > Moving this to the Node team, specifically to the Memory manager > subcomponent. > > The subcomponent may be wrong, but I think now you're being redirected to > the correct component, Alex. Thanks a lot! *** This bug has been marked as a duplicate of bug 1904558 *** |