Bug 2126697
| Summary: | containers config.json gets empty after sudden power loss | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Linux 8 | Reporter: | Suhaas Bhat <subhat> | |
| Component: | podman | Assignee: | Jindrich Novy <jnovy> | |
| Status: | CLOSED ERRATA | QA Contact: | Alex Jia <ajia> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | 8.6 | CC: | arajendr, bbaude, dornelas, dwalsh, jligon, jnovy, lsm5, mheon, pthomas, tsweeney, umohnani, ypu | |
| Target Milestone: | rc | Keywords: | Triaged, ZStream | |
| Target Release: | --- | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | podman-4.2.1-3.el8 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 2130236 2130237 2136270 (view as bug list) | Environment: | ||
| Last Closed: | 2023-05-16 08:21:12 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 2130236, 2130237, 2136270, 2136278 | |||
There is no expectation that config.json is consistent and safe to use across reboots; it is rebuilt every time a container is started. The issue is that inspect is attempting to use the on-disk config.json even when it is not valid. Upstream fix via https://github.com/containers/podman/pull/15799 Backport in progress via https://github.com/containers/podman/pull/15915 This bug has been verified on podman-4.2.1-3.module+el8.8.0+16760+2f749ab7 with netavark network backend. [root@test-8-6-0-bz2126697 ~]# podman inspect foo | grep -iA2 args WARN[0000] Error unmarshalling container 559f06b6c0779437ff516602e2f1330aa03ced98439a066ae2cb411b107d99b4 config: readObjectStart: expect { or n, but found , error found in #0 byte of ...||..., bigger context ...||... "Args": [ "infinity" ], [root@test-8-6-0-bz2126697 ~]# cat /etc/redhat-release Red Hat Enterprise Linux release 8.6 (Ootpa) [root@test-8-6-0-bz2126697 ~]# rpm -q podman runc netavark systemd kernel podman-4.2.1-3.module+el8.8.0+16760+2f749ab7.x86_64 runc-1.1.4-1.module+el8.8.0+16760+2f749ab7.x86_64 netavark-1.0.1-35.module+el8.6.0+15917+093ca6f8.x86_64 systemd-239-58.el8.x86_64 kernel-4.18.0-372.9.1.el8.x86_64 [root@test-8-6-0-bz2126697 ~]# grep network_backend /usr/share/containers/containers.conf #network_backend = "" network_backend = "netavark" *** Bug 2136270 has been marked as a duplicate of this bug. *** Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: container-tools:rhel8 security, bug fix, and enhancement update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2023:2758 |
Description of problem: After some tries of implementing #echo b > /proc/sysrq-trigger , /var/lib/containers/storage/overlay-containers/<container_id>/userdata/config.json gets empty and the outcome is that we cannot see the inspect output which returns below error : Error: error unmarshalling container config: readObjectStart: expect { or n, but found , error found in #0 byte of ...||..., bigger context Version-Release number of selected component (if applicable): podman-4.1.1-2.module+el8.6.0+15917+093ca6f8.x86_64 How reproducible: Fully Steps to Reproduce: 1. Install clean RHEL 8.6 with podman + netavark + runc 2. Spawn some containers using ubi8:minimal image (do not configure them to start on boot) 3. Cut the power / do `echo b > /proc/sysrq-trigger` 4. Run inspect on containers after regaining access (if no errors found) 5. Start all containers 6. Repeat from step #3 I was able to reproduce this in second try. Actual results: Integrity of containers config.json is affected. Expected results: Containers data shouldn't be affected. Additional info: