Description of problem: The qemu RHCOS image consumed by openshift-install contains libpod 1.0.2-dev, and this appears to have several bugs which have been fixed in newer versions. Two issues for current KNI work (ref https://github.com/openshift-metal3/kni-installer/pull/100) are: - podman logs randomly returns an error instead of actual logs, particularly a problem when the pod has exited with an error and you can't find out why :) - When mounting a shared volume with multiple containers inside a pod, often the pod delete is racy, and you end up with a spurious volume in use error and an undeletable pod (you can work around this by running the containers outside of a pod, but it's not ideal) Version-Release number of selected component (if applicable): libpod 1.0.2-dev rhcos-410.8.20190520.0-qemu.qcow2 How reproducible: Both problems have been fairly reproducable for me, although not every single test run (hence suspecting some racy behavior somewhere). Steps to Reproduce: 1. Deploy with kni-installer PR above applied 2. Some pod error happens 3. Attempt to get podman logs to discover the cause Actual results: [core@localhost ~]$ sudo podman ps -a | grep Exited | grep ironic 9023e35e4bb1 quay.io/metal3-io/ironic:master /bin/runironic 3 minutes ago Exited (137) 2 minutes ago ironic 790bc391cc94 quay.io/metal3-io/ironic-ipa-downloader:master /usr/local/bin/ge... 4 minutes ago Exited (0) 4 minutes ago ipa-downloader [core@localhost ~]$ sudo podman logs 9023e35e4bb1 failed to open log file "/var/lib/containers/storage/overlay-containers/9023e35e4bb1a808cf5510311a39d22f508d8a94fa148b9c645fb58bc495e58a/userdata/ctr.log": open /var/lib/containers/storage/overlay-containers/9023e35e4bb1a808cf5510311a39d22f508d8a94fa148b9c645fb58bc495e58a/userdata/ctr.log: no such file or directory [core@localhost ~]$ sudo podman logs 790bc391cc94 failed to open log file "/var/lib/containers/storage/overlay-containers/790bc391cc94a83edbec7aac051c28db573dfb2f9a193d7e62a9e4916caed1b5/userdata/ctr.log": open /var/lib/containers/storage/overlay-containers/790bc391cc94a83edbec7aac051c28db573dfb2f9a193d7e62a9e4916caed1b5/userdata/ctr.log: no such file or directory [core@localhost ~]$ podman --version podman version 1.0.2-dev Expected results: I expect to see the logs for all pods via `podman logs` Additional info: With some help from @walters I upgraded in-place with rpm-ostree override replace to podman-1.4.2-1.module+el8.1.0+3423+f0eda5e0.x86_64.rpm and the podman logs error was solved. I don't have notes on the shared-volume error atm, but if necessary I can reproduce and provide steps taken (I've since switched to a script that doesn't use a pod to work around the issue).
> Two issues for current KNI work Which is targeted for 4.2 right? So I think this should be a 4.2 bug.
(In reply to Colin Walters from comment #4) > > Two issues for current KNI work > > Which is targeted for 4.2 right? So I think this should be a 4.2 bug. Yes targeted at 4.2 - currently we're consuming pinned releases from upstream CI and expect to target the 4.2 builds for customer usage.
Lokesh has requested tagging of updated packages from RCM. Will update once they make it in.
Using 42.80.20190729.0, it looks like it has the requested packages: ``` $ rpm-ostree status State: idle AutomaticUpdates: disabled Deployments: ● pivot://quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3ea1bccf8de033ff9ec6782dc9de3b7d0dd1efe0312fde4e71942dad1408a91d CustomOrigin: Image generated via coreos-assembler Version: 42.80.20190729.0 (2019-07-29T09:00:02Z) $ rpm -q cri-o podman cri-o-1.14.10-0.5.dev.rhaos4.2.gitcf4220b.el8.x86_64 podman-1.4.2-1.module+el8.1.0+3423+f0eda5e0.x86_64 ```
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2922