Bug 1724660

Description of problem:

The qemu RHCOS image consumed by openshift-install contains libpod 1.0.2-dev, and this appears to have several bugs which have been fixed in newer versions.

Two issues for current KNI work (ref https://github.com/openshift-metal3/kni-installer/pull/100) are:

- podman logs randomly returns an error instead of actual logs, particularly a problem when the pod has exited with an error and you can't find out why :)

- When mounting a shared volume with multiple containers inside a pod, often the pod delete is racy, and you end up with a spurious volume in use error and an undeletable pod (you can work around this by running the containers outside of a pod, but it's not ideal)

Version-Release number of selected component (if applicable):

libpod 1.0.2-dev

rhcos-410.8.20190520.0-qemu.qcow2

How reproducible:
Both problems have been fairly reproducable for me, although not every single test run (hence suspecting some racy behavior somewhere).

Steps to Reproduce:

1. Deploy with kni-installer PR above applied
2. Some pod error happens
3. Attempt to get podman logs to discover the cause

Actual results:

[core@localhost ~]$ sudo podman ps -a | grep Exited | grep ironic
9023e35e4bb1  quay.io/metal3-io/ironic:master                                               /bin/runironic        3 minutes ago  Exited (137) 2 minutes ago         ironic
790bc391cc94  quay.io/metal3-io/ironic-ipa-downloader:master                                /usr/local/bin/ge...  4 minutes ago  Exited (0) 4 minutes ago           ipa-downloader

[core@localhost ~]$ sudo podman logs 9023e35e4bb1
failed to open log file "/var/lib/containers/storage/overlay-containers/9023e35e4bb1a808cf5510311a39d22f508d8a94fa148b9c645fb58bc495e58a/userdata/ctr.log": open /var/lib/containers/storage/overlay-containers/9023e35e4bb1a808cf5510311a39d22f508d8a94fa148b9c645fb58bc495e58a/userdata/ctr.log: no such file or directory

[core@localhost ~]$ sudo podman logs 790bc391cc94
failed to open log file "/var/lib/containers/storage/overlay-containers/790bc391cc94a83edbec7aac051c28db573dfb2f9a193d7e62a9e4916caed1b5/userdata/ctr.log": open /var/lib/containers/storage/overlay-containers/790bc391cc94a83edbec7aac051c28db573dfb2f9a193d7e62a9e4916caed1b5/userdata/ctr.log: no such file or directory

[core@localhost ~]$ podman --version
podman version 1.0.2-dev


Expected results:

I expect to see the logs for all pods via `podman logs`

Additional info:

With some help from @walters I upgraded in-place with rpm-ostree override replace to podman-1.4.2-1.module+el8.1.0+3423+f0eda5e0.x86_64.rpm and the podman logs error was solved.

I don't have notes on the shared-volume error atm, but if necessary I can reproduce and provide steps taken (I've since switched to a script that doesn't use a pod to work around the issue).