Bug 1796666

Summary: During minor update (z8 > z10) having issues with Docker until systems are rebooted
Product: Red Hat OpenStack Reporter: Brendan Shephard <bshephar>
Component: python-paunchAssignee: Sofer Athlan-Guyot <sathlang>
Status: CLOSED DUPLICATE QA Contact: nlevinki <nlevinki>
Severity: medium Docs Contact:
Priority: high    
Version: 13.0 (Queens)CC: amoralej, jfrancoa, sathlang
Target Milestone: ---Keywords: Triaged, ZStream
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-03-20 18:07:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Brendan Shephard 2020-01-30 21:41:54 UTC
Description of problem:
During a minor update, we're getting issues like this 

        "stderr: nsenter: failed to setns to /var/run/docker/netns/default: Invalid argument",                                 
        "container_linux.go:247: starting container process caused \"process_linux.go:245: running exec setns process for init 
caused \\\"exit status 17\\\"\"",

And 

        "stderr: /usr/bin/docker-current: Error response from daemon: oci runtime error: container_linux.go:247: starting conta
iner process caused \"process_linux.go:359: container init caused \\\"rootfs_linux.go:89: jailing process inside rootfs caused 
\\\\\\\"pivot_root invalid argument\\\\\\\"\\\"\".",

Version-Release number of selected component (if applicable):
RHOSP13 z8 > z10
I'm not sure of exactly what is causing the issue yet. Seems to be Docker but need clarification.

How reproducible:
Repeatable

Steps to Reproduce:
1. openstack overcloud update run --nodes Compute
2.
3.

Actual results:
Get the above errors during the update and the update fails.

Rebooting has resolved the issues on the Controllers, but it not feasible on the Compute nodes.

Expected results:
Update would succeed without needing to reboot

Additional info:
I have a list of all the packages that were updated prior to the issue occurring. Will attach to the BZ along with the full error output.

Comment 5 Sofer Athlan-Guyot 2020-03-04 15:55:57 UTC
Hi,

so we had this error but only when updating to rhel-7.8[1]

So we would need more information:
 - in rhel updated to 7.8 ?
 - could we have sos-report of one compute node that failed (yum history doesn't show any update of docker)
 - output of the update commands would be helpful too.

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1795376

Comment 6 Sofer Athlan-Guyot 2020-03-20 18:07:22 UTC
Hi,

the mentioned bug was fixed by a new version of docker.

Closing this one a duplicate, if you have more questions please feel free to re-open it.

*** This bug has been marked as a duplicate of bug 1795376 ***