Bug 1866380 - must-gather pull policy should be changed to "Always"
Summary: must-gather pull policy should be changed to "Always"
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: oc
Version: 4.5
Hardware: Unspecified
OS: Unspecified
medium
medium
Target Milestone: ---
: 4.6.0
Assignee: Sally
QA Contact: RamaKasturi
URL:
Whiteboard:
Depends On:
Blocks: 1859972
TreeView+ depends on / blocked
 
Reported: 2020-08-05 13:08 UTC by Yossi Segev
Modified: 2020-10-27 16:25 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: No Doc Update
Doc Text:
Clone Of:
Environment:
Last Closed: 2020-10-27 16:24:53 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Github openshift oc pull 518 0 None closed Bug 1866380: must-gather ImagePullPolicy should be 'Always' 2020-10-08 20:37:02 UTC
Red Hat Product Errata RHBA-2020:4196 0 None None None 2020-10-27 16:25:10 UTC

Description Yossi Segev 2020-08-05 13:08:59 UTC
Description of problem:
must-gather pull-policy is "IfNotPresent". This sometimes ends in using a non-latest build of must-gather image, e.g.:
When running:
$ oc adm must-gather --image=registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-cnv-must-gather-rhel8:v2.4.0 --dest-dir=/home/cnv-qe-jenkins/yossi/mg

Let's assume there's already an image of v2.4.0 in the cluster (which remained there from a previous run of must-gather), e.g. from build #42, hence v2.4.0-42.
If the build *in the image repo* advanced, and now the latest there is #43 (v2.4.0-43), then the "IfNotPresent" pull-policy would prevent pulling the latest build.


Version-Release number of selected component (if applicable):
$ oc version
Client Version: 4.5.0-rc.1
Server Version: 4.5.0-rc.5
Kubernetes Version: v1.18.3+f291db1


How reproducible:
Always, although it is not easy to catch, as a user. There should be a significant difference between the existing, outdated image in the cluster and the one in the repo.


Steps to Reproduce:
1. Run a simple must-gather action, e.g.:
$ oc adm must-gather --image=registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-cnv-must-gather-rhel8:v2.4.0 --dest-dir=/home/cnv-qe-jenkins/yossi/mg


Actual results:
If an older image if the same version (v2.4.0 in this example) exists in the cluster - no new image will be pulled from the repo, even if the image in the repo is a newer build of the same version.


Expected results:
Always use the latest version (and build) of an image.


Additional info:
This was found whne using a CNV (Openshift Virtualization) must-gather image, but was debugged and found to be an issue the stems from the OCP must-gather pull-policy (in bz#1859972).

Comment 1 Sally 2020-08-20 19:34:53 UTC
The linked PR should merge this sprint to resolve this.

Comment 4 RamaKasturi 2020-08-26 10:59:53 UTC
Verified bug in the payload below and i see that the imagepullpolicy has been changed to Always.

[ramakasturinarra@dhcp35-60 verification-tests]$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.0-0.nightly-2020-08-26-032807   True        False         4h6m    Cluster version is 4.6.0-0.nightly-2020-08-26-032807
[ramakasturinarra@dhcp35-60 verification-tests]$ oc version -o yaml
clientVersion:
  buildDate: "2020-08-26T05:14:57Z"
  compiler: gc
  gitCommit: b08e3b04f1b19d9a7265b4c44372b44090262a87
  gitTreeState: clean
  gitVersion: 4.6.0-202008260439.p0-b08e3b0
  goVersion: go1.14.4
  major: ""
  minor: ""
  platform: linux/amd64
openshiftVersion: 4.6.0-0.nightly-2020-08-26-032807
serverVersion:
  buildDate: "2020-08-25T13:43:16Z"
  compiler: gc
  gitCommit: aaf4ce1cf2d4201a23e332030ccfeab6c17351fd
  gitTreeState: dirty
  gitVersion: v1.19.0-rc.2+aaf4ce1-dirty
  goVersion: go1.14.4
  major: "1"
  minor: 19+
  platform: linux/amd64


[ramakasturinarra@dhcp35-60 verification-tests]$ oc get pod must-gather-sm42d -o yaml | grep imagePullPolicy
            f:imagePullPolicy: {}
            f:imagePullPolicy: {}
    imagePullPolicy: Always
    imagePullPolicy: Always

Based on the above moving the bug to verified state

Comment 5 Yuval Turgeman 2020-09-14 06:05:01 UTC
Looks like the fix for this was reverted - https://github.com/openshift/oc/commit/de7c254f96746682bd7bbf7006a1f4da1751c114

Comment 6 Yossi Segev 2020-09-14 06:52:08 UTC
Good catch! Thanks, Yuval.
I submitted a needInfo question in the bug that is the source of this fix (https://bugzilla.redhat.com/show_bug.cgi?id=1874856#c5), with an ask to revert this.
Let's wait and see how that turns out.

Comment 7 Yuval Turgeman 2020-09-14 08:18:26 UTC
Well, you can look at the discussion on the PR https://github.com/openshift/oc/pull/549
I guess this one will be resolved as WONTFIX :/

Comment 13 errata-xmlrpc 2020-10-27 16:24:53 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (OpenShift Container Platform 4.6 GA Images), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2020:4196


Note You need to log in before you can comment on or make changes to this bug.