Bug 1866380 - must-gather pull policy should be changed to "Always" [NEEDINFO]
Summary: must-gather pull policy should be changed to "Always"
Alias: None
Product: OpenShift Container Platform
Classification: Red Hat
Component: oc
Version: 4.5
Hardware: Unspecified
OS: Unspecified
Target Milestone: ---
: 4.6.0
Assignee: Sally
QA Contact: RamaKasturi
Depends On:
Blocks: 1859972
TreeView+ depends on / blocked
Reported: 2020-08-05 13:08 UTC by Yossi Segev
Modified: 2020-09-14 13:43 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Last Closed:
Target Upstream Version:
ysegev: needinfo? (ncredi)

Attachments (Terms of Use)

System ID Priority Status Summary Last Updated
Github openshift oc pull 518 None closed Bug 1866380: must-gather ImagePullPolicy should be 'Always' 2020-09-14 05:58:23 UTC

Description Yossi Segev 2020-08-05 13:08:59 UTC
Description of problem:
must-gather pull-policy is "IfNotPresent". This sometimes ends in using a non-latest build of must-gather image, e.g.:
When running:
$ oc adm must-gather --image=registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-cnv-must-gather-rhel8:v2.4.0 --dest-dir=/home/cnv-qe-jenkins/yossi/mg

Let's assume there's already an image of v2.4.0 in the cluster (which remained there from a previous run of must-gather), e.g. from build #42, hence v2.4.0-42.
If the build *in the image repo* advanced, and now the latest there is #43 (v2.4.0-43), then the "IfNotPresent" pull-policy would prevent pulling the latest build.

Version-Release number of selected component (if applicable):
$ oc version
Client Version: 4.5.0-rc.1
Server Version: 4.5.0-rc.5
Kubernetes Version: v1.18.3+f291db1

How reproducible:
Always, although it is not easy to catch, as a user. There should be a significant difference between the existing, outdated image in the cluster and the one in the repo.

Steps to Reproduce:
1. Run a simple must-gather action, e.g.:
$ oc adm must-gather --image=registry-proxy.engineering.redhat.com/rh-osbs/container-native-virtualization-cnv-must-gather-rhel8:v2.4.0 --dest-dir=/home/cnv-qe-jenkins/yossi/mg

Actual results:
If an older image if the same version (v2.4.0 in this example) exists in the cluster - no new image will be pulled from the repo, even if the image in the repo is a newer build of the same version.

Expected results:
Always use the latest version (and build) of an image.

Additional info:
This was found whne using a CNV (Openshift Virtualization) must-gather image, but was debugged and found to be an issue the stems from the OCP must-gather pull-policy (in bz#1859972).

Comment 1 Sally 2020-08-20 19:34:53 UTC
The linked PR should merge this sprint to resolve this.

Comment 4 RamaKasturi 2020-08-26 10:59:53 UTC
Verified bug in the payload below and i see that the imagepullpolicy has been changed to Always.

[ramakasturinarra@dhcp35-60 verification-tests]$ oc get clusterversion
NAME      VERSION                             AVAILABLE   PROGRESSING   SINCE   STATUS
version   4.6.0-0.nightly-2020-08-26-032807   True        False         4h6m    Cluster version is 4.6.0-0.nightly-2020-08-26-032807
[ramakasturinarra@dhcp35-60 verification-tests]$ oc version -o yaml
  buildDate: "2020-08-26T05:14:57Z"
  compiler: gc
  gitCommit: b08e3b04f1b19d9a7265b4c44372b44090262a87
  gitTreeState: clean
  gitVersion: 4.6.0-202008260439.p0-b08e3b0
  goVersion: go1.14.4
  major: ""
  minor: ""
  platform: linux/amd64
openshiftVersion: 4.6.0-0.nightly-2020-08-26-032807
  buildDate: "2020-08-25T13:43:16Z"
  compiler: gc
  gitCommit: aaf4ce1cf2d4201a23e332030ccfeab6c17351fd
  gitTreeState: dirty
  gitVersion: v1.19.0-rc.2+aaf4ce1-dirty
  goVersion: go1.14.4
  major: "1"
  minor: 19+
  platform: linux/amd64

[ramakasturinarra@dhcp35-60 verification-tests]$ oc get pod must-gather-sm42d -o yaml | grep imagePullPolicy
            f:imagePullPolicy: {}
            f:imagePullPolicy: {}
    imagePullPolicy: Always
    imagePullPolicy: Always

Based on the above moving the bug to verified state

Comment 5 Yuval Turgeman 2020-09-14 06:05:01 UTC
Looks like the fix for this was reverted - https://github.com/openshift/oc/commit/de7c254f96746682bd7bbf7006a1f4da1751c114

Comment 6 Yossi Segev 2020-09-14 06:52:08 UTC
Good catch! Thanks, Yuval.
I submitted a needInfo question in the bug that is the source of this fix (https://bugzilla.redhat.com/show_bug.cgi?id=1874856#c5), with an ask to revert this.
Let's wait and see how that turns out.

Comment 7 Yuval Turgeman 2020-09-14 08:18:26 UTC
Well, you can look at the discussion on the PR https://github.com/openshift/oc/pull/549
I guess this one will be resolved as WONTFIX :/

Note You need to log in before you can comment on or make changes to this bug.