Bug 1914712

Summary: [OCP 4.7] 4.7.0-fc.0 and 4.7.0-fc.1 builds' images contain invalid docker tag format (with the = sign)
Product: OpenShift Container Platform Reporter: krmoser
Component: Multi-ArchAssignee: Dennis Gilmore <dgilmore>
Status: CLOSED DUPLICATE QA Contact: Barry Donahue <bdonahue>
Severity: high Docs Contact:
Priority: unspecified    
Version: 4.7CC: aos-bugs, chanphil, christian.lapolt, Holger.Wolf, psundara, wking, wvoesch
Target Milestone: ---   
Target Release: ---   
Hardware: s390x   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-01-11 18:20:54 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1903544    

Description krmoser 2021-01-11 00:53:11 UTC
Description of problem:
1. The OCP 4.7 4.7.0-fc.0 and 4.7.0-fc.1 builds' images, available at https://mirror.openshift.com/pub/openshift-v4/s390x/clients/ocp-dev-preview/4.7.0-fc.0/ and https://mirror.openshift.com/pub/openshift-v4/s390x/clients/ocp-dev-preview/4.7.0-fc.1/, respectively, have invalid docker tag formats, as they include the "=" (equals) sign.

2. As an example, instead of the docker image tag prefixed with "4.7.0-fc.1-s390x" for an image, the image is prefixed with "4.7.0-fc.1=s390x".


Version-Release number of selected component (if applicable):
1. OCP 4.7 4.7.0-fc.0 
2. OCP 4.7 4.7.0-fc.1 

How reproducible:
Consistently reproducible.

Steps to Reproduce:
1. Attempt to mirror either the OCP 4.7 4.7.0-fc.0 and 4.7.0-fc.1 builds to local repositories.

2. As an example using 4.7.0-fc.1, the manifests will show:
manifests:
      sha256:0454d32b5907914b54dff58c74b856312458854361f0831bba8731141d9db79b -> 4.7.0-fc.1=s390x-network-metrics-daemon
      sha256:08e66af1f351e3131027068b5ab3177589f8e474dd4fb202da8843e1857ef47d -> 4.7.0-fc.1=s390x-insights-operator
      sha256:08e8c62834f87a9f9b5f161b86bc18d891f54a29310ae2b62ac331f54741aeda -> 4.7.0-fc.1=s390x-multus-admission-controller
      sha256:099d472c10a38827e08fb6817579545122a7dca8f26f719a3f3d2396fc2eb9e9 -> 4.7.0-fc.1=s390x-cluster-config-operator
      sha256:0aa312d14f684c7c8f94b8348b0c0ed2d9270cc2e9acc68cdbb044eeda50b65a -> 4.7.0-fc.1=s390x-local-storage-static-provisioner
      sha256:0bb235b42d9a4289ecdb86e63959c5bd6ef1e1bd4230173b4ea35fda36f4407d -> 4.7.0-fc.1=s390x-grafana
      sha256:0ec35d1ba4c9a9c0d3b3780b22366c663561e95753b456c4c31c302c2a907733 -> 4.7.0-fc.1=s390x-csi-external-attacher
      sha256:0fad36c15c5e24c251c17c9e1c71ffd7e44af1090281fe52f88e9b5e6f05562c -> 4.7.0-fc.1=s390x-machine-api-operator
      sha256:10f5a2fe6c0c5d22101dc5127544243e83b7e15562589238b653362946ececa1 -> 4.7.0-fc.1=s390x-csi-node-driver-registrar
      sha256:12a838a3d885b926b3c5de4a5d1dba8ce2496c514eb56d462e8b29ab789fda68 -> 4.7.0-fc.1=s390x-aws-machine-controllers
      sha256:12a838a3d885b926b3c5de4a5d1dba8ce2496c514eb56d462e8b29ab789fda68 -> 4.7.0-fc.1=s390x-azure-machine-controllers
      sha256:12a838a3d885b926b3c5de4a5d1dba8ce2496c514eb56d462e8b29ab789fda68 -> 4.7.0-fc.1=s390x-cli
      sha256:12a838a3d885b926b3c5de4a5d1dba8ce2496c514eb56d462e8b29ab789fda68 -> 4.7.0-fc.1=s390x-csi-driver-manila
      sha256:12a838a3d885b926b3c5de4a5d1dba8ce2496c514eb56d462e8b29ab789fda68 -> 4.7.0-fc.1=s390x-csi-driver-manila-operator
      sha256:12a838a3d885b926b3c5de4a5d1dba8ce2496c514eb56d462e8b29ab789fda68 -> 4.7.0-fc.1=s390x-csi-driver-nfs
      sha256:12a838a3d885b926b3c5de4a5d1dba8ce2496c514eb56d462e8b29ab789fda68 -> 4.7.0-fc.1=s390x-gcp-machine-controllers



Actual results:
The docker image tag contains the string "=s390x" instead of "-s390x".

Expected results:
The docker image tag should contain a "-s390x" string instead of "=s390x", if the docker image tag is now required to include an architectural component.

Additional info:
1. In OCP 4.6, the OCP build images did not contain the architecture type in the image name.  
2. Is the inclusion of the intended architecture type, here "s390x", a new standard set for OCP 4.7, or is this possibly an unintentional change that has been introduced?


Thank you.

Comment 1 Gabe Montero 2021-01-11 14:12:00 UTC
the tags of specific OCP s390x images with the OCP mirror is not the providence of the build component

the multi arch team needs to work with the folks at #forum-testplatform and #aos-art on slack how this issue should be addressed

among other things, it is possible they tackle issues vs. Jira bugs vs. bugzilla bugs

sending to the multi arch component to initiate those discussions with testplatform/art as needed.

Comment 2 Prashanth Sundararaman 2021-01-11 16:31:36 UTC
xref: https://bugzilla.redhat.com/show_bug.cgi?id=1878972

looks like this should already be fixed by: https://github.com/openshift/oc/pull/680 ?

Comment 3 W. Trevor King 2021-01-11 18:20:54 UTC
Per comment 2, I'm going to close this as a dup of bug 1878972.  Comment 0 here talks about two different versions of release image being mirrored, but does not talk about the version of 'oc' being used to do the mirroring, and it's the 'oc' version that matters.  Relevant history:

  $ git --no-pager log --first-parent --format='%ad %h %s' --date=short -3 origin/master -- pkg/cli/admin/r
elease
  2021-01-08 7cbe99757 Merge pull request #687 from sallyom/bz1908484-creds-request
  2020-12-21 8fbc95fdb Merge pull request #680 from wking/equal-not-valid-tag-character
  2020-12-09 0229219f5 Merge pull request #646 from jottofar/bug1878972

Use 'oc version --client' to see which version of 'oc' you have, e.g.:

  $ oc version --client
  Client Version: v4.2.0-alpha.0-873-ge575833

is using commit e575833 (you need to drop the 'g' to get the commit hash), which is from November, long before 0229219f5.  0229219f5 introduced the = issue, and 8fbc95fdb fixed it.

*** This bug has been marked as a duplicate of bug 1878972 ***

Comment 4 Prashanth Sundararaman 2021-01-11 18:43:27 UTC
Talking to @wking, the oc version command needs to be modified a little bit for this case because the version command just gives:

[root@rock-zvm-3-1 ~]# oc version
Client Version: 4.7.0-fc.0


but

[root@rock-zvm-3-1 ~]# oc version --client -o json
{
  "clientVersion": {
    "major": "",
    "minor": "",
    "gitVersion": "4.7.0-202012180107.p0-5eda527",
    "gitCommit": "5eda527621888cb03691c1fe90fcc617777fcf96",
    "gitTreeState": "clean",
    "buildDate": "2020-12-18T02:20:22Z",
    "goVersion": "go1.15.2",
    "compiler": "gc",
    "platform": "linux/s390x"
  },
  "releaseClientVersion": "4.7.0-fc.0"
}

points to the git commit which is a couple of commits behind the commit which fixed this issue.

Comment 5 krmoser 2021-01-12 02:24:59 UTC
Prashanth and Trevor, 

Thank you for the information.  I can confirm that this issue is resolved with OCP 4.7 build 4.7.0-fc.2, available at:
  https://mirror.openshift.com/pub/openshift-v4/s390x/clients/ocp-dev-preview/4.7.0-fc.2/


Thank you,
Kyle