Bug 1554385

Summary: Synchronize pre-GA versions of Gluster/CNS/Heketi with mirror registry
Product: OpenShift Container Platform Reporter: Mike Fiedler <mifiedle>
Component: ReleaseAssignee: Justin Pierce <jupierce>
Status: CLOSED ERRATA QA Contact: Hongkai Liu <hongkliu>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 3.9.0CC: aos-bugs, hongkliu, jokerman, jupierce, mmccomas, smunilla, tbielawa
Target Milestone: ---   
Target Release: 3.10.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: No Doc Update
Doc Text:
undefined
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-07-30 19:10:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Mike Fiedler 2018-03-12 15:08:47 UTC
Description of problem:

Currently, the test team has to manually upload and docker load pre-GA CNS images to test with OCP on AWS/GCE/Azure.   This is a time consuming and error prone process.  If possible and in scope of what we can synch, it would be great if we could synchronize all tags for the  following images (internal links to be included in private comment):

rhgs3/rhgs-s3-server-rhel7
rhgs3/rhgs-server-rhel7
rhgs3/rhgs-volmanager-rhel7
rhgs3/rhgs-gluster-block-prov-rhel7


Version-Release number of selected component (if applicable): 3.9.0

Comment 4 Hongkai Liu 2018-04-24 13:40:39 UTC
Hi Justin,

How often do the images got synced?
Will all tags be synced?

For example, this is in brew but not yet in aws-reg:

checking image: rhgs3/rhgs-server-rhel7
3.3.1-14

Thanks.

Comment 5 Justin Pierce 2018-04-30 12:56:42 UTC
The job runs nightly. It pulls the most recently built image which is currently:

2018-04-30T04:07:19.880079 Pulling image: brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/rhgs3/rhgs-server-rhel7:3.3.1-10.3
2018-04-30T04:07:26.011240 [rpms/rhgs-server-container] Pushing image to mirror [retry=0]: registry.reg-aws.openshift.com:443/rhgs3/rhgs-server-rhel7:3.3.1-10.3
2018-04-30T04:07:31.851956 [rpms/rhgs-server-container] Pushing image to mirror [retry=0]: registry.reg-aws.openshift.com:443/rhgs3/rhgs-server-rhel7:3.3.1
2018-04-30T04:07:38.200023 [rpms/rhgs-server-container] Pushing image to mirror [retry=0]: registry.reg-aws.openshift.com:443/rhgs3/rhgs-server-rhel7:3.3.1-10
2018-04-30T04:07:43.725086 [rpms/rhgs-server-container] Pushing image to mirror [retry=0]: registry.reg-aws.openshift.com:443/rhgs3/rhgs-server-rhel7:3.3

Which appears correct according to:
https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=65120

Comment 6 Hongkai Liu 2018-04-30 15:08:14 UTC
Thanks Justin,

So the most recently built images are defined by build time, not the version number?

Just for my own record, i wrote down the mapping here from the product name to brew build (please correct me if my searching result is wrong):

rhgs3/rhgs-gluster-block-prov-rhel7
https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=65123

rhgs3/rhgs-volmanager-rhel7
https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=65121

rhgs3/rhgs-server-rhel7
https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=65120

rhgs3/rhgs-s3-server-rhel7
https://brewweb.engineering.redhat.com/brew/packageinfo?packageID=65122

Comment 7 Justin Pierce 2018-04-30 16:28:22 UTC
Yes - brew only really cares about build time.

Comment 8 Hongkai Liu 2018-06-07 15:34:36 UTC
Hi Justin,

it seems the sync stops working.
For example, https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=704239 on June 1 is not yet available in aws-reg.

Can you help check it out?

Comment 9 Hongkai Liu 2018-06-08 12:24:33 UTC
manually uploaded the most recent images:

checking image: rhgs3/rhgs-gluster-block-prov-rhel7
3.3.1-16
checking image: rhgs3/rhgs-volmanager-rhel7
3.3.1-17
checking image: rhgs3/rhgs-server-rhel7
3.3.1-18
checking image: rhgs3/rhgs-s3-server-rhel7
3.3.1-15

Comment 11 errata-xmlrpc 2018-07-30 19:10:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1816

Comment 12 Hongkai Liu 2018-08-28 12:01:14 UTC
Hi Justin,

the sync stopped working recently.
Could you please check?
Thanks.

Comment 14 Hongkai Liu 2018-08-28 16:28:12 UTC
Thanks for the info, Justin.

https://console.reg-aws.openshift.com/console/project/rhgs3/browse/images

CNS tags in comment 13 do not match the timestamps in aws-reg: eg,
rhgs-server-rhel7:3.4.0-5 is pushed today but it does not show in Comment 13, and
rhgs-volmanager-rhel7:3.4.0-1 has been synced 16 days ago, not today.

Anyway, i pushed the missing images manually and it is more important that sync is working in the feature. I will be watching.

Thanks again.

Comment 15 Justin Pierce 2018-08-28 17:03:22 UTC
Looking specifically at rhgs-server, it appears they have changed the branch they are building out of:

The 3.4.0 build you mention is building out of the cns-3.10.0-rhel-7 branch of http://dist-git.host.prod.eng.bos.redhat.com/cgit/containers/rhgs-server . The current synchronization job is aimed at the rhgs-3.3.1-rhel-7 branch. This would need to be reflected in the synchronization configuration: https://github.com/openshift/enterprise-images/blob/master/groups/sync-misc/images/rhgs-server.yml#L2

Comment 16 Hongkai Liu 2018-08-28 20:23:41 UTC
Justin, thanks, again.
Can we support multi-branches for syncing images?

Comment 17 Hongkai Liu 2018-09-17 14:39:55 UTC
Hello, Tim Bielawa,

the sync is not working right now.
Could you please help take a look?
Thanks.

Hongkai

Comment 18 Tim Bielawa 2018-10-08 12:55:05 UTC
We looked at this over email together. You team is building from a dist-git branch that does not match your brew tag. You have to ask RCM for a new brew tag that matches your dist-git branch (as I described in the email).

Comment 19 Hongkai Liu 2018-10-23 13:29:56 UTC
Hi Tim,

I am from qe team not and not sure if you CCed me in the email.

It stops working again.

brew-pulp-docker01.web.prod.ext.phx2.redhat.com:8888/ocs/rhgs-server-rhel7:3.11.0-5

has not been synced yet.