Bug 2106461

Summary: oc-mirror fails to sync to disconnected registry
Product: OpenShift Container Platform Reporter: Mike Allmen <mallmen>
Component: ocAssignee: Ross Peoples <rpeoples>
oc sub component: oc-mirror QA Contact: zhou ying <yinzhou>
Status: CLOSED WONTFIX Docs Contact:
Severity: unspecified    
Priority: unspecified CC: jpower, vdinh
Version: 4.11   
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2023-01-17 14:27:13 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
original config none

Description Mike Allmen 2022-07-12 17:55:54 UTC 
Created attachment 1896458 [details]
original config

Created attachment 1896458 [details]
original config

Description of problem:
When using an imageset-config.yaml that has an OpenShift release, a few Red Hat operators, and a Community operator, it fails to upload to the disconnected registry.

Version-Release number of selected component (if applicable):
$ podman exec -it oc-mirror oc-mirror version
Client Version: version.Info{Major:"", Minor:"", GitVersion:"v0.2.0-alpha.1-55-gf4d5f4b", GitCommit:"f4d5f4b5", GitTreeState:"dirty", BuildDate:"2022-06-17T22:07:53Z", GoVersion:"go1.18.3", Compiler:"gc", Platform:"linux/amd64"}

How reproducible:


Steps to Reproduce:
1.  Populate registry initially using OpenShift release, and a few Red Hat operators 
       oc-mirror --config imageset-config.yaml.orig file://archives
       oc-mirror --from archives docker://<registry>)
2.  Update imageset-config.yaml to add Grafana Community operator
       add grafana-community operator to images-config.yaml.orig
3.  Pull updates
       oc-mirror --config imageset-config.yaml file://archives
4.  Push of update fails
       oc-mirror --from archives docker://<registry>

Actual results:
info: Mirroring completed in 340ms (143.7kB/s)
Wrote release signatures to oc-mirror-workspace/results-1657636245
Rendering catalog image "registry-quay-quay.apps.ocp4.example.com/redhat/community-operator-index:v4.10" with file-based catalog
Rendering catalog image "registry-quay-quay.apps.ocp4.example.com/redhat/redhat-operator-index:v4.10" with file-based catalog
error: error rebuilding catalog images from file-based catalogs: error creating OCI layout: stat oc-mirror-workspace/images.1969864861/catalogs/registry.redhat.io/redhat/redhat-operator-index/v4.10/layout/index.json: no such file or directory

Expected results:
syncing finishes and generates catalog image, imagecontentsourcepolicy, etc.

Additional info:
Comment 3 Jennifer Power 2022-07-12 18:18:13 UTC 
For a bit more background. This bug has to do with how directories are unpacked when multiple imageset spans multiple archives. In this case, the 'archiveSize' is smaller than the combined size of the OCI layouts for these two catalogs. A possible solution is to update the structure returned from this function (https://github.com/openshift/oc-mirror/blob/c1cd75916009557cbb4c02402f0b0c14edb6de7b/pkg/bundle/files.go#L96) to allow multiple key values so the archive containing the directories is not overwritten.
Comment 4 Jennifer Power 2022-07-13 20:22:35 UTC 
Correction on the previous comment. Please disregard the proposed solution. The directory itself is only stored in one archive.