| Summary: | [Azure] While mirroring images to private registry throwing error: invalid character '<' looking for beginning of value | |||
|---|---|---|---|---|
| Product: | OpenShift Container Platform | Reporter: | Neeraj <nbhatt> | |
| Component: | oc | Assignee: | Maciej Szulik <maszulik> | |
| Status: | CLOSED ERRATA | QA Contact: | zhou ying <yinzhou> | |
| Severity: | high | Docs Contact: | ||
| Priority: | high | |||
| Version: | 4.6 | CC: | akaris, aos-bugs, christian.affolter, dgautam, igarciam, jocolema, jokerman, kechung, ksathe, maszulik, mfojtik, mmohan, oarribas, somalley, sychen, yinzhou | |
| Target Milestone: | --- | |||
| Target Release: | 4.8.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | Doc Type: | Bug Fix | ||
| Doc Text: |
Cause:
During mirroring we create an authorization header with all the image information.
Consequence:
Some registries have limits on header size which are violated by the size.
Fix:
Force --skip-multiple-scopes for oc adm catalog mirror command.
Result:
Mirroring works as expected.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1976284 (view as bug list) | Environment: | ||
| Last Closed: | 2021-07-27 22:57:54 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Bug Depends On: | ||||
| Bug Blocks: | 1976284 | |||
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory (Moderate: OpenShift Container Platform 4.8.2 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2021:2438 |
Description of problem: So when we are trying to mirror the image from the pruned image to private registry by running the below command : oc adm catalog mirror \ xyz.azurecr.io/redhat/redhat-operator-index:v4.6 \ xyz.azurecr.io \ -a ${REG_CREDS} \ --filter-by-os='.*' I get the below error for all the images that we are trying to push. below is just 1 sample. error: unable to push manifest to xyz.azurecr.io/rhel8/skopeo:9fc07dfb: Put "https://xyz.azurecr.io/v2/rhel8/skopeo/manifests/9fc07dfb": error parsing HTTP 414 response body: invalid character '<' looking for beginning of value: "<html>\r\n<head><title>414 Request-URI Too Large</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>414 Request-URI Too Large</h1></center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" Version-Release number of selected component (if applicable): - OCP 4.6 How reproducible: - everytime Steps to Reproduce: I have tested this in my test env on azure. These are the steps which I have followed: - Podman version should be >= 1.19 - podman login asmereg.azurecr.io -u asmereg -p <passwd> - podman login registry.redhat.io - podman run -p50051:50051 -it registry.redhat.io/redhat/redhat-operator-index:v4.7 - opm index prune -f registry.redhat.io/redhat/redhat-operator-index:v4.7 -p advanced-cluster-management,jaeger-product,quay-operator -t asmereg.azurecr.io/redhat/redhat-operator-index:v4.7 - podman push asmereg.azurecr.io/redhat/redhat-operator-index:v4.7 - REG_CREDS=/run/containers/0/auth.json - oc adm catalog mirror asmereg.azurecr.io/redhat/redhat-operator-index:v4.7 asmereg.azurecr.io -a ${REG_CREDS} Actual results: ~~~ /title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>400 Bad Request</h1></center>\r\n<center>Request Header Or Cookie Too Large</center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" error: unable to push manifest to asmereg.azurecr.io/rhacm2/thanos-rhel7:1eb45703: error parsing HTTP 400 response body: invalid character '<' looking for beginning of value: "<html>\r\n<head><title>400 Request Header Or Cookie Too Large</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>400 Bad Request</h1></center>\r\n<center>Request Header Or Cookie Too Large</center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" error: unable to push manifest to asmereg.azurecr.io/rhacm2/thanos-rhel7:47118979: error parsing HTTP 400 response body: invalid character '<' looking for beginning of value: "<html>\r\n<head><title>400 Request Header Or Cookie Too Large</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>400 Bad Request</h1></center>\r\n<center>Request Header Or Cookie Too Large</center>\r\n<hr><center>openresty</center>\r\n</body>\r\n</html>\r\n" info: Mirroring completed in 15m51.02s (0B/s) error mirroring image: one or more errors occurred no digest mapping available for asmereg.azurecr.io/redhat/redhat-operator-index:v4.7, skip writing to ImageContentSourcePolicy wrote mirroring manifests to manifests-redhat-operator-index-1617354430 ~~~ Expected results: It should mirror images without any errors.