Bug 1933716

Summary: pulp3: Syncing container content to the external Capsule fails
Product: Red Hat Satellite Reporter: Vladimír Sedmík <vsedmik>
Component: RepositoriesAssignee: Justin Sherrill <jsherril>
Status: CLOSED ERRATA QA Contact: Tasos Papaioannou <tpapaioa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.9.0CC: bmbouter, egolov, ggainey, ipanova, jsherril, mjia, pcreech, rchan, smallamp, ttereshc, zhunting
Target Milestone: 6.9.1Keywords: Triaged
Target Release: Unused   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: python-pulp-container-2.1.1,tfm-rubygem-katello-3.18.1.23-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2021-04-26 17:36:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Vladimír Sedmík 2021-03-01 14:40:38 UTC
Description of problem:
Having a Satellite with container type repositories synced and migrated to pulp3, consequential attempts to sync the repos to the external capsule fail.


Version-Release number of selected component (if applicable):
6.9.0 snap 12
python3-pulp-container-2.1.0-1.el7pc.noarch
tfm-rubygem-pulp_container_client-2.1.0-1.el7sat.noarch


How reproducible:
always


Steps to Reproduce:
1. Have a Satellite with registered external Capsule (Library LCE)
2. Sync some container images to the Satellite using Products > Repo Discovery
   (I used quay/busybox and registry.redhat.io/dotnet/dotnetcore-10-rhel7)
3. Run content migration and switchover (succeeds)
4. Trigger the Capsule sync


Actual results:
Capsule sync fails with error
Katello::Errors::PulpError: PLP0000: 500 Server Error: 'Internal Server Error' for url: https://<SAT>/v2/default_organizationnet_core-dotnet_dotnetcore-10-rhel7/manifests/1.0


Expected results:
Successful sync


Additional info:
Traceback:
[2021-02-22 16:50:44 +0000] [1909] [ERROR] Error handling request
Traceback (most recent call last):
File "/usr/lib64/python3.6/site-packages/aiohttp/web_protocol.py", line 418, in start
resp = await task
File "/usr/lib64/python3.6/site-packages/aiohttp/web_app.py", line 458, in _handle
resp = await handler(request)
File "/usr/lib/python3.6/site-packages/pulp_container/app/registry.py", line 144, in get_tag
return await Registry.dispatch_converted_schema(tag, accepted_media_types, path)
File "/usr/lib/python3.6/site-packages/pulp_container/app/registry.py", line 191, in dispatch_converted_schema
schema, converted, digest = schema1_converter.convert()
File "/usr/lib/python3.6/site-packages/pulp_container/app/schema_convert.py", line 42, in convert
converted_schema = self._convert_schema(legacy)
File "/usr/lib/python3.6/site-packages/pulp_container/app/schema_convert.py", line 57, in _convert_schema
return converter.convert()
File "/usr/lib/python3.6/site-packages/pulp_container/app/schema_convert.py", line 101, in convert
self.compute_layers()
File "/usr/lib/python3.6/site-packages/pulp_container/app/schema_convert.py", line 133, in compute_layers
layer_id, fs_layer, last_layer=(i == fs_layers_count - 1)
File "/usr/lib/python3.6/site-packages/pulp_container/app/schema_convert.py", line 177, in _compute_v1_compatibility_config
container_config=dict(Cmd=fs_layer.history["created_by"]),
KeyError: 'created_by'

Comment 1 pulp-infra@redhat.com 2021-03-01 16:10:00 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 2 pulp-infra@redhat.com 2021-03-01 16:10:02 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 3 pulp-infra@redhat.com 2021-03-01 20:06:50 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 4 Brad Buckingham 2021-03-04 15:33:45 UTC
Does this error only occur with a pulp3 server and pulp2 capsule configuration?

Or does it also occur for pulp2 server and pulp2 capsule?

Comment 5 Vladimír Sedmík 2021-03-04 15:52:11 UTC
This one occurs with pulp3 sat and pulp2 caps.

There is another one, pulp2 sat + pulp2 caps related here: https://bugzilla.redhat.com/show_bug.cgi?id=1933770

Comment 6 pulp-infra@redhat.com 2021-03-04 20:07:06 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 7 pulp-infra@redhat.com 2021-03-04 21:06:02 UTC
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.

Comment 8 pulp-infra@redhat.com 2021-03-15 12:08:26 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 9 pulp-infra@redhat.com 2021-03-15 12:08:28 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 12 Justin Sherrill 2021-03-30 14:02:41 UTC
Connecting redmine issue https://projects.theforeman.org/issues/32230 from this bug

Comment 13 pulp-infra@redhat.com 2021-03-30 14:12:10 UTC
Requesting needsinfo from upstream developer ipanova because the 'FailedQA' flag is set.

Comment 15 Bryan Kearney 2021-03-31 16:07:16 UTC
Upstream bug assigned to jsherril

Comment 16 Bryan Kearney 2021-03-31 16:07:18 UTC
Upstream bug assigned to jsherril

Comment 17 Evgeni Golov 2021-04-01 11:59:07 UTC
is it right, this needs pulp-container AND the katello fix, and the latter one is what's missing?

Comment 18 Justin Sherrill 2021-04-01 12:56:37 UTC
Thats correct, the latter one is a newly identified issue

Comment 20 Tasos Papaioannou 2021-04-13 15:19:16 UTC
Verified on 6.9.1 snap 1.0.

Comment 26 errata-xmlrpc 2021-04-26 17:36:58 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Satellite 6.9.1 Async Bug Fix Update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2021:1371