Bug 1690070

Summary: publishing promoting large docker repos in a content view can take a long time
Product: Red Hat Satellite Reporter: Justin Sherrill <jsherril>
Component: PulpAssignee: amacdona <austin>
Status: CLOSED ERRATA QA Contact: Brian Herring <bherring>
Severity: medium Docs Contact:
Priority: unspecified    
Version: 6.5.0CC: austin, bherring, bmbouter, daviddavis, dkliban, egolov, ggainey, ipanova, mmccune, pmoravec, rchan, ttereshc
Target Milestone: 6.6.0Keywords: Triaged
Target Release: Unused   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: pulp-docker-3.2.3.1-2 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1756048 (view as bug list) Environment:
Last Closed: 2019-10-22 12:47:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
pulp-docker-plugins-3.2.2-2.HOTFIXRHBZ1690070.el7sat.noarch.rpm none

Description Justin Sherrill 2019-03-18 17:34:48 UTC
Description of problem:

As part of content view publish/promote, we remove docker content from an docker existing docker repo in a lifecycle environment.  For repos with large amounts of content, this can take a long time:

~300 docker manifests seems to take ~2 minutes
~2000 docker manifests seems to take ~30 minutes

Version-Release number of selected component (if applicable):

6.4.0

How reproducible:
Always

Steps to Reproduce:
1.  Sync a docker repo: https://quay.io   datawire/ambassador
2.  Publish the docker repo in a CV
3.  Re-publish the CV

Actual results:
Very slow

Expected results:
Should be fast

Additional info:


It appears that removing the docker manifests units in pulp is the slow piece

Comment 3 pulp-infra@redhat.com 2019-03-18 19:31:40 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 4 pulp-infra@redhat.com 2019-03-18 19:31:41 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 5 pulp-infra@redhat.com 2019-03-27 20:31:42 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 6 pulp-infra@redhat.com 2019-04-03 13:10:52 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 7 pulp-infra@redhat.com 2019-07-09 03:22:43 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 8 amacdona@redhat.com 2019-07-19 19:21:12 UTC
I am proceeding with a fix that won't require change katello changes.

There was some discussion about a potential workaround (https://pulp.plan.io/issues/4549#note-12) but that will not be necessary.

Comment 9 pulp-infra@redhat.com 2019-07-29 18:01:23 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 10 amacdona@redhat.com 2019-07-29 18:37:23 UTC
I have an open PR for this issue. 2 things need to be highlighted:

1. The patch (as a side effect) fixed a bug, so there will be a subtle behavior change. https://pulp.plan.io/issues/5181
2. The patch will increase memory usage for removal tasks. On the issue, I put up some numbers and recommend going forward with this patch: https://pulp.plan.io/issues/4549#note-17

Comment 11 pulp-infra@redhat.com 2019-08-01 17:32:44 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 12 pulp-infra@redhat.com 2019-08-01 18:02:45 UTC
All upstream Pulp bugs are at MODIFIED+. Moving this bug to POST.

Comment 13 amacdona@redhat.com 2019-08-08 15:48:21 UTC
Functional tests have been written (not yet merged): https://github.com/PulpQE/Pulp-2-Tests/pull/213

Comment 15 Mike McCune 2019-08-09 15:18:27 UTC
Created attachment 1602240 [details]
pulp-docker-plugins-3.2.2-2.HOTFIXRHBZ1690070.el7sat.noarch.rpm

Comment 16 Mike McCune 2019-08-09 15:19:36 UTC
*** Satellite 6.5.2 Hotfix Available ***

1) Download pulp-docker-plugins-3.2.2-2.HOTFIXRHBZ1690070.el7sat.noarch.rpm from this bugzilla to your Satellite

2) Install:

rpm -Uvh pulp-docker-plugins-3.2.2-2.HOTFIXRHBZ1690070.el7sat.noarch.rpm

3) restart:

foreman-maintain service restart

4) resume operations

Comment 18 pulp-infra@redhat.com 2019-09-25 21:02:11 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 21 errata-xmlrpc 2019-10-22 12:47:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2019:3172

Comment 22 Pavel Moravec 2019-12-02 14:49:50 UTC
*** Bug 1768226 has been marked as a duplicate of this bug. ***