Bug 1537568 - {RFE}Leverage multiple pulp-workers for custom repository uploads
Summary: {RFE}Leverage multiple pulp-workers for custom repository uploads
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Update Infrastructure for Cloud Providers
Classification: Red Hat
Component: RHUA
Version: 3.0.0
Hardware: x86_64
OS: Linux
unspecified
low
Target Milestone: 4.2.0
: 4.x
Assignee: RHUI Bug List
QA Contact: Radek Bíba
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-01-23 14:05 UTC by Taft Sanders
Modified: 2022-10-11 13:07 UTC (History)
3 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-10-11 13:07:17 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHUI-278 0 None None None 2022-06-15 17:41:09 UTC
Red Hat Product Errata RHEA-2022:6885 0 None None None 2022-10-11 13:07:21 UTC

Description Taft Sanders 2018-01-23 14:05:47 UTC
Description of problem:
When uploading custom repository content to the RHUA server, only 1 pulp worker is seen to be doing all the work. This can cause a custom repo of several thousand packages to take days to upload.

Version-Release number of selected component (if applicable):
3.0.1

How reproducible:
Everytime

Steps to Reproduce:
1. Use reposync to sync a repository to remote or local filesystem
2. Create custom repo
3. Upload repository directory to the new custom repo

Actual results:
Upload only uses 1 pulp worker

Expected results:
Multiple pulp workers are leveraged to speed up the upload

Additional info:
[root@rhua3 entitlement_certificates]# qpid-stat -q --ssl-certificate=/etc/pki/rhui/qpid_client_striped.crt -b amqps://localhost:5671 | grep reserved_resource_worker
  reserved_resource_worker-0.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-0.redhat.com.dq             Y    Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-1.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-1.redhat.com.dq             Y    Y                 0    18     18       0   19.0k    19.0k        1     2
  reserved_resource_worker-2.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-2.redhat.com.dq             Y    Y                 0    16     16       0   17.0k    17.0k        1     2
  reserved_resource_worker-3.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-3.redhat.com.dq             Y    Y                 0    12     12       0   12.7k    12.7k        1     2
  reserved_resource_worker-4.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-4.redhat.com.dq             Y    Y                 3   526    523    3.08k   550k     546k        1     2
[root@rhua3 entitlement_certificates]# qpid-stat -q --ssl-certificate=/etc/pki/rhui/qpid_client_striped.crt -b amqps://localhost:5671 | grep reserved_resource_worker
  reserved_resource_worker-0.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-0.redhat.com.dq             Y    Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-1.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-1.redhat.com.dq             Y    Y                 0    18     18       0   19.0k    19.0k        1     2
  reserved_resource_worker-2.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-2.redhat.com.dq             Y    Y                 0    16     16       0   17.0k    17.0k        1     2
  reserved_resource_worker-3.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-3.redhat.com.dq             Y    Y                 0    12     12       0   12.7k    12.7k        1     2
  reserved_resource_worker-4.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-4.redhat.com.dq             Y    Y                 3   562    559    3.08k   587k     584k        1     2
[root@rhua3 entitlement_certificates]# qpid-stat -q --ssl-certificate=/etc/pki/rhui/qpid_client_striped.crt -b amqps://localhost:5671 | grep reserved_resource_worker
  reserved_resource_worker-0.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-0.redhat.com.dq             Y    Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-1.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-1.redhat.com.dq             Y    Y                 0    18     18       0   19.0k    19.0k        1     2
  reserved_resource_worker-2.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-2.redhat.com.dq             Y    Y                 0    16     16       0   17.0k    17.0k        1     2
  reserved_resource_worker-3.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-3.redhat.com.dq             Y    Y                 0    12     12       0   12.7k    12.7k        1     2
  reserved_resource_worker-4.redhat.com.celery.pidbox       Y                 0     0      0       0      0        0         1     2
  reserved_resource_worker-4.redhat.com.dq             Y    Y                 3   602    599    3.08k   629k     626k        1     2

Comment 7 errata-xmlrpc 2022-10-11 13:07:17 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (RHUI 4.2.0 release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2022:6885


Note You need to log in before you can comment on or make changes to this bug.