Bug 1350848 - [lazy sync] on_demand streamer process download fails to download RPMs content if proxy is used
Summary: [lazy sync] on_demand streamer process download fails to download RPMs conten...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite
Classification: Red Hat
Component: Pulp
Version: 6.2.0
Hardware: Unspecified
OS: Unspecified
unspecified
medium vote
Target Milestone: Unspecified
Assignee: satellite6-bugs
QA Contact: jcallaha
URL:
Whiteboard:
Depends On:
Blocks: 1122832
TreeView+ depends on / blocked
 
Reported: 2016-06-28 13:44 UTC by Peter Vreman
Modified: 2019-08-12 14:08 UTC (History)
11 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-10-26 12:26:29 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Pulp Redmine 2038 High CLOSED - CURRENTRELEASE The nectar config pulp_streamer gets doesn't seem to include plugin-level settings 2016-08-19 14:00:41 UTC
Red Hat Product Errata RHBA-2016:2108 normal SHIPPED_LIVE Satellite 6.2.3 Async Bug Release 2016-10-26 16:21:52 UTC

Description Peter Vreman 2016-06-28 13:44:33 UTC
Description of problem:
We are using a proxy to download all content from the internet. 
The proxy is configured by the satellite-installer using the --katello-proxy-url option.
This works correctly for all parts except the pulp_streamer processes. Even the pulp metadata download worked with the proxy settings.
But when it comes to download an RPM file on the client.

https://li-lc-1578.hag.hilti.com/pulp/repos/Hilti/Library/hg-crash/content/dist/rhel/server/6/6.8/x86_64/os/redhat-release-server-6Server-6.8.0.5.el6.x86_64.rpm: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 503 Service Unavailable"


On the pulp server with streamer we see:

Jun 28 13:30:41 li-lc-1578 pulp_streamer: nectar.downloaders.threaded:DEBUG: Attempting to connect to https://cdn.redhat.com/content/dist/rhel/server/6/6.8/x86_64/os/Packages/redhat-release-server-6Server-6.8.0.5.el6.x86_64.rpm.
Jun 28 13:30:41 li-lc-1578 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (2): cdn.redhat.com
Jun 28 13:30:47 li-lc-1578 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to cdn.redhat.com due to repeated connection failures: HTTPSConnectionPool(host='cdn.redhat.com', port=443): Max retries exceeded with url: /content/dist/rhel/server/6/6.8/x86_64/os/Packages/redhat-release-server-6Server-6.8.0.5.el6.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x36a68d0>, 'Connection to cdn.redhat.com timed out. (connect timeout=6.05)'))
Jun 28 13:30:47 li-lc-1578 pulp_streamer: [-] 127.0.0.1 - - [28/Jun/2016:13:30:46 +0000] "GET /var/lib/pulp/content/units/rpm/de/4659f4f8037b876ca7fc0b45b614289f28f8985bccaa64b8c037e065e6d19a/redhat-release-server-6Server-6.8.0.5.el6.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.9.1 yum/3.2.29"

On our upstream proxy, there is no request to cdn.redhat.com visible. This makes me believe that the proxy configuration is not used by pulp_streamer.
Doing a grep for proxy configuration in /etc/pulp shows that the pulp_streamer has no proxy configuration defined:

[crash] root@li-lc-1578:/etc/pulp# grep -Rl proxy
server/plugins.conf.d/docker_importer.json
server/plugins.conf.d/yum_importer.json
server/plugins.conf.d/puppet_importer.json
server/plugins.conf.d/iso_importer.json
server.conf



Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Use external proxy to connect to CDN, makes sure to not allow direct internet access from the satellite
2. Create clean satellite with the --katello-proxy-url option to the the external proxy
2. Enable on_demand in settings, this must be before any repo is created to have it set as default
3. Sync RedHat manifest
4. Select some RedHat repos to sync
5. Create content view and activation key
6. Assign RedHat repos to a content hosts
7. yum repolist
8. yum update (or install, just make sure a package will be downlaoded)

Actual results:
Step 7 repolist works
Step 8 package download fails

Expected results:
Success

Additional info:

Comment 3 pulp-infra@redhat.com 2016-07-18 19:30:33 UTC
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.

Comment 4 pulp-infra@redhat.com 2016-07-18 19:30:36 UTC
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.

Comment 5 pulp-infra@redhat.com 2016-07-20 19:00:25 UTC
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.

Comment 6 pulp-infra@redhat.com 2016-07-22 00:00:26 UTC
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.

Comment 7 Peter Vreman 2016-07-22 11:22:22 UTC
Confirmed the patch from https://patch-diff.githubusercontent.com/raw/pulp/pulp/pull/2649.diff works.

Comment 8 pulp-infra@redhat.com 2016-07-26 18:30:34 UTC
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.

Comment 9 pulp-infra@redhat.com 2016-07-26 20:30:29 UTC
The Pulp upstream bug status is at VERIFIED. Updating the external tracker on this bug.

Comment 11 pulp-infra@redhat.com 2016-08-19 14:00:43 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 13 jcallaha 2016-10-25 18:59:54 UTC
Verified in Satellite 6.2.3 Snap 4.

Successfully streamed over 2,500 packages to multiple hosts at once from a Satellite behind a proxy. No errors seen on either Satellite or Host ends.

Comment 15 errata-xmlrpc 2016-10-26 12:26:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2016:2108


Note You need to log in before you can comment on or make changes to this bug.