Description of problem:
We are using a proxy to download all content from the internet.
The proxy is configured by the satellite-installer using the --katello-proxy-url option.
This works correctly for all parts except the pulp_streamer processes. Even the pulp metadata download worked with the proxy settings.
But when it comes to download an RPM file on the client.
https://li-lc-1578.hag.hilti.com/pulp/repos/Hilti/Library/hg-crash/content/dist/rhel/server/6/6.8/x86_64/os/redhat-release-server-6Server-188.8.131.52.el6.x86_64.rpm: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 503 Service Unavailable"
On the pulp server with streamer we see:
Jun 28 13:30:41 li-lc-1578 pulp_streamer: nectar.downloaders.threaded:DEBUG: Attempting to connect to https://cdn.redhat.com/content/dist/rhel/server/6/6.8/x86_64/os/Packages/redhat-release-server-6Server-184.108.40.206.el6.x86_64.rpm.
Jun 28 13:30:41 li-lc-1578 pulp_streamer: requests.packages.urllib3.connectionpool:INFO: Starting new HTTPS connection (2): cdn.redhat.com
Jun 28 13:30:47 li-lc-1578 pulp_streamer: nectar.downloaders.threaded:ERROR: Skipping requests to cdn.redhat.com due to repeated connection failures: HTTPSConnectionPool(host='cdn.redhat.com', port=443): Max retries exceeded with url: /content/dist/rhel/server/6/6.8/x86_64/os/Packages/redhat-release-server-6Server-220.127.116.11.el6.x86_64.rpm (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x36a68d0>, 'Connection to cdn.redhat.com timed out. (connect timeout=6.05)'))
Jun 28 13:30:47 li-lc-1578 pulp_streamer: [-] 127.0.0.1 - - [28/Jun/2016:13:30:46 +0000] "GET /var/lib/pulp/content/units/rpm/de/4659f4f8037b876ca7fc0b45b614289f28f8985bccaa64b8c037e065e6d19a/redhat-release-server-6Server-18.104.22.168.el6.x86_64.rpm HTTP/1.1" 503 - "-" "urlgrabber/3.9.1 yum/3.2.29"
On our upstream proxy, there is no request to cdn.redhat.com visible. This makes me believe that the proxy configuration is not used by pulp_streamer.
Doing a grep for proxy configuration in /etc/pulp shows that the pulp_streamer has no proxy configuration defined:
[crash] root@li-lc-1578:/etc/pulp# grep -Rl proxy
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Use external proxy to connect to CDN, makes sure to not allow direct internet access from the satellite
2. Create clean satellite with the --katello-proxy-url option to the the external proxy
2. Enable on_demand in settings, this must be before any repo is created to have it set as default
3. Sync RedHat manifest
4. Select some RedHat repos to sync
5. Create content view and activation key
6. Assign RedHat repos to a content hosts
7. yum repolist
8. yum update (or install, just make sure a package will be downlaoded)
Step 7 repolist works
Step 8 package download fails
The Pulp upstream bug status is at ASSIGNED. Updating the external tracker on this bug.
The Pulp upstream bug priority is at High. Updating the external tracker on this bug.
The Pulp upstream bug status is at POST. Updating the external tracker on this bug.
The Pulp upstream bug status is at MODIFIED. Updating the external tracker on this bug.
Confirmed the patch from https://patch-diff.githubusercontent.com/raw/pulp/pulp/pull/2649.diff works.
The Pulp upstream bug status is at ON_QA. Updating the external tracker on this bug.
The Pulp upstream bug status is at VERIFIED. Updating the external tracker on this bug.
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.
Verified in Satellite 6.2.3 Snap 4.
Successfully streamed over 2,500 packages to multiple hosts at once from a Satellite behind a proxy. No errors seen on either Satellite or Host ends.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.