Bug 2046337

Summary: Certain manifest, subscription and repository related actions are broken while using HTTP Proxy as content_default_http_proxy in Satellite 6.10
Product: Red Hat Satellite Reporter: Sayan Das <saydas>
Component: Subscription ManagementAssignee: satellite6-bugs <satellite6-bugs>
Status: CLOSED ERRATA QA Contact: Cole Higgins <chiggins>
Severity: urgent Docs Contact:
Priority: high    
Version: 6.10.1CC: ahumbe, jkrajice, jpathan, jsenkyri, jsherril, jyejare, pcreech
Target Milestone: 6.11.0Keywords: Triaged
Target Release: Unused   
Hardware: All   
OS: All   
Whiteboard:
Fixed In Version: tfm-rubygem-katello-4.3.0.3-1 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 2059372 (view as bug list) Environment:
Last Closed: 2022-07-05 14:32:22 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
reproducer logs
none
reproducer task export
none
traceback from customer's satellite
none
reproducer squid logs none

Description Sayan Das 2022-01-26 16:01:30 UTC
Description of problem:

Certain actions like "Manifest Refresh", "Add Subscriptions" to manifest, Auto-Attach on Hosts, Docker type repo discovery and repo creation etc are failing while using an HTTP type proxy as the content_default_http_proxy, as the Satellite server tries to connect with candleping localhost:23443 at some point but via the proxy server and that action gets denied from proxt on localhost. 


Version-Release number of selected component (if applicable):

Satellite 6.10+

How reproducible:

Always


Steps to Reproduce:
1. Install Satellite 6.10+

2. Import a manifest with some subs in it

3. Try syncing some repos, performing manifest refresh and some repo discovery+creation of docker type content from registry.redhat.io .

4. Setup an external squid proxy which is HTTP in nature [i.e. no HTTPS\Authentication is required ]

5. Create a HTTP proxy in Satellite UI using the details of the external proxy and then Set that proxy entry as the content_default_http_proxy in Satellite.

6. Repeat the testing of all actions from step 3 while observing the satellite logs as well as the /var/log/squid/access.log from the external squid server./



Actual results:

At step 3, without proxy everything works fine.

At step 6, 

Repo sync : works fine

Manifest Refresh: Fails

Add Subscriptions: Fails

Auto-Attach on hosts: Fails

Discovery of Docker type repo: Successful But as soon as we try to create the discovery repo it fails and the task goes to Paused state.


With each of these failed attempts one constant issue is that, 

foreman reports:

403 "Forbidden" (Net::HTTPServerException)

As Satellite tries to connect to localhost:23443 via the external http proxy and proxy denies the same i.e.

1643210976.873      0 10.xx.xx.y TCP_DENIED/403 3903 CONNECT localhost:23443 - HIER_NONE/- text/html


Expected results:

Whether the proxy is HTTPS or HTTP in nature, no matter what actions are being performed, any connection to localhost or localhost:23443 should be done locally but not via proxy. 


Additional info:

For other satellite users,  the underlying cause is same but the error cound be "502 "cannotconnect" (Net::HTTPFatalError)" as well.

I have noticed that with HTTPS type proxy, certain action seems to work i.e. Manifest Refresh , Reduction of entitlement quantity of manifest etc but the action which fails, will show responses like:

503 "Service Unavailable" (Net::HTTPFatalError)

and external HTTPS squid proxy will show:

1643205659.835      2 10.xx.yy.xx TAG_NONE/503 0 CONNECT localhost:23443 satproxy HIER_NONE/- -


So again the underlying reason is the same i.e. Satellite tries to connect to candlepin on localhost:23443 via external proxy, whether the proxy is HTTP or HTTPS in nature.

Comment 1 Sayan Das 2022-01-26 16:08:58 UTC
Created attachment 1855543 [details]
reproducer logs

Comment 2 Sayan Das 2022-01-26 16:09:44 UTC
Created attachment 1855544 [details]
reproducer task export

Comment 3 Sayan Das 2022-01-26 16:10:26 UTC
Created attachment 1855546 [details]
traceback from customer's satellite

Comment 4 Sayan Das 2022-01-26 16:10:59 UTC
Created attachment 1855547 [details]
reproducer squid logs

Comment 10 Jonathon Turel 2022-02-08 20:56:16 UTC
Connecting redmine issue https://projects.theforeman.org/issues/34417 from this bug

Comment 15 Justin Sherrill 2022-02-14 19:02:57 UTC
related to https://bugzilla.redhat.com/show_bug.cgi?id=2054174   (This resolve that bz as well)

Comment 18 jcallaha 2022-04-28 18:10:22 UTC
Verified in Satellite 6.11 Snap 18


Steps: Same as the initial reporter, but with Satellite 6.11

Results:
Repo sync - Pass
Manifest Refresh - Pass
Add Subscriptions - Pass
Auto-Attach on hosts - Pass
Discovery of yum repos - Pass
    https://fixtures.pulpproject.org/
Discovery of Docker type repo - Pass
    quay.io/satelliteqe/broker

Comment 21 errata-xmlrpc 2022-07-05 14:32:22 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: Satellite 6.11 Release), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:5498