Bug 1132980 - [RFE] Support bypassing proxy for some repository URLs
Summary: [RFE] Support bypassing proxy for some repository URLs
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Satellite 6
Classification: Red Hat
Component: Pulp
Version: 6.0.3
Hardware: Unspecified
OS: Unspecified
high
high vote
Target Milestone: Unspecified
Assignee: Justin Sherrill
QA Contact: Roman Plevka
URL:
Whiteboard:
: 1173335 1190197 1262966 1444999 1526578 (view as bug list)
Depends On:
Blocks: 1132363 1317530 CEE_Sat6_Top_BZs, GSS_Sat6_Top_Bugs 1546813 1459226 1544542
TreeView+ depends on / blocked
 
Reported: 2014-08-22 13:22 UTC by Stephen Benjamin
Modified: 2019-10-10 09:22 UTC (History)
64 users (show)

Fixed In Version: tfm-rubygem-katello-3.4.5.50-1,pulp-2.13.4.8-1
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-04-13 13:29:48 UTC


Attachments (Terms of Use)
Description of workaround for proxy issue using Squid (2.58 KB, text/plain)
2015-11-16 20:46 UTC, Eric Lavarde
no flags Details


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHBA-2018:1126 None None None 2018-04-13 13:31:33 UTC
Pulp Redmine 3210 Normal CLOSED - CURRENTRELEASE Update Importer configuration fails. 2018-02-20 21:32:04 UTC
Foreman Issue Tracker 21706 None None None 2017-11-20 14:46:37 UTC
Red Hat Knowledge Base (Solution) 2026163 None None None Never

Description Stephen Benjamin 2014-08-22 13:22:45 UTC
As a user, I would like to be able to synchronize locally hosted yum repositories without going through the configured proxy.  Possibly by accepting a proxy PAC, or some other list of excluded IP ranges.

Comment 1 RHEL Product and Program Management 2014-08-22 13:43:20 UTC
Since this issue was entered in Red Hat Bugzilla, the release flag has been
set to ? to ensure that it is properly evaluated for this release.

Comment 3 Stephen Benjamin 2014-09-26 13:51:04 UTC
It seems you can specify the proxy at the time of repo creation instead of yum_importer.json.  If we did that, then the user could have a check box to turn off the proxy


http://pulp-rpm-user-guide.readthedocs.org/en/latest/recipes.html#configure-proxy

Comment 4 Stephen Benjamin 2015-04-23 13:54:01 UTC
*** Bug 1173335 has been marked as a duplicate of this bug. ***

Comment 6 Bryan Kearney 2015-05-12 14:59:01 UTC
*** Bug 1190197 has been marked as a duplicate of this bug. ***

Comment 11 Sean Mullen 2015-08-03 15:18:47 UTC
This would effectively provide "Per Repo" proxy settings.  We definitely need this feature in my organization.  I'm facing the exact same issue.  Currently, with proxy set up, the Red Hat repos sync correctly but internal ones bomb out.  If I unset the proxy, Local repos are fine but Red Hat repos bomb out.

The only other option available at the moment that I'm aware of is using cntlm to as the proxy on the local host.  In CNTLM you can list URLs to not proxy.  This solution is a band aid for us though because I need to use personal credentials to authenticate cntlm into the proxy for the red hat connectivity.  

(In reply to Stephen Benjamin from comment #3)
> It seems you can specify the proxy at the time of repo creation instead of
> yum_importer.json.  If we did that, then the user could have a check box to
> turn off the proxy

> http://pulp-rpm-user-guide.readthedocs.org/en/latest/recipes.html#configure-
> proxy

Comment 14 vdhande 2015-09-17 13:04:56 UTC
*** Bug 1262966 has been marked as a duplicate of this bug. ***

Comment 15 pulp-infra@redhat.com 2015-09-18 15:00:15 UTC
The Pulp upstream bug status is at NEW. Updating the external tracker on this bug.

Comment 16 pulp-infra@redhat.com 2015-09-18 15:00:18 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 22 Eric Lavarde 2015-11-16 20:46:19 UTC
Created attachment 1095074 [details]
Description of workaround for proxy issue using Squid

Cleaned-up version of description how to install Squid in order to workaround the fact that certain corporate proxies don't redirect internal requests and that Satellite can only support one proxy address.

Comment 24 Sean Mullen 2015-11-16 20:53:39 UTC
Thanks Eric, cntlm can be used similarly for working around this, especially helpful if you need to log in to the outbound proxy.  That said, I hope neither of these are designated as a long term solution because adding another component (effectively a proxy for a proxy) to configure and maintain is more crap to go wrong.

Comment 41 Bryan Kearney 2016-08-04 20:19:29 UTC
Moving 6.2 bugs out to sat-backlog.

Comment 51 Brad Buckingham 2016-11-21 17:31:01 UTC
Created redmine issue http://projects.theforeman.org/issues/17425 from this bug

Comment 54 scarlet.remilia0 2016-12-15 21:41:34 UTC
The /etc/squid/squid.conf file is Puppet-managed on a Satellite 6.2 system. This needs a new workaround. RH support told me to "set up a new machine" to install Squid on, which, is just stupid.

Comment 56 Michael Hrivnak 2016-12-19 20:05:59 UTC
Changing component so Katello can take the next steps.

Comment 57 Michael Hrivnak 2016-12-19 20:12:18 UTC
If this needs to be resolved soon, I suggest disassociating the Pulp redmine tracker. That level of new feature would not likely be doable until Pulp 3, and even then it would require substantial changes to how Katello interacts with Pulp's download settings.

Comment 66 Theophanis Kontogiannis 2017-07-13 12:53:04 UTC
(In reply to Michael Hrivnak from comment #57)
> If this needs to be resolved soon, I suggest disassociating the Pulp redmine
> tracker. That level of new feature would not likely be doable until Pulp 3,
> and even then it would require substantial changes to how Katello interacts
> with Pulp's download settings.

Implementing it, will alter the constant necessity to install customized SQUIDs in secure corporate environments.

Working on a similar case right now and this would have untied our hands.

Humble opinion. Needs to be implemented asap like now.

Placing myself in CC to provide feedback as needed.

Comment 67 Bryan Kearney 2017-10-11 21:32:10 UTC
*** Bug 1444999 has been marked as a duplicate of this bug. ***

Comment 70 Michael Hrivnak 2017-11-02 15:30:02 UTC
I'm removing the Pulp issue, since we're not planning to pursue that option as part of any near-term resolution. We will however pursue that for Pulp 3.

Comment 71 Og Maciel 2017-11-02 15:32:52 UTC
Michale, since the solution would still be possibly pursued as part of a Pulp release, shouldn't we still keep the component?

Comment 73 Michael Hrivnak 2017-11-14 18:36:37 UTC
Pulp will try to make it easier to do this in the future, but there is no reason to block this issue on any Pulp work. Usually if a Pulp issue is associated via an external tracker, that means it blocks the BZ, so I think in this case we should leave it off.

Comment 75 Justin Sherrill 2017-11-20 14:46:21 UTC
Connecting redmine issue http://projects.theforeman.org/issues/21706 from this bug

Comment 78 Brad Buckingham 2017-12-08 19:39:27 UTC
Moving to POST, since the PR associated with http://projects.theforeman.org/issues/21706 is merged.

Comment 88 Justin Sherrill 2018-02-12 16:50:58 UTC
After some discussion with roman, I can now reproduce.  Here's the summary:

* The presence of this change seems to cause an issue when updating a repository in pulp if and only if Basic auth credentials are set (upstream username, upstream password)

* You do not even have to update the 'ignore proxy' setting, even updating the url will cause the error.

* I believe this is fixed upstream in https://github.com/pulp/pulp/commit/80c7b96afbf80937e0406b3c22857169aafe5b46

Steps to reproduce:

1.   satellite-installer  --katello-proxy-url=http://foo.com --katello-proxy-username=bar --katello-proxy-password=foo --disable-system-checks  --katello-proxy-port=1234

2.  Create a repository, set:
  Name: anything
  Type: yum
  Upstream Url:  Any url
  Upstream Username: admin
  Upstream password: password
Leave everything else as the default

3.  tail /var/log/messages  grepping for 'basic_auth_password'
4.  Attempt to change the upstream url to any other valid url

notice the error: 

Task pulp.server.managers.repo.importer.update_importer_config[15422909-7fcd-4926-81ca-b7b22ed629d9] raised unexpected: OperationError(u"Could not save document (Cannot update 'config.basic_auth_password' and 'config' at the same time)",)


If you revert the change this bz introduced, the problem seems to go away.

Comment 89 Justin Sherrill 2018-02-12 16:54:26 UTC
I did confirm that https://pulp.plan.io/issues/3210  (https://github.com/pulp/pulp/commit/80c7b96afbf80937e0406b3c22857169aafe5b46.patch)  appears to resolve the issue.

Comment 90 Justin Sherrill 2018-02-12 19:38:56 UTC
Split the inability to update the option from hammer into a 2nd bug:  https://bugzilla.redhat.com/show_bug.cgi?id=1544542

Comment 91 Brad Buckingham 2018-02-20 18:50:14 UTC
Since the current failure appears to be solved by a fix in pulp, updating the component to pulp and associating the upstream issue.

Comment 92 pulp-infra@redhat.com 2018-02-20 21:32:05 UTC
The Pulp upstream bug status is at CLOSED - CURRENTRELEASE. Updating the external tracker on this bug.

Comment 93 pulp-infra@redhat.com 2018-02-20 21:32:24 UTC
The Pulp upstream bug priority is at Normal. Updating the external tracker on this bug.

Comment 94 pulp-infra@redhat.com 2018-02-20 22:02:35 UTC
Requesting needsinfo from upstream developer ttereshc@redhat.com because the 'FailedQA' flag is set.

Comment 95 Tanya Tereshchenko 2018-02-21 11:36:44 UTC
Pulp part is fixed upstream. I'm removing FailedQA flag and moving BZ to POST.

Comment 98 Justin Sherrill 2018-03-23 18:48:07 UTC
*** Bug 1526578 has been marked as a duplicate of this bug. ***

Comment 99 Roman Plevka 2018-04-03 14:21:42 UTC
VERIFIED
on sat6.3.1-1

- even the authed repos now respect the flag and work properly.

Comment 102 errata-xmlrpc 2018-04-13 13:29:48 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2018:1126

Comment 103 Bryan Kearney 2018-06-29 16:32:50 UTC
This is the shizzle


Note You need to log in before you can comment on or make changes to this bug.