Bug 1443164 - no_proxy does not match the host name when *.redhat.com is used
Summary: no_proxy does not match the host name when *.redhat.com is used
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: subscription-manager
Version: 7.4
Hardware: Unspecified
OS: Unspecified
medium
low
Target Milestone: rc
: ---
Assignee: Jiri Hnidek
QA Contact: John Sefler
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2017-04-18 16:10 UTC by Rehana
Modified: 2017-08-01 19:22 UTC (History)
4 users (show)

Fixed In Version: python-rhsm-1.19.8-1
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-08-01 19:22:43 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Github candlepin subscription-manager pull 1635 None closed 1443164: no_proxy matches the host name when *.redhat.com is used 2020-08-18 15:41:46 UTC
Red Hat Product Errata RHBA-2017:2083 normal SHIPPED_LIVE python-rhsm and subscription-manager bug fix and enhancement update 2017-08-01 18:14:19 UTC

Description Rehana 2017-04-18 16:10:24 UTC
Description of problem:
As per the requirement both no_proxy environment variable and --noproxy  option should match the host name when Asterisks are used to match a hostnames

Version-Release number of selected component (if applicable):
subscription management server: 0.9.51.21-1
subscription management rules: 5.15.1
subscription-manager: 1.19.8-1.el7
python-rhsm: 1.19.5-1.el7


How reproducible:
always

Steps to Reproduce:
1.Try to register using 
no_proxy=*.redhat.com subscription-manager register --username=stage_test_** --password=*** --proxy=auto-services.usersys.redhat.com:3128 --proxyuser=redhat --proxypassword=redhat --force
 OR 

subscription-manager register --username=stage_test_** --password=*** --proxy=auto-services.usersys.redhat.com:3128 --proxyuser=redhat
--proxypassword=redhat --force  --noproxy=*.redhat.com 


Actual results:
Observed that though the hostname on the system was subscriptions.rhsm.stage.com , the no_proxy option didnot find the match , and due to which the traffic was going though proxy server 

Expected results:
both no_proxy environment variable and --noproxy should find the match with wildcard * is used

Additional info:

Comment 1 Jiri Hnidek 2017-05-23 13:08:53 UTC
The problem is in python module urllib (part of Python installation):

def proxy_bypass_environment(host):
    """Test if proxies should not be used for a particular host.

    Checks the environment for a variable named no_proxy, which should
    be a list of DNS suffixes separated by commas, or '*' for all hosts.
    """
    no_proxy = os.environ.get('no_proxy', '') or os.environ.get('NO_PROXY', '')
    # '*' is special case for always bypass
    if no_proxy == '*':
        return 1
    # strip port off host
    hostonly, port = splitport(host)
    # check if the host ends with any of the DNS suffixes
    no_proxy_list = [proxy.strip() for proxy in no_proxy.split(',')]
    for name in no_proxy_list:
        if name and (hostonly.endswith(name) or host.endswith(name)):
            return 1
    # otherwise, don't bypass
    return 0

No wildcard characters are supported. To add support for *.example.com it would be enough to change comprehension list:

    no_proxy_list = [proxy.strip() for proxy in no_proxy.split(',')]

into:

    no_proxy_list = [proxy.strip().strip('*') for proxy in no_proxy.split(',')]

Comment 3 Rehana 2017-05-31 09:58:25 UTC
Reproducing the failure:
==========================

# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: 2.1.1-1
subscription management rules: 5.23
subscription-manager: 1.19.16-1.el7
python-rhsm: 1.19.7-1.el7


[root@dhcp151-206 ~]# no_proxy=*.redhat.com subscription-manager register --username=**** --password=*** --proxy=auto-services.usersys.redhat.com:3128 --proxyuser=redhat --proxypassword=redhat --forceRegistering to: F21-candlepin.usersys.redhat.com:8443/candlepin
Organization: admin
The system has been registered with ID: a45eb406-0c6d-4370-a888-e7b8b31d17a2 

[root@auto-services ~]# tail -f /var/log/squid/access.log 
1496222149.551   1205 10.16.151.206 TCP_MISS/200 1745 CONNECT f21-candlepin.usersys.redhat.com:8443 redhat HIER_DIRECT/10.70.35.226 -
1496222150.930   1215 10.16.151.206 TCP_MISS/200 2752 CONNECT f21-candlepin.usersys.redhat.com:8443 redhat HIER_DIRECT/10.70.35.226 -
1496222158.543   1407 10.16.151.206 TCP_MISS/200 2609 CONNECT f21-candlepin.usersys.redhat.com:8443 redhat HIER_DIRECT/10.70.35.226 -
1496222160.567   1980 10.16.151.206 TCP_MISS/200 14310 CONNECT f21-candlepin.usersys.redhat.com:8443 redhat HIER_DIRECT/10.70.35.226 -
1496222161.836   1222 10.16.151.206 TCP_MISS/200 2609 CONNECT f21-candlepin.usersys.redhat.com:8443 redhat HIER_DIRECT/10.70.35.226 -
1496222163.133   1255 10.16.151.206 TCP_MISS/200 1745 CONNECT f21-candlepin.usersys.redhat.com:8443 redhat HIER_DIRECT/10.70.35.226 -
1496222164.415   1239 10.16.151.206 TCP_MISS/200 2609 CONNECT f21-candlepin.usersys.redhat.com:8443 redhat HIER_DIRECT/10.70.35.226 -
1496222165.655   1196 10.16.151.206 TCP_MISS/200 1745 CONNECT f21-candlepin.usersys.redhat.com:8443 redhat HIER_DIRECT/10.70.35.226 -
1496222166.989   1290 10.16.151.206 TCP_MISS/200 1233 CONNECT f21-candlepin.usersys.redhat.com:8443 redhat HIER_DIRECT/10.70.35.226 -
1496222168.280   1247 10.16.151.206 TCP_MISS/200 1969 CONNECT f21-candlepin.usersys.redhat.com:8443 redhat HIER_DIRECT/10.70.35.226 -

^^ register request is passed through proxy though the candlepin server hostname was matching the criteria *.redhat.com

Retesting with latest subscription-manager packages :
=======================================================





scenario 1: register with no_proxy configured with *.redhat.com , expect the registration to pass by bypassing the proxy server
 
[root@dhcp151-206 ~]# no_proxy=*.redhat.com subscription-manager register --username=admin --password=admin --proxy=auto-services.usersys.redhat.com:3128 --proxyuser=redhat --proxypassword=redhat --force
Registering to: F21-candlepin.usersys.redhat.com:8443/candlepin
Organization: admin
The system has been registered with ID: bf90061f-81f9-44d5-a859-e7d60fbe8d0b 
[root@dhcp151-206 ~]# 

^^No log entry for this request was found in proxy logs , hence scenario 1 is PASS

scenario 2 : Set a bad proxy on the system , set no_proxy variable to *.usersys.redhat.com and verify the subscription-manager auto-attach request passes

#  subscription-manager config --server.proxy_hostname=notarealproxy.example.com --server.proxy_port=1100

# subscription-manager attach --auto

Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.

^^ with out no_proxy configured the auto-attach request failed with bad proxy --> as expected.

# no_proxy=*.usersys.redhat.com subscription-manager attach --auto

Installed Product Current Status:
Product Name: Awesome OS Instance Server Bits
Status:       Subscribed

Product Name: Red Hat Enterprise Linux 7 Server High Touch Beta
Status:       Not Subscribed

Unable to find available subscriptions for all your installed products.

 no_proxy=*.usersys.redhat.com matched "F21-candlepin.usersys.redhat.com" hostname ,so the auto-attach passed despite the bad proxy details in the system.

however a new regression issue caused due this fix is tracked in a separate bug 1457197

holding on Verifying this bug , until bug 1457197 is fixed

Comment 4 Rehana 2017-05-31 10:03:29 UTC
Missed to mention the package version used to retest the fix :

# subscription-manager version
server type: Red Hat Subscription Management
subscription management server: 2.1.1-1
subscription management rules: 5.23
subscription-manager: 1.19.17-1.el7
python-rhsm: 1.19.8-1.el7

Comment 5 Rehana 2017-06-08 10:22:29 UTC
Retesting post the verification of bug 1457197 

Verifying on :
=============

subscription management server: 2.1.1-1
subscription management rules: 5.23
subscription-manager: 1.19.18-1.el7
python-rhsm: 1.19.9-1.el7


Scenario 1:
===========
 Register with no_proxy configured with *.redhat.com , expect the registration to pass by bypassing the proxy server

Verification : 

# no_proxy=*.redhat.com subscription-manager register --username=admin --password=admin --proxy=auto-services.usersys.redhat.com:3128 --proxyuser=redhat --proxypassword=redhat
Registering to: F21-candlepin.usersys.redhat.com:8443/candlepin
Organization: admin
The system has been registered with ID: 62c30ee2-5d03-43b8-98e8-ba1a5d21fa49 

No log entry for this request was found in proxy logs , hence scenario 1 is PASS


Scenario 2:
============
   Set a bad proxy on the system , set no_proxy variable to *.usersys.redhat.com and verify the subscription-manager auto-attach request passes

Verification:

# subscription-manager config --server.proxy_hostname=notarealproxy.example.com --server.proxy_port=1100

# cat /etc/rhsm/rhsm.conf | grep proxy_
proxy_hostname =notarealproxy.example.com
proxy_port =1100
proxy_user =
proxy_password =

#  subscription-manager attach --auto
Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.

with no no_proxy the auto-attach request fails --Expected 

# no_proxy=*.usersys.redhat.com subscription-manager attach --auto

Installed Product Current Status:
Product Name: Red Hat Enterprise Linux Server
Status:       Not Subscribed

Product Name: Awesome OS Instance Server Bits
Status:       Subscribed

Product Name: Awesome OS Server Bits
Status:       Subscribed

Unable to find available subscriptions for all your installed products.
[root@dhcp150-207 ~]# 

 no_proxy=*.usersys.redhat.com matched "F21-candlepin.usersys.redhat.com" hostname ,so the auto-attach passed despite the bad proxy details in the system

Scenario 3: 
===========
Verify that --noproxy option honour "*" to find the match

# subscription-manager list --available --noproxy=*foo.com
Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.

# subscription-manager list --available --noproxy=*.test,foo.com
Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.

^^ on both the case *foo.com and *.test,foo.com didnot match the hostname so the subscription-manager command failed -- > test PASSED 

# subscription-manager list --available --noproxy=*redhat.com
+-------------------------------------------+
    Available Subscriptions
+-------------------------------------------+
Subscription Name:   RAM/Cores Package (8GB, 4 cores)
Provides:            RAM Limiting Product
SKU:                 ram-cores-8gb-4cores
Contract:            0
Pool ID:             8ac6a3625c7cdb93015c7cdda5e2037d
Provides Management: No
Available:           5
Suggested:           1
Service Level:       
Service Type:        
Subscription Type:   Standard
Ends:                06/05/2018
System Type:         Physical


--noproxy=*redhat.com matches the hostname, hence test PASSED

Scenario 4:
===========
Verify NO_PROXY (upper case ) variable continues to work 

# NO_PROXY=*.usersys.redhat.com subscription-manager remove --all
2 subscriptions removed at the server.
2 local certificates have been deleted.

Match found hence test PASSED

# NO_PROXY=*foo.com subscription-manager remove --all
Network error, unable to connect to server. Please see /var/log/rhsm/rhsm.log for more information.

NO Match found ,so failed to remove the subscription --> test  PASSED

# NO_PROXY=*foo.com,redhat.com subscription-manager remove --all
0 subscriptions removed at the server.

# NO_PROXY=* subscription-manager unregister
Unregistering from: F21-candlepin.usersys.redhat.com:8443/candlepin
System has been unregistered.

Match found hence test PASSED

Based on the above test observations , Moving the bug to Verified!!

Comment 6 errata-xmlrpc 2017-08-01 19:22:43 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2017:2083


Note You need to log in before you can comment on or make changes to this bug.