Bug 1309611

Summary: virt-who http proxy connection failed with 404 error for hyperv and esx
Product: Red Hat Enterprise Linux 6 Reporter: Eko <hsun>
Component: virt-whoAssignee: Chris Snyder <csnyder>
Status: CLOSED CURRENTRELEASE QA Contact: Eko <hsun>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.8CC: hsun, ktordeur, patalber, redakkan, sgao, wpinheir
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2022-09-05 15:47:58 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Eko 2016-02-18 09:48:07 UTC 
Description of problem:
config virt-who with http proxy, virt-who return 404 error and the connection failed

Version-Release number of selected component (if applicable):
RHEL-6.8-20160212.2 + ESX + Satellite
virt-who-0.16-3.el6.noarch
subscription-manager-1.16.8-2.el6.x86_64

How reproducible:
always

Steps to Reproduce:
1.config http proxy for esx mode with https_proxy=https://xxxxxxxxxx, such as:
https_proxy=https://10.73.2.81:3128
VIRTWHO_ESX=1
VIRTWHO_ESX_OWNER=ACME_Corporation
VIRTWHO_ESX_ENV=Library
VIRTWHO_ESX_SERVER=10.73.2.95
VIRTWHO_ESX_USERNAME=Administrator
VIRTWHO_ESX_PASSWORD=Welcome1!

2.restart virt-who service and check rhsm.log
2016-02-18 17:43:01,580 [virtwho.init INFO] MainProcess(14858):MainThread @virtwho.py:parseOptions:630 - Using reporter_id='hp-z220-09.qe.lab.eng.nay.redhat.com'
2016-02-18 17:43:01,582 [virtwho.init INFO] MainProcess(14858):MainThread @virtwho.py:main:721 - Using configuration "env/cmdline" ("esx" mode)
2016-02-18 17:43:01,602 [virtwho.main DEBUG] MainProcess(14860):MainThread @virtwho.py:run:229 - Starting infinite loop with 60 seconds interval
2016-02-18 17:43:01,719 [virtwho.env_cmdline DEBUG] Esx-1(14868):MainThread @virt.py:run:358 - Virt backend 'env/cmdline' started
2016-02-18 17:43:01,721 [virtwho.env_cmdline DEBUG] Esx-1(14868):MainThread @esx.py:_prepare:55 - Log into ESX
2016-02-18 17:43:01,721 [virtwho.env_cmdline DEBUG] Esx-1(14868):MainThread @esx.py:login:229 - ESX module using proxy: https://10.73.2.81:3128
2016-02-18 17:43:02,056 [virtwho.env_cmdline DEBUG] Esx-1(14868):MainThread @esx.py:_prepare:58 - Creating ESX event filter
2016-02-18 17:43:02,240 [virtwho.env_cmdline DEBUG] Esx-1(14868):MainThread @virt.py:enqueue:351 - Report gathered, putting to queue for sending
2016-02-18 17:43:02,241 [virtwho.env_cmdline DEBUG] Esx-1(14868):MainThread @esx.py:_run:150 - Waiting for ESX changes
2016-02-18 17:43:02,251 [virtwho.main DEBUG] MainProcess(14860):MainThread @subscriptionmanager.py:_connect:121 - Authenticating with certificate: /etc/pki/consumer/cert.pem
2016-02-18 17:43:02,258 [virtwho.main ERROR] MainProcess(14860):MainThread @virtwho.py:send:214 - Error in communication with subscription manager:
Traceback (most recent call last):
  File "/usr/share/virt-who/virtwho.py", line 197, in send
    self._sendGuestAssociation(report)
  File "/usr/share/virt-who/virtwho.py", line 224, in _sendGuestAssociation
    manager.hypervisorCheckIn(report, self.options)
  File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 161, in hypervisorCheckIn
    self._connect(report.config)
  File "/usr/share/virt-who/manager/subscriptionmanager/subscriptionmanager.py", line 129, in _connect
    if not self.connection.ping()['result']:
  File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 876, in ping
    return self.conn.request_get("/status/")
  File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 681, in request_get
    return self._request("GET", method)
  File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 571, in _request
    conn.request(request_type, handler, body=body, headers=headers)
  File "/usr/lib64/python2.6/httplib.py", line 936, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib64/python2.6/httplib.py", line 973, in _send_request
    self.endheaders()
  File "/usr/lib64/python2.6/site-packages/rhsm/connection.py", line 273, in endheaders
    httpslib.HTTPSConnection.endheaders(self)
  File "/usr/lib64/python2.6/httplib.py", line 930, in endheaders
    self._send_output()
  File "/usr/lib64/python2.6/httplib.py", line 802, in _send_output
    self.send(msg)
  File "/usr/lib64/python2.6/httplib.py", line 761, in send
    self.connect()
  File "/usr/lib64/python2.6/site-packages/M2Crypto/httpslib.py", line 203, in connect
    raise socket.error, "Proxy connection failed: %d" % code
error: Proxy connection failed: 404
2016-02-18 17:43:02,260 [virtwho.main DEBUG] MainProcess(14860):MainThread @virtwho.py:send_current_report:166 - Report from "env/cmdline" failed to sent


Actual results:
it seems virt-who can access the vcenter server normally with squid proxy, but the send function return 404 error and connection failed.

Expected results:
virt-who can run normally with squid proxy for esx and hyper-V

Additional info:
Comment 2 Radek Novacek 2016-02-18 11:41:07 UTC 
I believe that the problem is that setting `https_proxy` variable sets it both for ESX and the SubscriptionManager. So while the ESX succeeds via the proxy, virt-who also tries to connect to candlepin via the proxy and that fails with error 404.

Please try to set `no_proxy` to the hostname from /etc/rhsm/rhsm.conf.
Comment 3 Eko 2016-02-19 07:12:58 UTC 
hi Radek, I don't set any values for proxy options in /etc/rhsm/rhsm.conf, all the proxy options' value are None by default, so there is no reason rhsm send the request by proxy? 
proxy_hostname = 
proxy_port =
proxy_user =
proxy_password =

on the other hand, I also add no_proxy = "VIRTHOST_HOSTNAME" option to /etc/rhsm/rhsm.conf for testing, but still "Proxy connection failed: 404" error.
Comment 4 Radek Novacek 2016-02-23 08:34:30 UTC 
Sorry, I did not make myself clear.

Please set

no_proxy=hostname

in the /etc/sysconfig/virt-who, where no_proxy is the hostname of candlepin.
Comment 5 Radek Novacek 2016-02-25 07:59:34 UTC 
The no_proxy option might not work at the moment, please see bug 1311429.
Comment 6 Radek Novacek 2016-03-10 07:51:08 UTC 
I retried this today with:

virt-who-0.16-6.el6.noarch
python-rhsm-1.16.6-1.el6.x86_64

And it is working correctly. When I set hostname of candlepin server to no_proxy variable in /etc/sysconfig/virt-who, the proxy is indeed bypassed and virt-who can connect directly to the candlepin server while using proxy for esx.

Can you please retry it?
Comment 7 Radek Novacek 2016-03-15 11:45:54 UTC 
The no_proxy option is operational, but requires python-rhsm-1.15.1-1 or newer.
Comment 8 Eko 2016-03-16 01:48:21 UTC 
verified, with no_proxy option, virt-who can fetch and send host/guests mapping json normally.