Bug 1743589

Summary: virt-who run fails with TypeError: argument of type 'NoneType' is not iterable
Product: Red Hat Enterprise Linux 7 Reporter: Michal Dekan <mdekan>
Component: virt-whoAssignee: candlepin-bugs
Status: CLOSED CURRENTRELEASE QA Contact: Eko <hsun>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.6CC: csnyder, kuhuang, redakkan, wpoteat, yuefliu
Target Milestone: rcKeywords: Triaged
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of:
: 1882382 (view as bug list) Environment:
Last Closed: 2023-12-07 08:51:48 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1882382    

Description Michal Dekan 2019-08-20 09:17:46 UTC
Description of problem:

virt-who-0.24.7-1.el7.noarch running on satellite 6.4.3 is failing to connect to vcenter:

2019-08-17 15:34:56,516 [virtwho.main DEBUG] MainProcess(25436):Thread-2 @esx.py:_prepare:130 - Log into ESX
2019-08-17 15:34:56,670 [virtwho.main ERROR] MainProcess(25436):Thread-2 @virt.py:run:421 - Thread 'virt-who-config-5' fails with exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/virtwho/virt/virt.py", line 412, in run
    self._run()
  File "/usr/lib/python2.7/site-packages/virtwho/virt/esx/esx.py", line 143, in _run
    self._prepare()
  File "/usr/lib/python2.7/site-packages/virtwho/virt/esx/esx.py", line 131, in _prepare
    self.login()
  File "/usr/lib/python2.7/site-packages/virtwho/virt/esx/esx.py", line 349, in login
    self.sc = self.client.service.RetrieveServiceContent(_this=self.moRef)
  File "/usr/lib/python2.7/site-packages/suds/client.py", line 542, in __call__
    return client.invoke(args, kwargs)
  File "/usr/lib/python2.7/site-packages/suds/client.py", line 602, in invoke
    result = self.send(soapenv)
  File "/usr/lib/python2.7/site-packages/suds/client.py", line 641, in send
    reply = transport.send(request)
  File "/usr/lib/python2.7/site-packages/virtwho/virt/esx/esx.py", line 102, in send
    if 'application/soap+xml' not in ct and 'text/xml' not in ct:
TypeError: argument of type 'NoneType' is not iterable

Version-Release number of selected component (if applicable):

virt-who-0.24.7-1.el7.noarch
satellite-6.4.3-2.el7sat.noarch
redhat-release-server-7.6-4.el7.x86_64

How reproducible:


Steps to Reproduce:
1.
2.
3.

Actual results:

TypeError: argument of type 'NoneType' is not iterable

Expected results:

Real error message instead of TypeError, suggested this to the customer, to get real error:

/usr/lib/python2.7/site-packages/virtwho/virt/esx/esx.py

- ct = resp.headers.get('content-type')
+ ct = resp.headers.get('content-type', '')

Comment 3 Michal Dekan 2019-08-20 14:42:15 UTC
virt-who -od with code modification mentioned in the description:

2019-08-20 21:33:34,899 [virtwho.rhsm_log DEBUG] MainProcess(13380):MainThread @config.py:init_config:1499 - [virt-who-config-5]: Value for "simplified_vim" not set, using default: True
2019-08-20 21:33:34,899 [virtwho.rhsm_log WARNING] MainProcess(13380):MainThread @config.py:init_config:1499 - [virt-who-config-5]: Ignoring unknown configuration option "env"
2019-08-20 21:33:34,899 [virtwho.rhsm_log DEBUG] MainProcess(13380):MainThread @config.py:init_config:1499 - [global]: Value for "interval" not set, using default: 3600
2019-08-20 21:33:34,899 [virtwho.rhsm_log DEBUG] MainProcess(13380):MainThread @config.py:init_config:1499 - [global]: Value for "log_per_config" not set, using default: False
2019-08-20 21:33:34,899 [virtwho.rhsm_log DEBUG] MainProcess(13380):MainThread @config.py:init_config:1499 - [global]: Value for "print_" not set, using default: False
2019-08-20 21:33:34,899 [virtwho.rhsm_log DEBUG] MainProcess(13380):MainThread @config.py:init_config:1499 - [global]: Value for "reporter_id" not set, using default: satellite-1393b5edfa9d10bf51f71e9600000016
2019-08-20 21:33:34,900 [virtwho.rhsm_log DEBUG] MainProcess(13380):MainThread @config.py:init_config:1499 - [global]: Value for "log_dir" not set, using default: /var/log/rhsm
2019-08-20 21:33:34,900 [virtwho.rhsm_log DEBUG] MainProcess(13380):MainThread @config.py:init_config:1499 - [global]: Value for "configs" not set, using default: []
2019-08-20 21:33:34,900 [virtwho.rhsm_log DEBUG] MainProcess(13380):MainThread @config.py:init_config:1499 - [global]: Value for "log_file" not set, using default: rhsm.log
2019-08-20 21:33:34,900 [virtwho.rhsm_log INFO] MainProcess(13380):MainThread @executor.py:__init__:54 - Using config named 'virt-who-config-5'
2019-08-20 21:33:34,901 [virtwho.rhsm_log INFO] MainProcess(13380):MainThread @main.py:main:162 - Using configuration "virt-who-config-5" ("esx" mode)
2019-08-20 21:33:34,901 [virtwho.rhsm_log INFO] MainProcess(13380):MainThread @main.py:main:164 - Using reporter_id='satellite-1393b5edfa9d10bf51f71e9600000016'
2019-08-20 21:33:34,915 [rhsm.https DEBUG] MainProcess(13380):MainThread @https.py:<module>:56 - Using standard libs to provide httplib and ssl
2019-08-20 21:33:34,922 [virtwho.main DEBUG] MainProcess(13380):Thread-2 @virt.py:run:407 - Thread 'virt-who-config-5' started
2019-08-20 21:33:34,923 [virtwho.main DEBUG] MainProcess(13380):Thread-2 @esx.py:_prepare:130 - Log into ESX
2019-08-20 21:33:35,013 [virtwho.main ERROR] MainProcess(13380):Thread-2 @virt.py:run:416 - Thread 'virt-who-config-5' fails with error: 403 Client Error: Forbidden
2019-08-20 21:33:35,013 [virtwho.main INFO] MainProcess(13380):Thread-2 @virt.py:_send_data:969 - Report for config "virt-who-config-5" gathered, placing in datastore
2019-08-20 21:33:35,014 [virtwho.main DEBUG] MainProcess(13380):Thread-2 @virt.py:run:435 - Thread 'virt-who-config-5' stopped after running once
2019-08-20 21:33:35,924 [virtwho.destination_6931689818818942579 DEBUG] MainProcess(13380):Thread-3 @virt.py:run:407 - Thread 'destination_6931689818818942579' started
2019-08-20 21:33:35,924 [virtwho.destination_6931689818818942579 DEBUG] MainProcess(13380):Thread-3 @virt.py:_send_data:649 - ErrorReport received for source: virt-who-config-5
2019-08-20 21:33:35,925 [virtwho.destination_6931689818818942579 DEBUG] MainProcess(13380):Thread-3 @virt.py:_send_data:744 - At least one report for each connected source has been sent. Terminating.
2019-08-20 21:33:35,925 [virtwho.destination_6931689818818942579 DEBUG] MainProcess(13380):Thread-3 @virt.py:run:426 - Thread 'destination_6931689818818942579' terminated
2019-08-20 21:33:36,926 [virtwho.main DEBUG] MainProcess(13380):MainThread @__main__.py:main:25 - virt-who terminated
2019-08-20 21:33:36,926 [virtwho.main DEBUG] MainProcess(13380):MainThread @executor.py:terminate:225 - virt-who is shutting down

Comment 10 William Poteat 2019-11-11 20:50:57 UTC
Michal - Can you give us more information as to how you were able to get the 403 error to be returned [a bogus configuration, perhaps?] That seems to be the missing piece for validation on our end.

Comment 11 Michal Dekan 2019-11-13 13:22:56 UTC
William - When I checked attach case, it seems that they were getting 403 when accessing vcernter hostname using curl:

 curl -k -v https://<vcenter_fqdn>/mob

    <title>
      Web Filter Violation
    </title>

It looks like corporate proxy was intercepting SSL connection when virt-who was trying to connect to virt-who.

Comment 12 Michal Dekan 2019-11-13 13:25:49 UTC
EDIT: My English is falling apart: "It looks like corporate proxy was intercepting SSL connection when virt-who was trying to connect to virt-who." Should be It looks like corporate proxy was intercepting SSL connection when virt-who was trying to connect to vcenter.....

--Michal