Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.

Bug 1596952

Summary: vdsm error
Product: [oVirt] ovirt-hosted-engine-ha Reporter: Pascal DeMilly <pascal>
Component: AgentAssignee: Milan Zamazal <mzamazal>
Status: CLOSED CURRENTRELEASE QA Contact: meital avital <mavital>
Severity: high Docs Contact:
Priority: unspecified    
Version: ---CC: bugs, pagranat, rbarry
Target Milestone: ovirt-4.3.1Flags: rbarry: ovirt-4.3?
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: ve
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2019-03-01 10:24:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: SLA RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
extracts of mom.log vdsm.log and supervdsm.log none

Description Pascal DeMilly 2018-06-30 17:17:03 UTC
Created attachment 1455685 [details]
extracts of mom.log vdsm.log and supervdsm.log

Description of problem:

After starting a pool of Windows x32 VMs, the host system becomes unresponsive in the ovirt console. Checking the logs I can see errors in vdsm.log and mom.log

Version-Release number of selected component (if applicable):

The actual version for vdsm is actually 4.20.32

How reproducible:


Steps to Reproduce:
1. create a pool of 50 VM and start 50 VM
2.  after a few minutes hosts becomes unresponsive
3.

Actual results:



Expected results:

Host should never become unresponsive.

Additional info:

ERROR reported by vdsm:

File: "/usr/lib64/python2.7/threading.py", line 785, in __bootstrap
  self.__bootstrap_inner()
File: "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner
  self.run()
File: "/usr/lib64/python2.7/threading.py", line 765, in run
  self.__target(*self.__args, **self.__kwargs)
File: "/usr/lib/python2.7/site-packages/vdsm/common/concurrent.py", line 194, in run
  ret = func(*args, **kwargs)
File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 301, in _run
  self._execute_task()
File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 315, in _execute_task
  task()
File: "/usr/lib/python2.7/site-packages/vdsm/executor.py", line 391, in __call__
  self._callable()
File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 523, in __call__
  self._handler(self._ctx, self._req)
File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 566, in _serveRequest
  response = self._handle_request(req, ctx)
File: "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 606, in _handle_request
  res = method(**params)
File: "/usr/lib/python2.7/site-packages/vdsm/rpc/Bridge.py", line 197, in _dynamicMethod
  result = fn(*methodArgs)
File: "<string>", line 2, in getStats
File: "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method
  ret = func(*args, **kwargs)
File: "/usr/lib/python2.7/site-packages/vdsm/API.py", line 1402, in getStats
  sampling.host_samples.stats())}
File: "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 77, in get_stats
  ret['haStats'] = _getHaInfo()
File: "/usr/lib/python2.7/site-packages/vdsm/host/api.py", line 182, in _getHaInfo
  stats = instance.get_all_stats()
File: "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/client/client.py", line 94, in get_all_stats
  stats = broker.get_stats_from_storage()
File: "/usr/lib/python2.7/site-packages/ovirt_hosted_engine_ha/lib/brokerlink.py", line 135, in get_stats_from_storage
  result = self._proxy.get_stats()
File: "/usr/lib64/python2.7/xmlrpclib.py", line 1233, in __call__
  return self.__send(self.__name, args)
File: "/usr/lib64/python2.7/xmlrpclib.py", line 1591, in __request
  verbose=self.__verbose
File: "/usr/lib64/python2.7/xmlrpclib.py", line 1273, in request
  return self.single_request(host, handler, request_body, verbose)
File: "/usr/lib64/python2.7/xmlrpclib.py", line 1303, in single_request
  response = h.getresponse(buffering=True)
File: "/usr/lib64/python2.7/httplib.py", line 1113, in getresponse
  response.begin()
File: "/usr/lib64/python2.7/httplib.py", line 444, in begin
  version, status, reason = self._read_status()
File: "/usr/lib64/python2.7/httplib.py", line 400, in _read_status
  line = self.fp.readline(_MAXLINE + 1)
File: "/usr/lib64/python2.7/socket.py", line 476, in readline
  data = self._sock.recv(self._rbufsize) (executor:363)
2018-06-30 09:10:13,069-0700 INFO  (vmrecovery) [vdsm.api] START getConnectedStoragePoolsList(options=None) from=internal, task_id=484268e1-8ad4-476a-a30f-5f1ccf149c95 (api:46)

Comment 1 Martin Perina 2019-01-05 20:05:29 UTC
Moving to ovirt-hosted-engine-ha to align with the patch

Comment 2 Pedut 2019-01-23 14:54:19 UTC
Verified on 4.3.0-0.8.master.20190122102235.git7a1ef10.el7.

Comment 4 Sandro Bonazzola 2019-02-26 08:04:26 UTC
Fix included in 2.3.1 shipped in ovirt-4.3.0

Comment 5 Raz Tamir 2019-02-27 16:30:11 UTC
QE verification bot: the bug was verified upstream

Comment 6 Sandro Bonazzola 2019-03-01 10:24:39 UTC
Verified in 4.3.1 scope, moving back to 4.3.1.

This bugzilla is included in oVirt 4.3.1 release, published on February 28th 2019.

Since the problem described in this bug report should be
resolved in oVirt 4.3.1 release, it has been closed with a resolution of CURRENT RELEASE.

If the solution does not work for you, please open a new bug report.