Bug 1271737 - [vdsClient] getStoragedomainsList query hangs for a long time
Summary: [vdsClient] getStoragedomainsList query hangs for a long time
Keywords:
Status: CLOSED INSUFFICIENT_DATA
Alias: None
Product: vdsm
Classification: oVirt
Component: Services
Version: 4.17.7
Hardware: Unspecified
OS: Unspecified
unspecified
low
Target Milestone: ovirt-3.6.1
: 4.17.14
Assignee: Nir Soffer
QA Contact: Aharon Canan
URL:
Whiteboard: storage
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-10-14 14:51 UTC by Ori Gofen
Modified: 2016-05-26 01:50 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-10-28 00:55:44 UTC
oVirt Team: Storage
Embargoed:
amureini: ovirt-3.6.z?
amureini: ovirt-4.0.0?
ogofen: planning_ack?
ogofen: devel_ack?
ogofen: testing_ack?


Attachments (Terms of Use)

Description Ori Gofen 2015-10-14 14:51:27 UTC
Description of problem:
when executing "vdsClient -s 0 getStoragedomainsList" from a 3.6 hypervisor, the operation seem to hang/get stuck before it returns with an answer, this has nothing to do with the amount of domains on the pool.
In order to emphasize the problem I have executed 10 times this command on a 3.5 and 3.6 hypervisors and compared the average execution time.

3.5 hypervisor:
[root@green-vdsc ~]# time vdsClient -s 0 getStorageDomainsList
9c95978b-0f3a-4887-9c15-61b36490ae28
b9feec08-20d8-45d6-bee5-f58a7357798c
294efd08-19bf-4bda-8c78-79d720e634d1


real    0m0.388s
user    0m0.110s
sys     0m0.015s

3.6 hypervisor:
[root@purple-vds1 ~]# time vdsClient -s 0 getStorageDomainsList
f8a1ae58-e886-4d3a-b0f4-06d32e61a339


real    2m0.655s
user    0m0.165s
sys     0m0.027s

The average 'real' execution time differences are pretty big almost 3 minutes on 3.6 hypervisor in comparison to less than half a SECOND on a 3.5 hypervisor.


Version-Release number of selected component (if applicable):
vdsm-infra-4.17.7-1.el7ev.noarch
vdsm-4.17.7-1.el7ev.noarch
vdsm-xmlrpc-4.17.7-1.el7ev.noarch
vdsm-cli-4.17.7-1.el7ev.noarch
vdsm-yajsonrpc-4.17.7-1.el7ev.noarch
vdsm-jsonrpc-4.17.7-1.el7ev.noarch
vdsm-python-4.17.7-1.el7ev.noarch

How reproducible:
100%

Steps to Reproduce:
1.as descripted above

Comment 1 Allon Mureinik 2015-10-15 08:31:05 UTC
Does this happen only in vdsClient, or in engine flows as well?

Comment 2 Ori Gofen 2015-10-18 11:59:22 UTC
I guess Every action that calls "HSMGetStorageDomainsListVDSCommand.java" will suffer from a longer delay than it used too.

Comment 3 Yaniv Bronhaim 2015-10-18 14:24:34 UTC
signing as a regression and I would call it a blocker. Probably this the reason for the long vdsm startup hanging as well

Comment 4 Nir Soffer 2015-10-18 15:09:07 UTC
Are you testing the same storage from the same hypervisor in both cases?

On the 3.5 version, I see 3 domains. In the 3.6, 1 domain (with different id)

Please repeat the test using the same storage and the same hypervisor:

1. Install 3.5 vdsm
2. Create couple of domains
3. time the vdsClient operation
4. Upgrade to 3.6 vdsm
5. time the vdsClient operation

If you get same results, try in the opposite order:

1. Install 3.6 vdsm
2. Create couple of domains
3. time the vdsClient operation
4. Downgrade to 3.5 vdsm
5. time the vdsClient operation

Removing the regression until we have results for this test.

Please keep vdsm logs for each test - remove vdsm log before installing
new vdsm version and keep the vdsm logs from each test in another
location for attaching to the bug later.

Comment 5 Allon Mureinik 2015-10-18 15:14:02 UTC
(In reply to Ori Gofen from comment #2)
> I guess Every action that calls "HSMGetStorageDomainsListVDSCommand.java"
> will suffer from a longer delay than it used too.
The word "guess" has no place here. I was asking if you actually observed this while using engine (which may mean it's an issue worth examining), or whether it's a problem in vdsClient's code (which isn't supported anyway).

Comment 6 Red Hat Bugzilla Rules Engine 2015-10-19 10:57:35 UTC
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.

Comment 7 Nir Soffer 2015-10-28 00:55:44 UTC
Closing as we did not get any response. Please reopen when you have more
data.


Note You need to log in before you can comment on or make changes to this bug.