Bug 1404563

Summary: [scale] Register RHVH to RHVM failed when too many LUNs (100) are connected to the host
Product: Red Hat Enterprise Virtualization Manager Reporter: cshao <cshao>
Component: vdsmAssignee: Nir Soffer <nsoffer>
Status: CLOSED WORKSFORME QA Contact: Raz Tamir <ratamir>
Severity: low Docs Contact:
Priority: unspecified    
Version: 4.0.6CC: bazulay, cshao, lsurette, srevivo, tnisan, ycui, ykaul
Target Milestone: ovirt-4.2.0Keywords: Performance
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-05-08 07:28:26 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Attachments:
Description Flags
/var/log/*.*; /tmp/log; sosreport; ks
none
register failed
none
all log-part1
none
all log-part2 none

Description cshao 2016-12-14 06:43:28 UTC
Created attachment 1231449 [details]
/var/log/*.*; /tmp/log; sosreport; ks

Description of problem:
Install RHVH on machine which have more than 100 luns. Then register to RHVM.
The registration will fail, the status of RHVH host always show as "connecting -> initializing" in RHVM web UI.

VDSM cshao_100luns command failed: Heartbeat exceeded
VDSM cshao_100luns command failed: Vds timeout occured
Host cshao_100luns is initializing. Message: Recovering from crash or Initializing


Version-Release number of selected component (if applicable):
rhvh-4.0-0.20161012.0+1
RHVH-4.0-20161018.0-RHVH-x86_64-dvd1.iso
imgbased-0.8.5-0.1.el7ev.noarch
vdsm-4.18.13-1.el7ev.x86_64
RHVM 4.0 (rhevm-4.0.6.3-0.1.el7ev.noarch)

How reproducible:
100%

Steps to Reproduce:
1. Install RHVH on machine which have more than 100 luns. 
2. Register RHVH to RHVM.
3. Check the host status in RHVM web UI.

Actual results:
The registration will fail, the status of RHVH host always show as "connecting -> initializing" in RHVM web UI.

Expected results:
Register RHVH to RHVM can successful when too many LUNs are connected to the host.

Additional info:

Comment 1 cshao 2016-12-14 06:48:09 UTC
Created attachment 1231451 [details]
register failed

Comment 2 Dan Kenigsberg 2016-12-14 07:17:22 UTC
I suspect that would be the same in non-node installations

storageRefresh::DEBUG::2016-12-13 15:28:16,910::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/taskset --cpu-list 0-3 /usr/bin/sudo -n /usr/sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ '\''a|/dev/mapper/36005076300810b3e0000000000000022|/dev/mapper/36005076300810b3e0000000000000023|/dev/mapper/36005076300810b3e0000000000000024|/dev/mapper/36005076300810b3e000000000000002d|/dev/mapper/36005076300810b3e000000000000002e|/dev/mapper/36005076300810b3e000000000000002f|/dev/mapper/36005076300810b3e0000000000000030|/dev/mapper/36005076300810b3e0000000000000031|/dev/mapper/36005076300810b3e0000000000000032|/dev/mapper/36005076300810b3e0000000000000033|/dev/mapper/36005076300810b3e0000000000000034|/dev/mapper/36005076300810b3e0000000000000035|/dev/mapper/36005076300810b3e0000000000000036|/dev/mapper/36005076300810b3e0000000000000037|/dev/mapper/36005076300810b3e0000000000000038|/dev/mapper/36005076300810b3e0000000000000039|/dev/mapper/36005076300810b3e000000000000003a|/dev/mapper/36005076300810b3e000000000000003b|/dev/mapper/36005076300810b3e000000000000003c|/dev/mapper/36005076300810b3e000000000000003d|/dev/mapper/36005076300810b3e000000000000003e|/dev/mapper/36005076300810b3e000000000000003f|/dev/mapper/36005076300810b3e0000000000000040|/dev/mapper/36005076300810b3e0000000000000041|/dev/mapper/36005076300810b3e0000000000000042|/dev/mapper/36005076300810b3e0000000000000043|/dev/mapper/36005076300810b3e0000000000000044|/dev/mapper/36005076300810b3e0000000000000045|/dev/mapper/36005076300810b3e0000000000000046|/dev/mapper/36005076300810b3e0000000000000047|/dev/mapper/36005076300810b3e0000000000000048|/dev/mapper/36005076300810b3e0000000000000049|/dev/mapper/36005076300810b3e000000000000004a|/dev/mapper/36005076300810b3e000000000000004b|/dev/mapper/36005076300810b3e000000000000004c|/dev/mapper/36005076300810b3e000000000000004d|/dev/mapper/36005076300810b3e000000000000004e|/dev/mapper/36005076300810b3e000000000000004f|/dev/mapper/36005076300810b3e0000000000000050|/dev/mapper/36005076300810b3e0000000000000051|/dev/mapper/36005076300810b3e0000000000000052|/dev/mapper/36005076300810b3e0000000000000053|/dev/mapper/36005076300810b3e0000000000000054|/dev/mapper/36005076300810b3e0000000000000055|/dev/mapper/36005076300810b3e0000000000000056|/dev/mapper/36005076300810b3e0000000000000057|/dev/mapper/36005076300810b3e0000000000000058|/dev/mapper/36005076300810b3e0000000000000059|/dev/mapper/36005076300810b3e000000000000005a|/dev/mapper/36005076300810b3e000000000000005b|/dev/mapper/36005076300810b3e000000000000005c|/dev/mapper/36005076300810b3e000000000000005d|/dev/mapper/36005076300810b3e000000000000005e|/dev/mapper/36005076300810b3e000000000000005f|/dev/mapper/36005076300810b3e0000000000000060|/dev/mapper/36005076300810b3e0000000000000061|/dev/mapper/36005076300810b3e0000000000000062|/dev/mapper/36005076300810b3e0000000000000063|/dev/mapper/36005076300810b3e0000000000000064|/dev/mapper/36005076300810b3e0000000000000065|/dev/mapper/36005076300810b3e0000000000000066|/dev/mapper/36005076300810b3e0000000000000067|/dev/mapper/36005076300810b3e0000000000000068|/dev/mapper/36005076300810b3e0000000000000069|/dev/mapper/36005076300810b3e000000000000006a|/dev/mapper/36005076300810b3e000000000000006b|/dev/mapper/36005076300810b3e000000000000006c|/dev/mapper/36005076300810b3e000000000000006d|/dev/mapper/36005076300810b3e000000000000006e|/dev/mapper/36005076300810b3e000000000000006f|/dev/mapper/36005076300810b3e0000000000000070|/dev/mapper/36005076300810b3e0000000000000071|/dev/mapper/36005076300810b3e0000000000000072|/dev/mapper/36005076300810b3e0000000000000073|/dev/mapper/36005076300810b3e0000000000000074|/dev/mapper/36005076300810b3e0000000000000075|/dev/mapper/36005076300810b3e0000000000000076|/dev/mapper/36005076300810b3e0000000000000077|/dev/mapper/36005076300810b3e0000000000000078|/dev/mapper/36005076300810b3e0000000000000079|/dev/mapper/36005076300810b3e000000000000007a|/dev/mapper/36005076300810b3e000000000000007b|/dev/mapper/36005076300810b3e000000000000007c|/dev/mapper/36005076300810b3e000000000000007d|/dev/mapper/36005076300810b3e000000000000007e|/dev/mapper/36005076300810b3e000000000000007f|/dev/mapper/36005076300810b3e0000000000000080|/dev/mapper/36005076300810b3e0000000000000081|/dev/mapper/36005076300810b3e0000000000000082|/dev/mapper/36005076300810b3e0000000000000083|/dev/mapper/36005076300810b3e0000000000000084|/dev/mapper/36005076300810b3e0000000000000085|/dev/mapper/36005076300810b3e0000000000000086|/dev/mapper/36005076300810b3e0000000000000087|/dev/mapper/36005076300810b3e0000000000000088|/dev/mapper/36005076300810b3e0000000000000089|/dev/mapper/36005076300810b3e000000000000008a|/dev/mapper/36005076300810b3e000000000000008b|/dev/mapper/36005076300810b3e000000000000008c|/dev/mapper/36005076300810b3e000000000000008d|/dev/mapper/36005076300810b3e000000000000008e|/dev/mapper/36005076300810b3e000000000000008f|/dev/mapper/36005076300810b3e0000000000000090|/dev/mapper/36005076300810b3e0000000000000091|/dev/mapper/36005076300810b3e0000000000000092|/dev/mapper/36005076300810b3e0000000000000093|/dev/mapper/36005076300810b3e0000000000000094|/dev/mapper/36005076300810b3e0000000000000095|/dev/mapper/36005076300810b3e0000000000000096|/dev/mapper/36005076300810b3e0000000000000097|/dev/mapper/36005076300810b3e0000000000000098|/dev/mapper/36005076300810b3e0000000000000099|/dev/mapper/36005076300810b3e000000000000009a|/dev/mapper/36005076300810b3e000000000000009b|/dev/mapper/36005076300810b3e000000000000009c|/dev/mapper/36005076300810b3e000000000000009d|/dev/mapper/36005076300810b3e000000000000009e|/dev/mapper/36005076300810b3e000000000000009f|/dev/mapper/36005076300810b3e00000000000000a0|/dev/mapper/36005076300810b3e00000000000000a1|/dev/mapper/36005076300810b3e00000000000000a2|/dev/mapper/36005076300810b3e00000000000000a3|/dev/mapper/36005076300810b3e00000000000000a4|/dev/mapper/36005076300810b3e00000000000000a5|/dev/mapper/36005076300810b3e00000000000000a6|/dev/mapper/36005076300810b3e00000000000000a7|/dev/mapper/36005076300810b3e00000000000000a8|/dev/mapper/36005076300810b3e00000000000000a9|/dev/mapper/36005076300810b3e00000000000000aa|/dev/mapper/36005076300810b3e00000000000000ab|/dev/mapper/36005076300810b3e00000000000000ac|/dev/mapper/36005076300810b3e00000000000000ad|/dev/mapper/36005076300810b3e00000000000000ae|/dev/mapper/36005076300810b3e00000000000000af|/dev/mapper/36005076300810b3e00000000000000b0|/dev/mapper/36005076300810b3e00000000000000b1|/dev/mapper/36005076300810b3e00000000000000b2|/dev/mapper/36005076300810b3e00000000000000b3|/dev/mapper/36005076300810b3e00000000000000b4|/dev/mapper/36005076300810b3e00000000000000b5|/dev/mapper/36005076300810b3e00000000000000b6|/dev/mapper/36005076300810b3e00000000000000b7|/dev/mapper/36005076300810b3e00000000000000b8|/dev/mapper/36005076300810b3e00000000000000b9|/dev/mapper/36005076300810b3e00000000000000ba|/dev/mapper/36005076300810b3e00000000000000bb|/dev/mapper/36005076300810b3e00000000000000bc|/dev/mapper/36005076300810b3e00000000000000bd|/dev/mapper/36005076300810b3e00000000000000be|/dev/mapper/36005076300810b3e00000000000000bf|/dev/mapper/36005076300810b3e00000000000000c0|/dev/mapper/36005076300810b3e00000000000000c1|/dev/mapper/36005076300810b3e00000000000000c2|/dev/mapper/36005076300810b3e00000000000000c3|/dev/mapper/36005076300810b3e00000000000000c4|/dev/mapper/36005076300810b3e00000000000000c5|/dev/mapper/36005076300810b3e00000000000000c6|/dev/mapper/36005076300810b3e00000000000000c7|/dev/mapper/36005076300810b3e00000000000000c8|/dev/mapper/36005076300810b3e00000000000000c9|/dev/mapper/36005076300810b3e00000000000000ca|/dev/mapper/36005076300810b3e00000000000000cb|/dev/mapper/36005076300810b3e00000000000000cc|/dev/mapper/36005076300810b3e00000000000000cd|/dev/mapper/36005076300810b3e00000000000000ce|/dev/mapper/36005076300810b3e00000000000000cf|/dev/mapper/36005076300810b3e00000000000000d0|/dev/mapper/36005076300810b3e00000000000000d1|/dev/mapper/36005076300810b3e00000000000000d2|/dev/mapper/36005076300810b3e00000000000000d3|/dev/mapper/36005076300810b3e00000000000000d4|/dev/mapper/36005076300810b3e00000000000000d5|/dev/mapper/36005076300810b3e00000000000000d6|/dev/mapper/36005076300810b3e00000000000000d7|/dev/mapper/36005076300810b3e00000000000000d8|/dev/mapper/36005076300810b3e00000000000000d9|/dev/mapper/36005076300810b3e00000000000000da|/dev/mapper/36005076300810b3e00000000000000db|/dev/mapper/36005076300810b3e00000000000000dc|/dev/mapper/36005076300810b3e00000000000000dd|/dev/mapper/36005076300810b3e00000000000000de|/dev/mapper/36005076300810b3e00000000000000df|/dev/mapper/36005076300810b3e00000000000000e0|/dev/mapper/36005076300810b3e00000000000000e1|/dev/mapper/36005076300810b3e00000000000000e2|/dev/mapper/36005076300810b3e00000000000000e3|/dev/mapper/36005076300810b3e00000000000000e4|/dev/mapper/36005076300810b3e00000000000000e5|/dev/mapper/36005076300810b3e00000000000000e6|/dev/mapper/36005076300810b3e00000000000000e7|/dev/mapper/36005076300810b3e00000000000000e8|/dev/mapper/36005076300810b3e00000000000000e9|/dev/mapper/36005076300810b3e00000000000000ea|/dev/mapper/36005076300810b3e00000000000000eb|/dev/mapper/36005076300810b3e00000000000000ec|/dev/mapper/36005076300810b3e00000000000000ed|/dev/mapper/36005076300810b3e00000000000000ee|/dev/mapper/36005076300810b3e00000000000000ef|/dev/mapper/36005076300810b3e00000000000000f0|/dev/mapper/36005076300810b3e00000000000000f1|/dev/mapper/36005076300810b3e00000000000000f2|/dev/mapper/36005076300810b3e00000000000000f3|/dev/mapper/36005076300810b3e00000000000000f4|/dev/mapper/36005076300810b3e00000000000000f5|/dev/mapper/36005076300810b3e00000000000000f6|/dev/mapper/36005076300810b3e00000000000000f7|/dev/mapper/36005076300810b3e00000000000000f8|/dev/mapper/36005076300810b3e00000000000000f9|/dev/mapper/36005076300810b3e00000000000000fa|/dev/mapper/36005076300810b3e00000000000000fb|/dev/mapper/36005076300810b3e00000000000000fc|/dev/mapper/36005076300810b3e00000000000000fd|/dev/mapper/36005076300810b3e00000000000000fe|/dev/mapper/36005076300810b3e00000000000000ff|/dev/mapper/36005076300810b3e0000000000000100|/dev/mapper/36005076300810b3e0000000000000101|/dev/mapper/36005076300810b3e0000000000000102|/dev/mapper/36005076300810b3e0000000000000103|/dev/mapper/36005076300810b3e0000000000000104|/dev/mapper/36005076300810b3e0000000000000105|/dev/mapper/36005076300810b3e0000000000000106|/dev/mapper/36005076300810b3e0000000000000107|/dev/mapper/36005076300810b3e0000000000000108|/dev/mapper/36005076300810b3e0000000000000109|/dev/mapper/36005076300810b3e000000000000010a|/dev/mapper/36005076300810b3e000000000000010b|/dev/mapper/36005076300810b3e000000000000010c|/dev/mapper/36005076300810b3e000000000000010d|/dev/mapper/36005076300810b3e000000000000010e|/dev/mapper/36005076300810b3e000000000000010f|/dev/mapper/36005076300810b3e0000000000000110|/dev/mapper/36005076300810b3e0000000000000111|/dev/mapper/36005076300810b3e0000000000000112|/dev/mapper/36005076300810b3e0000000000000113|/dev/mapper/36005076300810b3e0000000000000114|/dev/mapper/36005076300810b3e0000000000000115|/dev/mapper/36005076300810b3e0000000000000116|/dev/mapper/36005076300810b3e0000000000000117|/dev/mapper/36005076300810b3e0000000000000118|/dev/mapper/36005076300810b3e0000000000000119|/dev/mapper/36005076300810b3e000000000000011a|/dev/mapper/36005076300810b3e000000000000011b|/dev/mapper/36005076300810b3e000000000000011c|/dev/mapper/36005076300810b3e000000000000011d|/dev/mapper/36005076300810b3e000000000000011e|/dev/mapper/36005076300810b3e000000000000011f|/dev/mapper/36005076300810b3e0000000000000120|/dev/mapper/36005076300810b3e0000000000000121|/dev/mapper/36005076300810b3e0000000000000122|/dev/mapper/36005076300810b3e0000000000000123|/dev/mapper/36005076300810b3e0000000000000124|/dev/mapper/36005076300810b3e0000000000000125|/dev/mapper/36005076300810b3e0000000000000126|/dev/mapper/36005076300810b3e0000000000000127|/dev/mapper/36005076300810b3e0000000000000128|/dev/mapper/36005076300810b3e0000000000000129|'\'', '\''r|.*|'\'' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1  use_lvmetad=0 }  backup {  retain_min = 50  retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None)
storageRefresh::DEBUG::2016-12-13 15:28:21,428::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = "  WARNING: lvmetad is running but disabled. Restart lvmetad before enabling it!\n  Couldn't find device with uuid w7ghZS-G3D7-wpXw-ktzr-AD6U-drDN-2PUEsa.\n"; <rc> = 0
storageRefresh::ERROR::2016-12-13 15:28:21,431::lvm::335::Storage.LVM::(_reloadpvs) Missing pv: w7ghZS-G3D7-wpXw-ktzr-AD6U-drDN-2PUEsa in vg: rhvh_dell-per510-01
storageRefresh::DEBUG::2016-12-13 15:28:21,432::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/taskset --cpu-list 0-3 /usr/bin/sudo -n /usr/sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 filter = [ '\''a|/dev/mapper/36005076300810b3e0000000000000022|/dev/mapper/36005076300810b3e0000000000000023|/dev/mapper/36005076300810b3e0000000000000024|/dev/mapper/36005076300810b3e000000000000002d|/dev/mapper/36005076300810b3e000000000000002e|/dev/mapper/36005076300810b3e000000000000002f|/dev/mapper/36005076300810b3e0000000000000030|/dev/mapper/36005076300810b3e0000000000000031|/dev/mapper/36005076300810b3e0000000000000032|/dev/mapper/36005076300810b3e0000000000000033|/dev/mapper/36005076300810b3e0000000000000034|/dev/mapper/36005076300810b3e0000000000000035|/dev/mapper/36005076300810b3e0000000000000036|/dev/mapper/36005076300810b3e0000000000000037|/dev/mapper/36005076300810b3e0000000000000038|/dev/mapper/36005076300810b3e0000000000000039|/dev/mapper/36005076300810b3e000000000000003a|/dev/mapper/36005076300810b3e000000000000003b|/dev/mapper/36005076300810b3e000000000000003c|/dev/mapper/36005076300810b3e000000000000003d|/dev/mapper/36005076300810b3e000000000000003e|/dev/mapper/36005076300810b3e000000000000003f|/dev/mapper/36005076300810b3e0000000000000040|/dev/mapper/36005076300810b3e0000000000000041|/dev/mapper/36005076300810b3e0000000000000042|/dev/mapper/36005076300810b3e0000000000000043|/dev/mapper/36005076300810b3e0000000000000044|/dev/mapper/36005076300810b3e0000000000000045|/dev/mapper/36005076300810b3e0000000000000046|/dev/mapper/36005076300810b3e0000000000000047|/dev/mapper/36005076300810b3e0000000000000048|/dev/mapper/36005076300810b3e0000000000000049|/dev/mapper/36005076300810b3e000000000000004a|/dev/mapper/36005076300810b3e000000000000004b|/dev/mapper/36005076300810b3e000000000000004c|/dev/mapper/36005076300810b3e000000000000004d|/dev/mapper/36005076300810b3e000000000000004e|/dev/mapper/36005076300810b3e000000000000004f|/dev/mapper/36005076300810b3e0000000000000050|/dev/mapper/36005076300810b3e0000000000000051|/dev/mapper/36005076300810b3e0000000000000052|/dev/mapper/36005076300810b3e0000000000000053|/dev/mapper/36005076300810b3e0000000000000054|/dev/mapper/36005076300810b3e0000000000000055|/dev/mapper/36005076300810b3e0000000000000056|/dev/mapper/36005076300810b3e0000000000000057|/dev/mapper/36005076300810b3e0000000000000058|/dev/mapper/36005076300810b3e0000000000000059|/dev/mapper/36005076300810b3e000000000000005a|/dev/mapper/36005076300810b3e000000000000005b|/dev/mapper/36005076300810b3e000000000000005c|/dev/mapper/36005076300810b3e000000000000005d|/dev/mapper/36005076300810b3e000000000000005e|/dev/mapper/36005076300810b3e000000000000005f|/dev/mapper/36005076300810b3e0000000000000060|/dev/mapper/36005076300810b3e0000000000000061|/dev/mapper/36005076300810b3e0000000000000062|/dev/mapper/36005076300810b3e0000000000000063|/dev/mapper/36005076300810b3e0000000000000064|/dev/mapper/36005076300810b3e0000000000000065|/dev/mapper/36005076300810b3e0000000000000066|/dev/mapper/36005076300810b3e0000000000000067|/dev/mapper/36005076300810b3e0000000000000068|/dev/mapper/36005076300810b3e0000000000000069|/dev/mapper/36005076300810b3e000000000000006a|/dev/mapper/36005076300810b3e000000000000006b|/dev/mapper/36005076300810b3e000000000000006c|/dev/mapper/36005076300810b3e000000000000006d|/dev/mapper/36005076300810b3e000000000000006e|/dev/mapper/36005076300810b3e000000000000006f|/dev/mapper/36005076300810b3e0000000000000070|/dev/mapper/36005076300810b3e0000000000000071|/dev/mapper/36005076300810b3e0000000000000072|/dev/mapper/36005076300810b3e0000000000000073|/dev/mapper/36005076300810b3e0000000000000074|/dev/mapper/36005076300810b3e0000000000000075|/dev/mapper/36005076300810b3e0000000000000076|/dev/mapper/36005076300810b3e0000000000000077|/dev/mapper/36005076300810b3e0000000000000078|/dev/mapper/36005076300810b3e0000000000000079|/dev/mapper/36005076300810b3e000000000000007a|/dev/mapper/36005076300810b3e000000000000007b|/dev/mapper/36005076300810b3e000000000000007c|/dev/mapper/36005076300810b3e000000000000007d|/dev/mapper/36005076300810b3e000000000000007e|/dev/mapper/36005076300810b3e000000000000007f|/dev/mapper/36005076300810b3e0000000000000080|/dev/mapper/36005076300810b3e0000000000000081|/dev/mapper/36005076300810b3e0000000000000082|/dev/mapper/36005076300810b3e0000000000000083|/dev/mapper/36005076300810b3e0000000000000084|/dev/mapper/36005076300810b3e0000000000000085|/dev/mapper/36005076300810b3e0000000000000086|/dev/mapper/36005076300810b3e0000000000000087|/dev/mapper/36005076300810b3e0000000000000088|/dev/mapper/36005076300810b3e0000000000000089|/dev/mapper/36005076300810b3e000000000000008a|/dev/mapper/36005076300810b3e000000000000008b|/dev/mapper/36005076300810b3e000000000000008c|/dev/mapper/36005076300810b3e000000000000008d|/dev/mapper/36005076300810b3e000000000000008e|/dev/mapper/36005076300810b3e000000000000008f|/dev/mapper/36005076300810b3e0000000000000090|/dev/mapper/36005076300810b3e0000000000000091|/dev/mapper/36005076300810b3e0000000000000092|/dev/mapper/36005076300810b3e0000000000000093|/dev/mapper/36005076300810b3e0000000000000094|/dev/mapper/36005076300810b3e0000000000000095|/dev/mapper/36005076300810b3e0000000000000096|/dev/mapper/36005076300810b3e0000000000000097|/dev/mapper/36005076300810b3e0000000000000098|/dev/mapper/36005076300810b3e0000000000000099|/dev/mapper/36005076300810b3e000000000000009a|/dev/mapper/36005076300810b3e000000000000009b|/dev/mapper/36005076300810b3e000000000000009c|/dev/mapper/36005076300810b3e000000000000009d|/dev/mapper/36005076300810b3e000000000000009e|/dev/mapper/36005076300810b3e000000000000009f|/dev/mapper/36005076300810b3e00000000000000a0|/dev/mapper/36005076300810b3e00000000000000a1|/dev/mapper/36005076300810b3e00000000000000a2|/dev/mapper/36005076300810b3e00000000000000a3|/dev/mapper/36005076300810b3e00000000000000a4|/dev/mapper/36005076300810b3e00000000000000a5|/dev/mapper/36005076300810b3e00000000000000a6|/dev/mapper/36005076300810b3e00000000000000a7|/dev/mapper/36005076300810b3e00000000000000a8|/dev/mapper/36005076300810b3e00000000000000a9|/dev/mapper/36005076300810b3e00000000000000aa|/dev/mapper/36005076300810b3e00000000000000ab|/dev/mapper/36005076300810b3e00000000000000ac|/dev/mapper/36005076300810b3e00000000000000ad|/dev/mapper/36005076300810b3e00000000000000ae|/dev/mapper/36005076300810b3e00000000000000af|/dev/mapper/36005076300810b3e00000000000000b0|/dev/mapper/36005076300810b3e00000000000000b1|/dev/mapper/36005076300810b3e00000000000000b2|/dev/mapper/36005076300810b3e00000000000000b3|/dev/mapper/36005076300810b3e00000000000000b4|/dev/mapper/36005076300810b3e00000000000000b5|/dev/mapper/36005076300810b3e00000000000000b6|/dev/mapper/36005076300810b3e00000000000000b7|/dev/mapper/36005076300810b3e00000000000000b8|/dev/mapper/36005076300810b3e00000000000000b9|/dev/mapper/36005076300810b3e00000000000000ba|/dev/mapper/36005076300810b3e00000000000000bb|/dev/mapper/36005076300810b3e00000000000000bc|/dev/mapper/36005076300810b3e00000000000000bd|/dev/mapper/36005076300810b3e00000000000000be|/dev/mapper/36005076300810b3e00000000000000bf|/dev/mapper/36005076300810b3e00000000000000c0|/dev/mapper/36005076300810b3e00000000000000c1|/dev/mapper/36005076300810b3e00000000000000c2|/dev/mapper/36005076300810b3e00000000000000c3|/dev/mapper/36005076300810b3e00000000000000c4|/dev/mapper/36005076300810b3e00000000000000c5|/dev/mapper/36005076300810b3e00000000000000c6|/dev/mapper/36005076300810b3e00000000000000c7|/dev/mapper/36005076300810b3e00000000000000c8|/dev/mapper/36005076300810b3e00000000000000c9|/dev/mapper/36005076300810b3e00000000000000ca|/dev/mapper/36005076300810b3e00000000000000cb|/dev/mapper/36005076300810b3e00000000000000cc|/dev/mapper/36005076300810b3e00000000000000cd|/dev/mapper/36005076300810b3e00000000000000ce|/dev/mapper/36005076300810b3e00000000000000cf|/dev/mapper/36005076300810b3e00000000000000d0|/dev/mapper/36005076300810b3e00000000000000d1|/dev/mapper/36005076300810b3e00000000000000d2|/dev/mapper/36005076300810b3e00000000000000d3|/dev/mapper/36005076300810b3e00000000000000d4|/dev/mapper/36005076300810b3e00000000000000d5|/dev/mapper/36005076300810b3e00000000000000d6|/dev/mapper/36005076300810b3e00000000000000d7|/dev/mapper/36005076300810b3e00000000000000d8|/dev/mapper/36005076300810b3e00000000000000d9|/dev/mapper/36005076300810b3e00000000000000da|/dev/mapper/36005076300810b3e00000000000000db|/dev/mapper/36005076300810b3e00000000000000dc|/dev/mapper/36005076300810b3e00000000000000dd|/dev/mapper/36005076300810b3e00000000000000de|/dev/mapper/36005076300810b3e00000000000000df|/dev/mapper/36005076300810b3e00000000000000e0|/dev/mapper/36005076300810b3e00000000000000e1|/dev/mapper/36005076300810b3e00000000000000e2|/dev/mapper/36005076300810b3e00000000000000e3|/dev/mapper/36005076300810b3e00000000000000e4|/dev/mapper/36005076300810b3e00000000000000e5|/dev/mapper/36005076300810b3e00000000000000e6|/dev/mapper/36005076300810b3e00000000000000e7|/dev/mapper/36005076300810b3e00000000000000e8|/dev/mapper/36005076300810b3e00000000000000e9|/dev/mapper/36005076300810b3e00000000000000ea|/dev/mapper/36005076300810b3e00000000000000eb|/dev/mapper/36005076300810b3e00000000000000ec|/dev/mapper/36005076300810b3e00000000000000ed|/dev/mapper/36005076300810b3e00000000000000ee|/dev/mapper/36005076300810b3e00000000000000ef|/dev/mapper/36005076300810b3e00000000000000f0|/dev/mapper/36005076300810b3e00000000000000f1|/dev/mapper/36005076300810b3e00000000000000f2|/dev/mapper/36005076300810b3e00000000000000f3|/dev/mapper/36005076300810b3e00000000000000f4|/dev/mapper/36005076300810b3e00000000000000f5|/dev/mapper/36005076300810b3e00000000000000f6|/dev/mapper/36005076300810b3e00000000000000f7|/dev/mapper/36005076300810b3e00000000000000f8|/dev/mapper/36005076300810b3e00000000000000f9|/dev/mapper/36005076300810b3e00000000000000fa|/dev/mapper/36005076300810b3e00000000000000fb|/dev/mapper/36005076300810b3e00000000000000fc|/dev/mapper/36005076300810b3e00000000000000fd|/dev/mapper/36005076300810b3e00000000000000fe|/dev/mapper/36005076300810b3e00000000000000ff|/dev/mapper/36005076300810b3e0000000000000100|/dev/mapper/36005076300810b3e0000000000000101|/dev/mapper/36005076300810b3e0000000000000102|/dev/mapper/36005076300810b3e0000000000000103|/dev/mapper/36005076300810b3e0000000000000104|/dev/mapper/36005076300810b3e0000000000000105|/dev/mapper/36005076300810b3e0000000000000106|/dev/mapper/36005076300810b3e0000000000000107|/dev/mapper/36005076300810b3e0000000000000108|/dev/mapper/36005076300810b3e0000000000000109|/dev/mapper/36005076300810b3e000000000000010a|/dev/mapper/36005076300810b3e000000000000010b|/dev/mapper/36005076300810b3e000000000000010c|/dev/mapper/36005076300810b3e000000000000010d|/dev/mapper/36005076300810b3e000000000000010e|/dev/mapper/36005076300810b3e000000000000010f|/dev/mapper/36005076300810b3e0000000000000110|/dev/mapper/36005076300810b3e0000000000000111|/dev/mapper/36005076300810b3e0000000000000112|/dev/mapper/36005076300810b3e0000000000000113|/dev/mapper/36005076300810b3e0000000000000114|/dev/mapper/36005076300810b3e0000000000000115|/dev/mapper/36005076300810b3e0000000000000116|/dev/mapper/36005076300810b3e0000000000000117|/dev/mapper/36005076300810b3e0000000000000118|/dev/mapper/36005076300810b3e0000000000000119|/dev/mapper/36005076300810b3e000000000000011a|/dev/mapper/36005076300810b3e000000000000011b|/dev/mapper/36005076300810b3e000000000000011c|/dev/mapper/36005076300810b3e000000000000011d|/dev/mapper/36005076300810b3e000000000000011e|/dev/mapper/36005076300810b3e000000000000011f|/dev/mapper/36005076300810b3e0000000000000120|/dev/mapper/36005076300810b3e0000000000000121|/dev/mapper/36005076300810b3e0000000000000122|/dev/mapper/36005076300810b3e0000000000000123|/dev/mapper/36005076300810b3e0000000000000124|/dev/mapper/36005076300810b3e0000000000000125|/dev/mapper/36005076300810b3e0000000000000126|/dev/mapper/36005076300810b3e0000000000000127|/dev/mapper/36005076300810b3e0000000000000128|/dev/mapper/36005076300810b3e0000000000000129|'\'', '\''r|.*|'\'' ] }  global {  locking_type=1  prioritise_write_locks=1  wait_for_locks=1  use_lvmetad=0 }  backup {  retain_min = 50  retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None)

Comment 3 cshao 2016-12-14 08:37:12 UTC
Created attachment 1231496 [details]
all log-part1

Comment 4 cshao 2016-12-14 08:38:22 UTC
Created attachment 1231497 [details]
all log-part2

Comment 5 Nir Soffer 2017-03-08 21:21:19 UTC
This sounds like another case for lvm filter.

Similar to bug 1424853

Comment 6 Nir Soffer 2017-05-04 18:36:47 UTC
(In reply to shaochen from comment #0)
> Description of problem:
> Install RHVH on machine which have more than 100 luns. Then register to RHVM.
> The registration will fail, the status of RHVH host always show as
> "connecting -> initializing" in RHVM web UI.

Do you have vgs and lvs on these 100 luns?

With 300 luns, 3 lvs per lun, vdsm needs almost 10
minutes to deactivate all the lvs on the luns.

How much time did you wait for vdsm to become ready?

> VDSM cshao_100luns command failed: Heartbeat exceeded
> VDSM cshao_100luns command failed: Vds timeout occured
> Host cshao_100luns is initializing. Message: Recovering from crash or
> Initializing
> 
> 
> Version-Release number of selected component (if applicable):
> rhvh-4.0-0.20161012.0+1
> RHVH-4.0-20161018.0-RHVH-x86_64-dvd1.iso
> imgbased-0.8.5-0.1.el7ev.noarch
> vdsm-4.18.13-1.el7ev.x86_64

This is not current 4.0 version. Can you test again with 4.0.7?

We disabled lvmetad in this version. This change may solve the issue.

Comment 7 cshao 2017-05-05 02:23:16 UTC
(In reply to Nir Soffer from comment #6)
> (In reply to shaochen from comment #0)
> > Description of problem:
> > Install RHVH on machine which have more than 100 luns. Then register to RHVM.
> > The registration will fail, the status of RHVH host always show as
> > "connecting -> initializing" in RHVM web UI.
> 
> Do you have vgs and lvs on these 100 luns?
> 
> With 300 luns, 3 lvs per lun, vdsm needs almost 10
> minutes to deactivate all the lvs on the luns.
> 
> How much time did you wait for vdsm to become ready?
> 
> > VDSM cshao_100luns command failed: Heartbeat exceeded
> > VDSM cshao_100luns command failed: Vds timeout occured
> > Host cshao_100luns is initializing. Message: Recovering from crash or
> > Initializing
> > 
> > 
> > Version-Release number of selected component (if applicable):
> > rhvh-4.0-0.20161012.0+1
> > RHVH-4.0-20161018.0-RHVH-x86_64-dvd1.iso
> > imgbased-0.8.5-0.1.el7ev.noarch
> > vdsm-4.18.13-1.el7ev.x86_64
> 
> This is not current 4.0 version. Can you test again with 4.0.7?
> 
> We disabled lvmetad in this version. This change may solve the issue.

The original test env was gone, let me re-setup a new test env and have a try.

Comment 8 cshao 2017-05-08 07:28:26 UTC
Re-test on the latest 4.0.7 version, the original issue has been solved now, so close this bug.

Test version:
redhat-virtualization-host-4.0-20170307.1
imgbased-0.8.16-0.1.el7ev.noarch
vdsm-4.18.24-3.el7ev.x86_64
rhevm-4.0.7.4-0.1.el7ev

Test steps:
1. Install RHVH on machine which have more than 100 luns. 
2. Register RHVH to RHVM.
3. Check the host status in RHVM web UI.

Test result:
Register RHVH to RHVM can successful within acceptable time during too many LUNs are connected to the host.