Bug 875909

Summary: [Storage] Reusing LUN that was previously used by direct lun disk and storage domain causes to GetLunsByVgIdQuery fail with null pointer exception.
Product: Red Hat Enterprise Virtualization Manager Reporter: Leonid Natapov <lnatapov>
Component: ovirt-engineAssignee: Liron Aravot <laravot>
Status: CLOSED CURRENTRELEASE QA Contact: vvyazmin <vvyazmin>
Severity: high Docs Contact:
Priority: unspecified    
Version: unspecifiedCC: abaron, amureini, dyasny, hateya, iheim, laravot, lpeer, Rhev-m-bugs, sgrinber, yeylon, ykaul
Target Milestone: ---Keywords: ZStream
Target Release: 3.2.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: storage
Fixed In Version: sf3 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 889793 (view as bug list) Environment:
Last Closed: 2013-06-11 08:17:51 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: Storage RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 889793, 915537    
Attachments:
Description Flags
engine log
none
vdsm log
none
## Logs vdsm, rhevm none

Description Leonid Natapov 2012-11-12 19:37:37 UTC
Created attachment 643681 [details]
engine log

GetLunsByVgIdQuery fails with null pointer exception.

For some reason LUN IDs list that was sent by vdsm to engine didn't match the LUNs IDs in the database. One LUN ID was missing in the database. The scenario is the following:

2 ISCSI storage domains. One of the domains was extended with the LUN from another Storage Domain. 

engine and vdsm logs attached.

2012-11-12 19:10:42,678 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (ajp--127.0.0.1-8702-7) START, GetDeviceLi
stVDSCommand(HostName = purple-vds2, HostId = e0fdb2b8-2c14-11e2-bc65-97e1ef3fa41e, storageType=ISCSI), log id: 87eb108
2012-11-12 19:11:01,545 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (ajp--127.0.0.1-8702-9) START, GetDeviceLi
stVDSCommand(HostName = purple-vds2, HostId = e0fdb2b8-2c14-11e2-bc65-97e1ef3fa41e, storageType=null), log id: 6f3e4543
2012-11-12 19:11:03,407 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (ajp--127.0.0.1-8702-9) Failed in GetDeviceListVD
S method, for vds: purple-vds2; host: purple-vds2
2012-11-12 19:11:03,408 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (ajp--127.0.0.1-8702-9) Command GetDeviceListVDS execution fa
iled. Exception: NullPointerException: 
2012-11-12 19:11:03,408 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (ajp--127.0.0.1-8702-9) FINISH, GetDeviceL
istVDSCommand, log id: 6f3e4543
2012-11-12 19:11:03,409 ERROR [org.ovirt.engine.core.bll.GetLunsByVgIdQuery] (ajp--127.0.0.1-8702-9) Query GetLunsByVgIdQuery failed. Except
ion message is VdcBLLException: java.lang.NullPointerException
2012-11-12 19:11:15,395 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (ajp--127.0.0.1-8702-7) FINISH, GetDeviceL
istVDSCommand, return: [org.ovirt.engine.core.common.businessentities.LUNs@c8a47713, org.ovirt.engine.core.common.businessentities.LUNs@b773
8574, org.ovirt.engine.core.common.businessentities.LUNs@1ad92417, org.ovirt.engine.core.common.businessentities.LUNs@818029a, org.ovirt.eng
ine.core.common.businessentities.LUNs@eb0e5748], log id: 87eb108

Comment 1 Leonid Natapov 2012-11-12 19:38:41 UTC
Created attachment 643682 [details]
vdsm log

Comment 2 Leonid Natapov 2012-11-13 10:56:11 UTC
How to reproduce:

1. create iscsi domain with lun X
2. create disk with lun X
3. remove domain
4. remove disk
5. create domain with lun X
6. activate domain
7. edit storage domain.

Also when destroying storage domain after steps 1-7 ,the domain not removed from the database.

Comment 3 Liron Aravot 2012-11-13 17:15:44 UTC
http://gerrit.ovirt.org/9228

Comment 4 Liron Aravot 2012-11-13 17:21:25 UTC
correct patch : 
http://gerrit.ovirt.org/#/c/9229/

Comment 6 Allon Mureinik 2012-12-12 10:31:38 UTC
patches are merged upstream.

Comment 7 Simon Grinberg 2012-12-23 15:34:02 UTC
This bug has been cloned to the RHEV-M .z stream:            https://bugzilla.redhat.com/show_bug.cgi?id=889793

Comment 8 vvyazmin@redhat.com 2013-01-16 14:01:38 UTC
No issues are found on iSCSI DC environment 

Verified on RHEVM 3.2 - SF03 environment 

RHEVM: rhevm-3.2.0-4.el6ev.noarch
VDSM: vdsm-4.10.2-3.0.el6ev.x86_64
LIBVIRT: libvirt-0.10.2-13.el6.x86_64
QEMU & KVM: qemu-kvm-rhev-0.12.1.2-2.348.el6.x86_64
SANLOCK: sanlock-2.6-2.el6.x86_64

Comment 9 Yaniv Kaul 2013-01-30 09:35:10 UTC
(In reply to comment #8)
> No issues are found on iSCSI DC environment 
> 
> Verified on RHEVM 3.2 - SF03 environment 
> 
> RHEVM: rhevm-3.2.0-4.el6ev.noarch
> VDSM: vdsm-4.10.2-3.0.el6ev.x86_64
> LIBVIRT: libvirt-0.10.2-13.el6.x86_64
> QEMU & KVM: qemu-kvm-rhev-0.12.1.2-2.348.el6.x86_64
> SANLOCK: sanlock-2.6-2.el6.x86_64

Please verify on FC as well.

Comment 10 vvyazmin@redhat.com 2013-02-03 13:40:50 UTC
Failed on step 7: edit storage domain in FC DC environment

/var/log/ovirt-engine/engine.log

2013-02-03 17:36:43,580 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (ajp-/127.0.0.1:8702-8) START, GetDeviceListVDSCommand(HostName = green-vdsa, HostId = 82e5c19b-d738-4b00-95ad-890bd6edf0ee, storageType=FCP), log id: 79170b1e
2013-02-03 17:36:43,613 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (ajp-/127.0.0.1:8702-7) START, GetDeviceListVDSCommand(HostName = green-vdsa, HostId = 82e5c19b-d738-4b00-95ad-890bd6edf0ee, storageType=null), log id: 18484190
2013-02-03 17:36:43,613 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand] (ajp-/127.0.0.1:8702-7) Failed in GetDeviceListVDS method, for vds: green-vdsa; host: green-vdsa.qa.lab.tlv.redhat.com
2013-02-03 17:36:43,613 ERROR [org.ovirt.engine.core.vdsbroker.VDSCommandBase] (ajp-/127.0.0.1:8702-7) Command GetDeviceListVDS execution failed. Exception: NullPointerException: 
2013-02-03 17:36:43,613 INFO  [org.ovirt.engine.core.vdsbroker.vdsbroker.GetDeviceListVDSCommand] (ajp-/127.0.0.1:8702-7) FINISH, GetDeviceListVDSCommand, log id: 18484190
2013-02-03 17:36:43,614 ERROR [org.ovirt.engine.core.bll.GetLunsByVgIdQuery] (ajp-/127.0.0.1:8702-7) Query GetLunsByVgIdQuery failed. Exception message is VdcBLLException: java.lang.NullPointerException


RHEVM 3.2 - SF05 environment:

RHEVM: rhevm-3.2.0-6.el6ev.noarch
VDSM: vdsm-4.10.2-5.0.el6ev.x86_64
LIBVIRT: libvirt-0.10.2-17.el6.x86_64
QEMU & KVM: qemu-kvm-rhev-0.12.1.2-2.348.el6.x86_64
SANLOCK: sanlock-2.6-2.el6.x86_64

Comment 11 vvyazmin@redhat.com 2013-02-03 13:43:08 UTC
Created attachment 692309 [details]
## Logs vdsm, rhevm

Comment 12 Liron Aravot 2013-02-03 13:51:38 UTC
this is the bug for FC , it's not merged to 3.2 yet...
https://bugzilla.redhat.com/show_bug.cgi?id=896042

Comment 13 vvyazmin@redhat.com 2013-02-03 13:57:55 UTC
iSCSI DC environment: No issues are found
FC DC environment: Failed, open separate bug BZ896042

Comment 14 Itamar Heim 2013-06-11 08:17:51 UTC
3.2 has been released

Comment 15 Itamar Heim 2013-06-11 08:22:38 UTC
3.2 has been released