Bug 1209028

Notes to QA for testing in 3.5.1:
 - create an iSCSI domain
 - extend it
 - move to maintenance

All flows should pass.

Tested the scenario with rhev-hypervisor6-6.6-20150327.0.el6 which the bug was reported about.

The scenario I checked:
- Installed RHEV-H 6.6. Configured RHEVM in the admin TUI of RHEV-H
- From RHEV-M, approved the RHEV-H and upgraded it to the relevant build (the mentioned above). 
- Created a new LUN in the storage server and mapped it to the host.
- Attached the host to an uninitialized DC and added a new storage domain to it using the RHEV-H.

storage domain creation succeeds.


rhev-hypervisor6.noarch
rhevm-3.5.1-0.4.el6ev.noarch
vdsm-4.16.12.1-3.el6ev.x86_64
libvirt-0.10.2-46.el6_6.3.x86_64
sanlock-2.8-1.el6.x86_64
device-mapper-event-libs-1.02.90-2.el6_6.1.x86_64
device-mapper-multipath-libs-0.4.9-80.el6_6.3.x86_64
device-mapper-persistent-data-0.3.2-1.el6.x86_64
device-mapper-1.02.90-2.el6_6.1.x86_64
device-mapper-event-1.02.90-2.el6_6.1.x86_64
device-mapper-multipath-0.4.9-80.el6_6.3.x86_64
device-mapper-libs-1.02.90-2.el6_6.1.x86_64
ovirt-node-plugin-vdsm-0.2.0-20.el6ev.noarch
qemu-kvm-rhev-0.12.1.2-2.446.el6.x86_64



Nir, Douglas, can you please tell me what I've done wrongly?

attaching the logs

Created attachment 1015135 [details]
logs - 16.4.15

(In reply to Elad from comment #2)
> Nir, Douglas, can you please tell me what I've done wrongly?

I think the issue may happen when adding the first host in the engine.
Maybe you have other hosts in the dc or in other dcs, which are
connected to the same storage?

I would try to reproduce it by starting with clean engine, add
the first host using iscsi and create the first storage domain.

I tested also with a clean setup (created a new DC and storage domain) in addition to the steps I wrote in comment #2. The bug does not reproduce.

Here is the disconnectStoragePool :

Thread-419::INFO::2015-04-16 14:10:07,751::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '04f6cd60-7a25-424f-b601-17ce5eed98ed'}, {'status': 0, 'id': 'c856d145-ea86-40df-a88c-7fba5d682633'}, {'status': 0, 'id': 'bdd06a88-b941-4e16-9523-bd9568436685'}, {'status': 0, 'id': 'd53dbdb5-a8ad-4b69-82ec-8dd53cbafbf0'}]}


Attaching the logs.

Created attachment 1015247 [details]
logs - 16.4.15 (2)

Wasn't able to reproduce using the problematic build. Might need to test on the reporter's environment.

Elad, the difference between your setup and Douglas is the number of paths - you connect to server with 4 targets (#npath=4), but Douglass is using (#npath=1).

This is the flow on your setup:

1. create vg
2. create storage domain
3. disconnect from target 1
4. disconnect from target 2
5. disconnect from target 3
6. disconnect from target 4
7. connect from target 1
8. connect from target 2
9. connect from target 3
10. connect from target 4
11. connect to storage pool

When you connect to the storage pool, the devices from first connections are
ready when you connect to the storage pool, so the vg is available when
we connect to the storage pool.

In Douglass setup the flow is:

1. create vg
2. create storage domain
3. disconnect from target 1
4. connect from target 1
5. connect to storage pool

When connecting to storage pool, we run vgck, and it fails becasue
the vg is not ready yet.

Please repeat this test when only one path is exposed to the host. You may also
need to repeat this tests several times, as this is a timing issue.

(In reply to Nir Soffer from comment #11)
Nir, 
Tested again according to the flow you described, storage domain creation went fine while the host got connected and disconnected from a single target.

(In reply to Elad from comment #12)
> Tested again according to the flow you described, storage domain creation
> went fine while the host got connected and disconnected from a single target.

Flow went fine with new version and failed with the old version? Went fine with both?

Please attach vdsm logs.

Created attachment 1016068 [details]
logs-19.4.15

Tested using the build that you mentioned as 'bad' - 20150327

Attaching the logs

(In reply to Elad from comment #14)
> Tested using the build that you mentioned as 'bad' - 20150327
In the logs we see:

1. disconnectStorageServer (with one connection)
2. connectStorageServer
3. lvs failure - Device /dev/mapper/3514f0c5447601130 was disabled
   the pv is not available after connect
4. createStoragePool
5. vgck - succeeds - this fails in Douglas setup

Turns our that Elad was testing the same host, and was not installing a fresh host.
It is possible that this issue happens only when installing a fresh host.

(In reply to Nir Soffer from comment #15)


Tested again with a freshly installed RHEV-H. The bug does not reproduce with build 20150327

Attaching the logs

Created attachment 1016254 [details]
logs-20.4

 Douglas , 

As we are not able to reproduce for sure, can you try to verify on your setup as well?

(In reply to Aharon Canan from comment #18)
>  Douglas , 
> 
> As we are not able to reproduce for sure, can you try to verify on your
> setup as well?

Hello Aharon,

My setup has been working nicely. As the original scratch-build iso of report is not available anymore in brew, I could re-test it with the same iso you guys are testing to double check. If you like that idea, please let me know the iso.

Thanks!

Hi Douglas,

It would be great if you could check the scenario with the 2 RHEV-H builds (the old and the new). I will provide you with both, please ack and I'll contact you offline.
Thanks.

(In reply to Elad from comment #20)
> Hi Douglas,
> 
> It would be great if you could check the scenario with the 2 RHEV-H builds
> (the old and the new). I will provide you with both, please ack and I'll
> contact you offline.
> Thanks.

Hi Elad,

I am ready to test, please just let me know which isos you want me to test.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-0904.html