Bug 1269338

(In reply to nijin ashok from comment #0)
> Description of problem:
> 
> The installation of RHEV hypervisor is getting failed if any of the LUN
> which is mapped to the server contains a partial volume group. The
> installation failed with error "couldn't find any device with uuid".
> 
> 
> Version-Release number of selected component (if applicable):
> rhevh-6.7-20150911.0.el6ev
> 
> How reproducible:
> 100%
> 
> Steps to Reproduce:
> 1. Map a LUN to the server which is having a partial volume group
> 2. Install the RHEV hypervisor in any of the disk other than the LUN which
> contains partial volume group
> 3. Installation will fail just after allocating the space.
> 
> Actual results:
> Installation fails if the mapped LUN contains partial volume group
> 
> 
> Expected results:
> Installation should succeed
> 
> Additional info:

After reading the case and the sosreport, the failure to install is unrelated to any number of devices in the /dev directory.

I'll provide a "best effort" patch which attempts to use -P on nondestructive LVM commands (vgs, vgdisplay, pvdisplay, etc) and does not throw an exception if there's a non-zero exit code in this case.

However, since removing/masking the problem LUNs from the customer's system resolves it, there is a clear fix.

I am very hesitant to ignore LVM warnings in potentially destructive cases elsewhere in the installer (removing vgs/pvs/lvs, creating them), and no changes will be made there. If LVM also warns about not being able to find a device in those cases and the installer catches the exception, the recommended solution will be to mask or unmap the problematic LUNs, since any changes there could lead to a destructive overwrite of data the customer does care about.

If it continues to throw exceptions and a behavior change is desired, I'd suggest you open a bug against LVM, since RHEV-H cannot be expected to know the return codes, risks, and behaviors of all the LVM utilities, what is dangerous, and what is simply a warning.

I can't reproduce this issue.

Test version:
rhevh-6.7-20150911.0.el6ev + ovirt-node-3.2.3-20.el6.noarch
rhev-hypervisor6-6.7-20150828.0  +  ovirt-node-3.2.3-20.el6.noarch

Test machine:
multipath fc machine: dell-per510-01

Test scenario 1- auto install.
Test steps:
1. Install RHEV-H on lun 1.
2. Register to RHEV-M
3. Add fc/iscsi storage(lun 2)
4. Reboot
5. Auto install RHEV-H on lun 1.

Test result:
It will report "Device specified in storage_init does not exist"
Automatic installation failed.

Test scenario 2 - tui install.
Test steps:
1. TUI Install RHEV-H on lun 1.
2. Register to RHEV-M
3. Add fc/iscsi storage(lun 2)
4. Reboot
5. TUI install RHEV-H on lun 1.


Test result:
Install can successful.


Test scenario 2 - tui install.
Test steps:
1. TUI Install RHEV-H on lun 1.
2. Register to RHEV-M
3. Add fc/iscsi storage(lun 2)
4. Reboot
5. TUI install RHEV-H on lun 1.


Test result:
Install can successful.


Test scenario 3 - manually create pv.
Test steps:
1. TUI Install RHEV-H on lun 1.
2. Register to RHEV-M
3. Add fc/iscsi storage(lun 2).
4. manually create pv on lun 3.
5. Reboot
5. Auto install RHEV-H on lun 1.


Test result:
It will report "Device specified in storage_init does not exist"
Automatic installation failed.



Hi nashok,

 could you please provide the detail test steps for reproduce this bug?

Thanks!

To my understanding the bug should appear in the following setup:

1. Get a SAN with at least 3 LUNs
2. Create a VG (using RHEL or any other distro) over the first two LUNs (both LUNs become PVs)
3. Try to install RHEV-H on LUN 3

During step 4 the RHEV-H Installer will raise an exception

Instead of a SAN you can also use a host with three (3) disks

yes, the multipath fc machine just has 3 LUNs.

[root@unused admin]# multipath -ll
360050763008084e6e000000000000058 dm-7 IBM,2145
size=100G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 5:0:0:3 sdd 8:48 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  `- 5:0:1:3 sdg 8:96 active ready running
360050763008084e6e000000000000057 dm-6 IBM,2145
size=100G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 5:0:1:2 sdf 8:80 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  `- 5:0:0:2 sdc 8:32 active ready running
36782bcb03cdfa200174636ff055184dc dm-1 DELL,PERC 6/i
size=544G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
  `- 4:2:0:0 sda 8:0  active ready running
360050763008084e6e000000000000056 dm-0 IBM,2145
size=200G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 5:0:0:1 sdb 8:16 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  `- 5:0:1:1 sde 8:64 active ready running

[root@unused admin]# pvs
  PV                                              VG                                   Fmt  Attr PSize   PFree 
  /dev/mapper/360050763008084e6e000000000000056p4 HostVG                               lvm2 a--  199.28g 28.00m
  /dev/mapper/360050763008084e6e000000000000057   988c2726-07b7-43bb-adec-f421f7fb3969 lvm2 a--   99.62g 95.75g
[root@unused admin]# vgs
  VG                                   #PV #LV #SN Attr   VSize   VFree 
  988c2726-07b7-43bb-adec-f421f7fb3969   1   6   0 wz--n-  99.62g 95.75g
  HostVG                                 1   4   0 wz--n- 199.28g 28.00m

Test steps:
1. Get a SAN with at least 3 LUNs
2. Create a VG (using RHEL or any other distro) over the first two LUNs (both LUNs become PVs) (360050763008084e6e000000000000056+360050763008084e6e000000000000057)
3. Try to auto install RHEV-H on LUN 3(360050763008084e6e000000000000058)

Test result:
Auto install failed, it will report "HostVG/APPVG exists on separate disk", manual intervention required

Created attachment 1102105 [details]
Failed-3luns.png

My test steps were wrong, they should have been:

1. Get a SAN with at least 3 LUNs
2. Create a VG (using RHEL or any other distro) over the first two LUNs (both LUNs become PVs)
3. (NEW STEP) Blacklist/remove/disconnect one of the first two LUNs to make the VG losse one of it's PVs
4. Try to install RHEV-H on LUN 3

During step 4 the RHEV-H Installer will raise an exception

(In reply to Fabian Deutsch from comment #12)
> My test steps were wrong, they should have been:
> 
> 1. Get a SAN with at least 3 LUNs
> 2. Create a VG (using RHEL or any other distro) over the first two LUNs
> (both LUNs become PVs)
> 3. (NEW STEP) Blacklist/remove/disconnect one of the first two LUNs to make
> the VG losse one of it's PVs
> 4. Try to install RHEV-H on LUN 3
> 
> During step 4 the RHEV-H Installer will raise an exception

Still can't reproduce this issue with above steps.

It still report "HostVG/APPVG exists on separate disk", manual intervention required.

Please try it with a VG name different than HostVG.

(In reply to Fabian Deutsch from comment #14)
> Please try it with a VG name different than HostVG.

Reproduce it now.

Test version:
rhevh-6.7-20150911.0.el6ev
ovirt-node-3.2.3-20.el6.noarch

Test steps:
1. Insert USB disk to local machine.
2. Create a VG over the two disks (disk 1 + disk 2)
# pvcreate /dev/sdb;  pvcreate /dev/sdc; 
# vgcreate testVG /dev/sdb /dev/sdc
# lvcreate -L3000 testlv testVG

3. Pull one of the disk(disk 1) to make the VG losse one of it's PVs
4. Try to install RHEV-H on disk 2.

Thanks!

This is non critical, thus moving it out to a z-stream

Chen,

could you please check how anaconda behaves with partial volume groups?

OK, I will find time to trace this issue.

(In reply to Fabian Deutsch from comment #21)
> Chen,
> 
> could you please check how anaconda behaves with partial volume groups?


Anaconda can handle this correctly.

Test version:
rhev-hypervisor7-ng-3.6-20160518.0
imgbased-0.6-0.1.el7ev.noarch


Test steps:
1. Insert USB disk to local machine.
2. Anaconda interactive install NGN.
3. Press ctrl+alt=F2 enter shell mode
4. Create a VG over the two disks (disk 1 + disk 2)
# pvcreate /dev/sdb;  pvcreate /dev/sdc; 
# vgcreate testVG /dev/sdb /dev/sdc
# lvcreate -L3000 testVG
5. Pull one of the disk(disk 1) to make the VG losse one of it's PVs
6. Anaconda interactive install NGN on disk 2.

Test result:
Anaconda interactive install NGN on disk 2 can successful.

Does anaconda raise a warning or alike?

At least I am moving this bug to ON_QA according to comment 23

(In reply to Fabian Deutsch from comment #24)
> Does anaconda raise a warning or alike?
> 
> At least I am moving this bug to ON_QA according to comment 23

No warning, anaconda can remove all partial volume group smoothly.

Test version:
rhev-hypervisor7-ng-4.0-20160527.0 
imgbased-0.7.0-0.1.el7ev.noarch


Test steps:
1. Insert USB disk to local machine.
2. Anaconda interactive install NGN.
3. Press ctrl+alt=F2 enter shell mode
4. Create a VG over the two disks (disk 1 + disk 2)
# pvcreate /dev/sdb;  pvcreate /dev/sdc; 
# vgcreate testVG /dev/sdb /dev/sdc
# lvcreate -L3000 testVG
5. Pull one of the disk(disk 1) to make the VG losse one of it's PVs
6. Anaconda interactive install NGN on disk 2.

Test result:
Anaconda interactive install NGN on disk 2 can successful.

So the bug is fixed, change bug status to VERIFIED.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1702.html