Bug 1370414 - [TestOnly] Storage devices displayed wrong and RHVH installed failed with multpath devices
Summary: [TestOnly] Storage devices displayed wrong and RHVH installed failed with mul...
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: ovirt-node
Classification: oVirt
Component: Installation & Update
Version: 4.0
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ovirt-4.0.5
: ---
Assignee: Ryan Barry
QA Contact: Wei Wang
URL:
Whiteboard:
Depends On:
Blocks: ovirt-node-ng-platform
TreeView+ depends on / blocked
 
Reported: 2016-08-26 09:11 UTC by Wei Wang
Modified: 2016-11-29 10:43 UTC (History)
12 users (show)

Fixed In Version: redhat-virtualization-host-productimg-4.0-0.2
Doc Type: If docs needed, set a value
Doc Text:
Cause: Installing RHVH 4.0 on a host with multipath devices Consequence: The multipath device will not be shown in the "Local disks" section. The section will rather show all paths of the multipath device. Workaround (if any): To add a multipath device, the user can select the "Add a disk ..." from the "Specialized & Network Disks" section, and enter the "Multipath" tab to discover and select multipath devices. Result: By using the workaround multipath devices can be selected as an installation destination.
Clone Of:
Environment:
Last Closed: 2016-11-29 10:43:50 UTC
oVirt Team: Node
Embargoed:
rule-engine: ovirt-4.0.z+
rule-engine: blocker+
mgoldboi: planning_ack+
rule-engine: devel_ack+
ycui: testing_ack+


Attachments (Terms of Use)
pic1 (216.23 KB, image/png)
2016-08-26 09:11 UTC, Wei Wang
no flags Details
pic2 (188.36 KB, image/png)
2016-08-26 09:12 UTC, Wei Wang
no flags Details
pic3 (124.85 KB, image/png)
2016-08-26 09:14 UTC, Wei Wang
no flags Details
pic1 (117.63 KB, image/png)
2016-08-26 10:24 UTC, dguo
no flags Details
pic2 (116.80 KB, image/png)
2016-08-26 10:25 UTC, dguo
no flags Details
log file (24.05 KB, application/x-gzip)
2016-08-26 10:56 UTC, Wei Wang
no flags Details
iscsi_1 (52.95 KB, image/png)
2016-09-01 08:51 UTC, Wei Wang
no flags Details
iscsi_2 (61.32 KB, image/png)
2016-09-01 08:51 UTC, Wei Wang
no flags Details

Description Wei Wang 2016-08-26 09:11:40 UTC
Created attachment 1194252 [details]
pic1

Description of problem:
Storage devices displayed wrong and RHVH installed failed with multiple FC devices.

Version-Release number of selected component (if applicable):
RHVH-4.0-20160822.8-RHVH-x86_64-dvd1.iso


How reproducible:
100%


Steps to Reproduce:
1. Clean all storage devices on multiple FC devices
2. Interactive installation RHVH-4.0-20160822.8-RHVH-x86_64-dvd1.iso with multiple FC devices
3. Enter page of selecting storage devices for installation
4. Check the display
5. Select "automatically configure partitioning", then click "Done" button


Actual results:
1. The the storage devices are displayed wrong and RHVH cannot be installed successfully with multiple FC devices.(refer to attachment:wrong_822.png)
2. There is an error message displayed, the error detail is : device is not in the tree.(refer to attachment:error_detail.png)
3. After step 5, "No disk selected" error message is displayed under "INSTALLATION DESTINATION", and "Begin Installation" button is invalid.

Expected results:
1. The storage devices should be displayed right, and should be installed successfully with multiple FC devices.(refer to attachment:right_817.png)
2. There should not be any error message displayed.
3. After step 5, "Saved disk selected" message is displayed under "INSTALLATION DESTINATION", and "Begin Installation" button is valid.


Additional info:

Comment 1 Wei Wang 2016-08-26 09:12:17 UTC
Created attachment 1194253 [details]
pic2

Comment 2 Wei Wang 2016-08-26 09:14:03 UTC
Created attachment 1194254 [details]
pic3

Comment 3 Wei Wang 2016-08-26 09:18:15 UTC
Can not access shell for log fetching. If necessary, I will contact admin to help us for the log file.

Comment 4 Ying Cui 2016-08-26 09:35:15 UTC
(In reply to weiwang from comment #3)
> Can not access shell for log fetching. If necessary, I will contact admin to
> help us for the log file.

Wei, Let's try to get the log and eng-ops@ will support us in remote lab PEK2 today for catching the logs in these servers.

Comment 5 Wei Wang 2016-08-26 09:37:34 UTC
How reproducible:
Tested 5 times on machine hp-dl385pg8-11, reproduced 5 times on build RHVH-4.0-20160822.8-RHVH-x86_64

Log file will be provided later.

Comment 6 Fabian Deutsch 2016-08-26 10:21:00 UTC
Moved this to anconda, as this is with an ISO which we know is good, and it's way before the RHV content.

However, can you please try to reproduce this with a RHEL 7.2 or 7.3 beta ISO?

Comment 7 dguo 2016-08-26 10:24:10 UTC
Seems that reproduce on a iscsi machine.

This iscsi system has 3 iscsi LUN, 200GB/99GB/99GB,
but the disk shown on the picture is nine.
Please see the details on the picture I attached.

Comment 8 dguo 2016-08-26 10:24:40 UTC
Created attachment 1194286 [details]
pic1

Comment 9 dguo 2016-08-26 10:25:07 UTC
Created attachment 1194288 [details]
pic2

Comment 10 dguo 2016-08-26 10:34:26 UTC
Find another system to retest, but no such issue found.
So test 2 times on hp-z800-02, failed 2 times on build 20160816
test 2 times on dell-pet105-01, not reproduced.

Comment 11 Ying Cui 2016-08-26 10:39:20 UTC
(In reply to dguo from comment #10)
> Find another system to retest, but no such issue found.
> So test 2 times on hp-z800-02, failed 2 times on build 20160816

hp-z800-02 is multipath iSCSI.


> test 2 times on dell-pet105-01, not reproduced.

dell-pet105-01 is single path iSCSI.

So this is bug can be reproduced on multipath devices, at least in QE env. testing on build 0822.8.

Comment 12 Wei Wang 2016-08-26 10:56:07 UTC
Created attachment 1194304 [details]
log file

Comment 14 Wei Wang 2016-08-26 11:29:32 UTC
Sosreport size is too big. so please download from below link.
http://10.66.65.30/others/385pg8-11tmplog.tar.gz

Comment 16 Wei Wang 2016-08-29 09:02:51 UTC
Test Version:
RHEL-7.2-20151015.0-Server-x86_64-dvd1.iso

Test Steps:
1. Clean all storage devices on multiple FC devices
2. Interactive installation RHEL-7.2-20151015.0-Server-x86_64-dvd1.iso with multiple FC devices
3. Enter page of selecting storage devices for installation
4. Check the display
5. Select "automatically configure partitioning", then click "Done" button


Result:
1. The storage devices are displayed rightly and are installed successfully with multiple FC devices.
2. There is not any error message displayed.
3. After step 5, "Saved disk selected" message is displayed under "INSTALLATION DESTINATION", 
and "Begin Installation" button is valid.


So the bug cannot be reproduced with RHEL 7.2

Comment 17 Fabian Deutsch 2016-08-29 09:34:14 UTC
Ryan, can you please name the patches you cherry-picked into our anaconda branch, I suspect that this bug is a regression of them.

Comment 19 Ying Cui 2016-08-30 05:40:01 UTC
Adding Testblocker keywords, with this bug, we can not install NGN on multipath devices and can not continue FC and iSCSI storage testing.

Comment 20 Wei Wang 2016-08-30 06:11:22 UTC
I can not reproduce this issue on RHVH-4.0-20160817.1-RHVH-x86_64-dvd1.iso, only can reproduce this issue on RHVH-4.0-20160822.8-RHVH-x86_64-dvd1.iso, so consider it is a regression.

Comment 21 Red Hat Bugzilla Rules Engine 2016-08-31 07:46:16 UTC
Bug tickets must have version flags set prior to targeting them to a release. Please ask maintainer to set the correct version flags and only then set the target milestone.

Comment 22 Red Hat Bugzilla Rules Engine 2016-08-31 07:46:57 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 23 Fabian Deutsch 2016-08-31 08:14:58 UTC
I can reproduce this in a VM with a multipathed device.

But it's probably the correct behavior.

According to

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Installation_Guide/sect-storage-devices-x86.html

the paths of a multipath device are shown as disks in the primary screen.
To select the multipath device (aggregation of all it's paths), the user needs to select "Add a disk ..." from the "Specialized & Network Disks" section.
In the following screen, there is a tab called "Multipath devices" where - in my testing - the correct multipath device is shown.

Wei, can you confirm that this is working for you as well?

Note: The assumption why it looked "more correct" in RHVH-4.0-20160817.1-RHVH-x86_64-dvd1.iso was, that vdsm modified multipath.conf to show all devices as multipath devices, which changed how devices were shown in anaconda.

Comment 24 Wei Wang 2016-08-31 09:07:11 UTC
I think they are different. Since the multipath devices are added automatically in anaconda. This issue is that the local disk should be one disk but many disks are displayed in "local standard disk", then clicking the Done button, the disk cannot be selected successfully, it is still highlighted(red color) for the user to select, so installation cannot be continued.

Comment 25 Fabian Deutsch 2016-08-31 09:22:38 UTC
Yes ... It should be as you describe.
The multipath device should be shown in the "Specialized & Network disks" Section without any further action.

Using the command:
$ diff -u <(cd $rhvh ; find . ! -path "*/usr/lib/modules/*" | sort) <(cd $rhel ; find . ! -path "*/usr/lib/modules/*" | sort) | less

I was able to see that there are some differences between 0822.8 and RHEL 7.2 stage2. The most notable one for this bug:

-./etc/systemd/system/sysinit.target.wants/multipathd.service

Which means that multipathd.service is not enabled by default in the RHVH stage2 squashfs - which could explain the issue.

We need to understand why multipathd is not enabled in stage2.

Comment 27 Ying Cui 2016-09-01 06:50:50 UTC
This bug is not targeted to 4.0.3, so all mutlpaths storage testing(FC and iSCSI) on RHVH and vdsm relevant are blocked by this bug.

Comment 28 Fabian Deutsch 2016-09-01 07:14:44 UTC
Please use the "Specialized & Network Disks" section "Add disk ..." button to workaround this issue.

Comment 29 Ying Cui 2016-09-01 07:29:31 UTC
Wei, could you help to try the workaround in comment 3 on RHVH 4.0 async ISO?

Comment 30 Ying Cui 2016-09-01 07:30:04 UTC
(In reply to Ying Cui from comment #29)
> Wei, could you help to try the workaround in comment 3 on RHVH 4.0 async ISO?

The workaround should be in comment 28

Comment 31 Wei Wang 2016-09-01 08:47:29 UTC
Test Version
RHVH-4.0-20160831.4-RHVH-x86_64-dvd1.iso (Virtual media)


Test Steps:
1. Clean all storage devices on multiple iscsi devices
2. Interactive installation RHVH-4.0-20160822.8-RHVH-x86_64-dvd1.iso/redhat-virtualization-host-4.0-20160829.0.x86_64 with multiple iscsi devices
3. Enter page of selecting storage devices for installation
4. Click "Add disk ..." button in the "Specialized & Network Disks" section, then check the display.
5. Select "automatically configure partitioning", then click "Done" button

Results:
After step 4, there some redundant disks are displayed. (picture is attached)
After step 5, "No disk selected" error message is displayed under "INSTALLATION DESTINATION", and installation cannot be continued

More info:
Also test with redhat-virtualization-host-4.0-20160829.0.x86_64.liveimg.squashfs (using PXE), the issue does not occur

Comment 32 Wei Wang 2016-09-01 08:51:15 UTC
Created attachment 1196624 [details]
iscsi_1

Comment 33 Wei Wang 2016-09-01 08:51:53 UTC
Created attachment 1196625 [details]
iscsi_2

Comment 35 cshao 2016-09-08 07:54:37 UTC
Hi Rayn,

I'd like to ask whether the bug is fixed by build RHVH-4.0-20160907.4-RHVH-x86_64-dvd1.iso?

I noticed that the target milestone of this bug is set to 4.0.4, but seem the bug is gone from above 4.0.3 iso.
Install on multipath FC - pass(Dell R510-01)
Install on multipath iSCSI - pass(Dell R515-01)

Thanks.

Comment 36 Ryan Barry 2016-09-08 11:54:06 UTC
There were no changes in Anaconda, but I didn't test this on 4.0.3. I'll take your word for it

Comment 37 Fabian Deutsch 2016-09-10 12:25:33 UTC
Moving to 4.0.5 - because this will only be fixed with a new anaconda build, which will only appear in RHEL 7.3.

Comment 38 Fabian Deutsch 2016-11-28 15:48:26 UTC
Wei, could you please check if this bug is fixed with the 4.0.5 build, which is RHEL 7.3 based?

Comment 39 Red Hat Bugzilla Rules Engine 2016-11-28 15:48:37 UTC
This bug report has Keywords: Regression or TestBlocker.
Since no regressions or test blockers are allowed between releases, it is also being identified as a blocker for this release. Please resolve ASAP.

Comment 41 Wei Wang 2016-11-29 09:24:24 UTC
Test Version:
RHVH-4.0-20161116.0-RHVH-x86_64-dvd1.iso (Virtual media)

Test Steps:
1. Clean all storage devices on multiple FC devices
2. Interactive installation RHVH iso with multiple FC devices
3. Enter page of selecting storage devices for installation
4. Check the display
5. Select "automatically configure partitioning", then click "Done" button

Results:
After step 4, the storage devices are displayed rightly, and installation is successful.

Comment 42 Wei Wang 2016-11-29 09:25:58 UTC
remove needinfo of weiwang@

Comment 43 Fabian Deutsch 2016-11-29 10:43:50 UTC
Thanks. CLosing according to comment 41.


Note You need to log in before you can comment on or make changes to this bug.