Bug 621070

Summary: storage test failes using lvm swap partition on RHEL6
Product: [Retired] Red Hat Hardware Certification Program Reporter: qcui
Component: Test Suite (tests)Assignee: Greg Nichols <gnichols>
Status: CLOSED ERRATA QA Contact: Red Hat Kernel QE team <kernel-qe>
Severity: high Docs Contact:
Priority: low    
Version: 1.2CC: czhang, emcnabb, qcai, rlandry, ykun
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
The V7-1.2-14 storage test no longer fails on IBM System z Red Hat Enterprise Linux 6 and 64-bit PowerPC Red Hat Enterprise Linux 6.
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-09-20 12:14:52 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
output.log of storage test on RHEL6 s390x
none
output.log of storage test on RHEL6 ppc64
none
storage test patch changing failures on mkswap to warnings
none
storage test log on RHEL6 s390x with R20 none

Description qcui 2010-08-04 07:39:15 UTC
Created attachment 436458 [details]
output.log of storage test on RHEL6 s390x

Description of problem:
V7-1.2-14 storage test failed to run with 'Error:"dasdd is currently in use by LVM.It cannot be tested. You may need to reinstall.' on both s390x RHEL6 and ppc64 RHEL6. But it run successfully on s390x RHEL5.5 and ppc64 RHEL5.5.


Version-Release number of selected component (if applicable):
[root@ibm-z10-14 cpuscaling]# uname -a
Linux ibm-z10-14.rhts.eng.bos.redhat.com 2.6.32-52.el6.s390x #1 SMP Wed Jul 21 11:39:55 EDT 2010 s390x s390x s390x GNU/Linux
[root@ibm-z10-14 cpuscaling]# v7 version
V7 version 1.2, release 14


How reproducible:
Always

Steps to Reproduce:
1.Install v7-1.2-14.
2.# v7 run --test storage.

  
Actual results:
Fail

Expected results:
Pass

Comment 1 qcui 2010-08-04 07:46:17 UTC
Created attachment 436460 [details]
output.log of storage test on RHEL6 ppc64

Comment 2 Rob Landry 2010-08-12 17:05:00 UTC
This means the installation was not configured properly as the drive  was already in use by LVM and thus couldn't be used for the destructive testing.

https://fedorahosted.org/v7/wiki/HwCertHowToTestProcedures#storage

Comment 3 qcui 2010-08-13 10:04:16 UTC
But the installation was finished automatically through the Beaker. And the storage test on RHEL5.5 on the same machine is successful. So the failure on RHEL6 is really due to the installation was not configured properly?

Comment 5 Rob Landry 2010-08-17 18:33:23 UTC
Technically yes, but there is also a bug and probably a design change to consider.

Swap is used in RHEL5 because swap must be a physical block device and the storage test needs to write to the physical block device, since an lvm can be scattered across drives you could end up never testing the drive in question.

In RHEL6 swap can be on lvm which is where the problem comes in.  In RHEL6 swap is on lvm by default; when the storage test runs it decides swap is it's best bet to access a physical drive so it tries only to figure out later that it's on lvm and that it can't be used.

The bug is that it leaves this swap partition offline resulting in a system now being configured to fail the memory test.  The solution is the memory test should 1st check to verify the swap partition it would like to use is not part of an lvm set.

The partition layout is in the user control so this could all be avoided as it was with RHEL5 by a creating a conforming layout that provided either blank space or a swap partition on the physical drive block device.

Obviously this creates a situation where we probably want to think about a design change for the storage test because it will be painful if the default behavior and valid configuration of RHEL6 is not certifiable.  Depending on the decisions of that discussion we may need to provide vendors with a work around or introduce some new code to address.

Comment 6 Greg Nichols 2010-08-19 17:40:24 UTC
The reason swap doesn't get restored is that when the test calls mkswap (line 210, storage.py), mkswap has output on standard error:

# mkswap /dev/dm-1
mkswap: /dev/dm-1: warning: don't erase bootbits sectors
        on whole disk. Use -f to force.
Setting up swapspace version 1, size = 10371068 KiB
no label, UUID=3e0210c4-6340-44d0-b081-8e2e783660f4

mkswap seems to succeed.  The problem is that a V7CommandException is raised due to output on stderr, so the next call to swapon (line 211) does not happen.

This warning seems spurious to me.   I'd rather we didn't pass in -f, so instead we could print stderr as a warning for this command.   Also, exception handling needs to change so that swapon is called in all situations.

Comment 7 Greg Nichols 2010-08-19 17:41:54 UTC
This bug is independant of arch - changing summary accordingly.

Comment 8 Greg Nichols 2010-08-19 20:19:40 UTC
Created attachment 439777 [details]
storage test patch changing failures on mkswap to warnings

Comment 11 qcui 2010-08-24 11:14:22 UTC
Re-run storage test on server s390x with RHEL6.0-20100822.n.0 and R20. It still failed with error "dasdd is currently in use by LVM.It cannot be tested. You may need to reinstall." The log shows that "file descriptor leaked on lvs invocation". It did not appear before.

Comment 12 qcui 2010-08-24 11:16:38 UTC
Created attachment 440625 [details]
storage test log on RHEL6 s390x with R20

Comment 13 Greg Nichols 2010-08-24 14:01:05 UTC
(In reply to comment #11)
> Re-run storage test on server s390x with RHEL6.0-20100822.n.0 and R20. It still
> failed with error "dasdd is currently in use by LVM.It cannot be tested. You
> may need to reinstall." The log shows that "file descriptor leaked on lvs
> invocation". It did not appear before.

The "in use ny LVM" error is a test set-up/configuration issue.  This bug is where the storage test tries to use an lvm swap partition, then fails, and also fails to restore swap, causing other failures in the memory and core test.

I'm moving this back to ON_QA - please verify the lvm/swap issue is fixed.

Comment 14 Caspar Zhang 2010-08-26 06:19:53 UTC
I've tested in i386/x86_64, ppc64 with r20 and r21, lvm swap partition can be used for testing and also can be restored. 

Seem that the swap partition can't be enabled nor reinitialize via `mkswap' in Beaker so that the problem in #comment 11 appears. This should be a different problem. Talked to qcui and we decide to move this bug to VERIFIED

Comment 17 errata-xmlrpc 2010-09-20 12:14:52 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHBA-2010-0702.html

Comment 18 Jaromir Hradilek 2010-09-21 09:44:22 UTC
    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
The V7-1.2-14 storage test no longer fails on IBM System z Red Hat Enterprise Linux 6 and 64-bit PowerPC Red Hat Enterprise Linux 6.