It is not a regression bug, as the issue also can be reproduced with with qemu-kvm-rhev-2.5.0-3.el7.x86_64. So clear the version of qemu-kvm in the summary.

Please provide

Description of problem:
test cases 093 118 136 of qemu-iotests fail for raw with qemu-kvm-rhev-2.5.0-4.el7.x86_64

Version-Release number of selected component (if applicable):
[root@hp-dl385g7-07 qemu-iotests]# uname -r
3.10.0-394.el7.x86_64
[root@hp-dl385g7-07 qemu-iotests]# rpm -qa | grep qemu-kvm
qemu-kvm-rhev-2.5.0-4.el7.x86_64
[root@hp-dl385g7-07 qemu-iotests]# git branch -v
* (detached from qemu-kvm-rhev-2.5.0-4.el7) 7a90a72 Update to qemu-kvm-rhev-2.5.0-4.el7
  rhel7/master-1.5.3                        141e874 Update to qemu-kvm-1.5.3-111.el7

How reproducible:
2/2

Steps to Reproduce:
1.#git clone git://git.app.eng.bos.redhat.com/virt/rhel7/qemu-kvm.git
2.#git checkout qemu-kvm-rhev-2.5.0-4.el7
3.#./configure
4.#make
5.#cd tests/qemu-iotests
6.#export QEMU_PROG=/usr/libexec/qemu-kvm
7.#./check -raw

Actual results:
Not run: 007 013 014 015 017 018 019 020 022 023 024 026 027 028 029 030 031 034 035 036 037 038 039 040 041 042 043 044 046 047 049 050 051 053 054 056 057 058 059 060 061 062 064 065 066 067 068 069 070 072 073 074 075 076 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 094 095 096 097 098 102 103 105 107 108 110 111 112 113 114 115 116 119 121 122 123 124 129 130 131 134 135 137 138 139
Failures: 093 118 136
Failed 3 of 30 tests

Expected results:
All case passed except not run

Additional info:
Error log is same as the log of cases 041, 056 and 118 tested for qcow2.

Created attachment 1155956 [details]
log for raw qemu-iotests

At least most of these seem to be false positives. We may need to disable some
test cases and adjust some others to the real output. We got (for qcow2):

041: Fails because quorum is not whitelisted.
     We can fix TestRepairQuorum.has_quorum() to disable that part of the test.

051: There doesn't seem to be a valid stdin file descriptor. How does QE
     invoke qemu-iotests, and is this expected? We can either ignore the error
     in qemu-iotests or you can fix your invocation so that stdin is open.

056: This one is unclear. Does it reliably fail? It works for me locally.

118: null-co is not whitelisted. We can either whitelist it or disable 118.
     I'm leaning towards putting it on the whitelist.

138: tcmalloc produces a message on stderr before failing to allocate memory.
     We need to filter that message out.

139: blkverify and quorum are not whitelisted. Can do the same as for 041.


raw adds the following failing test case:

093: Like 056, unclear, produces QMPConnectError. Does it fail reproducibly?

136: Same.

Both work for me locally.

(In reply to Kevin Wolf from comment #6)
> At least most of these seem to be false positives. We may need to disable
> some
> test cases and adjust some others to the real output. We got (for qcow2):
> 
> 041: Fails because quorum is not whitelisted.
>      We can fix TestRepairQuorum.has_quorum() to disable that part of the
> test.
> 
> 051: There doesn't seem to be a valid stdin file descriptor. How does QE
>      invoke qemu-iotests, and is this expected? We can either ignore the
> error
>      in qemu-iotests or you can fix your invocation so that stdin is open.
> 
> 056: This one is unclear. Does it reliably fail? It works for me locally.
> 
> 118: null-co is not whitelisted. We can either whitelist it or disable 118.
>      I'm leaning towards putting it on the whitelist.
> 
> 138: tcmalloc produces a message on stderr before failing to allocate memory.
>      We need to filter that message out.
> 
> 139: blkverify and quorum are not whitelisted. Can do the same as for 041.
> 
> 
> raw adds the following failing test case:
> 
> 093: Like 056, unclear, produces QMPConnectError. Does it fail reproducibly?
> 
> 136: Same.
> 
> Both work for me locally.

Hi Kevin,
Below is my test steps:
For qcow2:
#git clone git://git.app.eng.bos.redhat.com/virt/rhel7/qemu-kvm.git
#git checkout qemu-kvm-rhev-2.5.0-4.el7
#yum install zlib* glib* -y
#git submodule update --init pixman
#git submodule update --init dtc
#./configure
#cd tests/qemu-iotests
#export QEMU_PROG=/usr/libexec/qemu-kvm
#./check -qcow2

For raw:
#git clone git://git.app.eng.bos.redhat.com/virt/rhel7/qemu-kvm.git
#git checkout qemu-kvm-rhev-2.5.0-4.el7
#yum install zlib* glib* -y
#git submodule update --init pixman
#git submodule update --init dtc
#./configure
#make tests/qemu-iotests/socket_scm_helper
#cd tests/qemu-iotests
#export QEMU_PROG=/usr/libexec/qemu-kvm
#./check -qcow2


051:
Could you share me how to fix my invocation so that stdin is open?

056, 093 and 136:
Both the case can be reliably reproduced from my side.
[root@hp-dl385g7-07 qemu-iotests]# uname -r
3.10.0-394.el7.x86_64
[root@hp-dl385g7-07 qemu-iotests]# rpm -qa | grep qemu-kvm
qemu-kvm-rhev-2.5.0-4.el7.x86_64

missing failed case for raw qemu-iotests without make source code: 081
081         - output mismatch (see 081.out.bad)
--- /home/test-raw-05092016/qemu-kvm/tests/qemu-iotests/081.out 2016-05-11 10:37:23.285776668 -0400
+++ 081.out.bad 2016-05-11 10:50:25.758239961 -0400
@@ -30,11 +30,8 @@
 QMP_VERSION
 {"return": {}}
 {"return": {}}
-{"return": {}}
-{"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "QUORUM_REPORT_BAD", "data": {"node-name": "drive2", "sectors-count": 20480, "sector-num": 0}}
-read 10485760/10485760 bytes at offset 0
-10 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-{"return": ""}
+{"error": {"class": "GenericError", "desc": "Driver 'quorum' is not whitelisted"}}
+{"return": "Device 'drive0-quorum' not found\r\n"}
 {"return": {}}
 {"timestamp": {"seconds":  TIMESTAMP, "microseconds":  TIMESTAMP}, "event": "SHUTDOWN"}

Created attachment 1156190 [details]
log for raw qemu-iotests without make source code

Created attachment 1156495 [details]
log for vmdk qemu-iotests

Description of problem:
test cases 059, 118 and 120 of qemu-iotests failed for vmdk.
case 059 failed due to the same reason as case 138.
case 120 failed due to vmdk can only be used for read-only devices. 

Version-Release number of selected component (if applicable):
[root@hp-dl385g7-07 qemu-iotests]# uname -r
3.10.0-394.el7.x86_64
[root@hp-dl385g7-07 qemu-iotests]# rpm -qa | grep qemu-kvm
qemu-kvm-rhev-2.5.0-4.el7.x86_64
[root@hp-dl385g7-07 qemu-iotests]# git branch -v
* (detached from qemu-kvm-rhev-2.5.0-4.el7) 7a90a72 Update to qemu-kvm-rhev-2.5.0-4.el7
  rhel7/master-1.5.3                        141e874 Update to qemu-kvm-1.5.3-111.el7

How reproducible:
2/2

Steps to Reproduce:
1.#git clone git://git.app.eng.bos.redhat.com/virt/rhel7/qemu-kvm.git
2.#git checkout qemu-kvm-rhev-2.5.0-4.el7
3.#yum install zlib* glib* -y
4.#git submodule update --init pixman dtc
5.#./configure
6.#cd tests/qemu-iotests
7.#export QEMU_PROG=/usr/libexec/qemu-kvm
8.#./check -vmdk

Actual results:
Not run: 007 013 014 015 022 023 024 025 026 028 029 030 031 035 036 038 039 040 041 042 043 044 045 046 047 048 049 050 051 053 054 055 056 057 058 060 061 062 064 065 066 067 068 070 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 101 102 103 107 108 109 112 113 114 115 116 119 121 122 123 124 128 129 130 131 132 134 135 136 137 138 139
Failures: 059 118 120
Failed 3 of 32 tests

Expected results:
All case passed except not run

Created attachment 1156522 [details]
log for vhdx qemu-iotests

Description of problem:
test cases 118 and 120 of qemu-iotests failed for vmdk.
case 059 failed due to the same reason as case 138.
case 120 failed due to vhdx can only be used for read-only devices. 

Version-Release number of selected component (if applicable):
[root@hp-dl385g7-07 qemu-iotests]# uname -r
3.10.0-394.el7.x86_64
[root@hp-dl385g7-07 qemu-iotests]# rpm -qa | grep qemu-kvm
qemu-kvm-rhev-2.5.0-4.el7.x86_64
[root@hp-dl385g7-07 qemu-iotests]# git branch -v
* (detached from qemu-kvm-rhev-2.5.0-4.el7) 7a90a72 Update to qemu-kvm-rhev-2.5.0-4.el7
  rhel7/master-1.5.3                        141e874 Update to qemu-kvm-1.5.3-111.el7

How reproducible:
1/1

Steps to Reproduce:
1.#git clone git://git.app.eng.bos.redhat.com/virt/rhel7/qemu-kvm.git
2.#git checkout qemu-kvm-rhev-2.5.0-4.el7
3.#yum install zlib* glib* -y
4.#git submodule update --init pixman dtc
5.#./configure
6.#cd tests/qemu-iotests
7.#export QEMU_PROG=/usr/libexec/qemu-kvm
8.#./check -vhdx

Actual results:
Not run: 007 013 014 015 017 018 019 020 022 023 024 025 026 027 028 029 030 031 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051 053 054 055 056 057 058 059 060 061 062 063 065 066 067 068 069 073 074 075 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094 095 096 097 098 101 102 103 107 108 109 110 111 112 113 114 115 116 119 121 122 123 124 128 129 130 131 132 134 135 136 137 138 139
Failures: 118 120
Failed 2 of 22 tests

Expected results:
All case passed except not run

(In reply to pingl from comment #10)
> Created attachment 1156522 [details]
> log for vhdx qemu-iotests
> 
> Description of problem:
> test cases 118 and 120 of qemu-iotests failed for vmdk.
> case 059 failed due to the same reason as case 138.
> case 120 failed due to vhdx can only be used for read-only devices. 
> 
> Version-Release number of selected component (if applicable):
> [root@hp-dl385g7-07 qemu-iotests]# uname -r
> 3.10.0-394.el7.x86_64
> [root@hp-dl385g7-07 qemu-iotests]# rpm -qa | grep qemu-kvm
> qemu-kvm-rhev-2.5.0-4.el7.x86_64
> [root@hp-dl385g7-07 qemu-iotests]# git branch -v
> * (detached from qemu-kvm-rhev-2.5.0-4.el7) 7a90a72 Update to
> qemu-kvm-rhev-2.5.0-4.el7
>   rhel7/master-1.5.3                        141e874 Update to
> qemu-kvm-1.5.3-111.el7
> 
> How reproducible:
> 1/1
> 
> Steps to Reproduce:
> 1.#git clone git://git.app.eng.bos.redhat.com/virt/rhel7/qemu-kvm.git
> 2.#git checkout qemu-kvm-rhev-2.5.0-4.el7
> 3.#yum install zlib* glib* -y
> 4.#git submodule update --init pixman dtc
> 5.#./configure
> 6.#cd tests/qemu-iotests
> 7.#export QEMU_PROG=/usr/libexec/qemu-kvm
> 8.#./check -vhdx
> 
> Actual results:
> Not run: 007 013 014 015 017 018 019 020 022 023 024 025 026 027 028 029 030
> 031 034 035 036 037 038 039 040 041 042 043 044 045 046 047 048 049 050 051
> 053 054 055 056 057 058 059 060 061 062 063 065 066 067 068 069 073 074 075
> 076 077 078 079 080 081 082 083 084 085 086 087 088 089 090 091 092 093 094
> 095 096 097 098 101 102 103 107 108 109 110 111 112 113 114 115 116 119 121
> 122 123 124 128 129 130 131 132 134 135 136 137 138 139
> Failures: 118 120
> Failed 2 of 22 tests
> 
> Expected results:
> All case passed except not run

Correct Description of problem:
test cases 118 and 120 of qemu-iotests failed for vhdx.
case 120 failed due to vhdx can only be used for read-only devices.

*** Bug 1427389 has been marked as a duplicate of this bug. ***

Created attachment 1276376 [details]
full log

Reproduce the issue with below packages.
qemu-kvm-rhev-2.9.0-2.el7
kernel-3.10.0-663.el7

Test steps:
1. #git clone git://git.app.eng.bos.redhat.com/virt/rhel7/qemu-kvm.git && cd qemu-kvm
2. #git checkout qemu-kvm-rhev-2.9.0-2.el7
3. #yum install -y glib2-devel pixman-devel
4. #git submodule update --init dtc
5. for qcow2/vmdk/vhdx:
    #./configure
   for raw:
    #./configure && make tests/qemu-iotests/socket_scm_helper
6. #export QEMU_PROG=/usr/libexec/qemu-kvm
7. #cd tests/qemu-iotests
8. #./check -$diskformat(qcow2/vhdx/vmdk/raw)

qcow2:
Failures: 030 041 051 055 056 118 138 139 142 147 155 172

raw:
Failures: 055 081 093 118 136 148

vmdk:
Failures: 059 118 120 147 157

vhdx:
Failures: 118 120 147 157

Created attachment 1404196 [details]
logs for qemu-iotests

Reproduced the issue with below packages:
kernel-3.10.0-858.el7.x86_64
qemu-kvm-rhev-2.10.0-21.el7

Test steps:
1. Check the installed qemu and get the source code
# brew download-build --rpm qemu-kvm-rhev-2.10.0-21.el7.src.rpm
# rpm -ivhf qemu-kvm-rhev-2.10.0-21.el7.src.rpm
# rpmbuild -bp /root/rpmbuild/SPECS/qemu-kvm.spec --nodeps

2. Generate common.env and export qemu-kvm to the environment
# cd /root/rpmbuild/BUILD/qemu-2.10.0
# ./configure
For raw format, below step must be run:
    #make tests/qemu-iotests/socket_scm_helper
# export QEMU_PROG=/usr/libexec/qemu-kvm

3. Run the tests
# cd tests/qemu-iotests
# ./check -qcow2(-raw/-vhdx/-vmdk)


Test results:
qcow2:
Failures: 030 041 051 055 118 138 139 142 147 153 172 186 195 198 200
Failed 15 of 147 tests

raw:
Failures: 055 081 093 118 136 148 195 198 200
Failed 9 of 53 tests

vhdx:
Failures: 118 120 147 157 192 194 195 198 200
Failed 9 of 35 tests

vmdk:
Failures: 059 118 120 147 157 192 194 195 198 200
Failed 10 of 46 tests

Created attachment 1471027 [details]
iotest log for qcow2, raw and luks

Retested packages:
qemu-kvm-rhev-2.12.0-8.el7
Kernel-3.10.0-926.el7

Test steps:
1. Check the installed qemu and get the source code
# brew download-build --rpm qemu-kvm-rhev-2.12.0-8.el7.src.rpm
# rpm -ivhf qemu-kvm-rhev-2.12.0-8.el7.src.rpm
# rpmbuild -bp /root/rpmbuild/SPECS/qemu-kvm.spec --nodeps
2. Generate common.env and export qemu-kvm to the environment
# cd /root/rpmbuild/BUILD/qemu-2.12.0
# ./configure
    For raw format, below step must be run:
        #make tests/qemu-iotests/socket_scm_helper
# export QEMU_PROG=/usr/libexec/qemu-kvm
# export QEMU_IMG_PROG=/usr/bin/qemu-img
# export QEMU_IO_PROG=/usr/bin/qemu-io
# export QEMU_NBD_PROG=/usr/bin/qemu-nbd
3. Run the test
# cd tests/qemu-iotests
# ./check -qcow2(-raw/-luks)


Test result:
qcow2
Failures: 030 041 051 055 118 138 139 172 186 214 215 217 223 226
Failed 14 of 168 tests

raw
Failures: 055 081 093 118 136 148 214 215 217 223 226
Failed 11 of 62 tests

luks
Failures: 149 214 215 217 223 226
Failed 6 of 35 tests

*** Bug 1623425 has been marked as a duplicate of this bug. ***

Our downstream CI team is now running qemu-iotests. Several of them are even enabled in %check as part of rpmbuild.

The ones that are broken in the CI or brew environment are being reported in bugzilla (there are several reported already).

So I'm closing all of the old qemu-iotest BZs. We'll work on them in this more controlled environment of CI and brew.