Bug 593287

Summary: Failed asserting during ide_dma_cancel
Product: Red Hat Enterprise Linux 6 Reporter: Kevin Wolf <kwolf>
Component: qemu-kvmAssignee: Kevin Wolf <kwolf>
Status: CLOSED CURRENTRELEASE QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: low    
Version: 6.0CC: bcao, juzhang, mjenner, virt-maint
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: qemu-kvm-0.12.1.2-2.59.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-07-26 10:06:08 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Kevin Wolf 2010-05-18 12:44:33 UTC
This problem was reported upstream as follows:

> i'm using qemu-kvm 0.12.3 with latest Kernel 2.6.33.3.
> As backend we use open-iSCSI with dm-multipath.
>
> Multipath is configured to queue i/o if no path is available.
>
> If we create a failure on all paths, qemu starts to consume 100%
> CPU due to i/o waits which is ok so far.
> [...]
> What es a really blocker is that KVM crashes with:
> kvm: /usr/src/qemu-kvm-0.12.3/hw/ide/internal.h:507: bmdma_active_if: 
> Assertion `bmdma->unit != (uint8_t)-1' failed.
>
> after the multipath has reestablisched at least one path.

This crash is only expected to happen with the combination of IDE and Linux AIO.

Comment 1 RHEL Program Management 2010-05-18 13:15:30 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 6 juzhang 2010-07-22 10:00:28 UTC
I both tested on qemu-kvm-0.12.1.2-2.53.el6(unfixed version) and  qemu-kvm-0.12.1.2-2.96.el6(fixed version),both didn't hit vm crash.

Fixed version compared to unfixed version.the only difference is "the unfixed version qemu-kvm hangs with 100% cpu in the process of failover(I created two math).the fixed version qemu-kvm is normal.

I use open-iSCSI with dm-multipath reproduce this issue,the following is steps in details.If any mistake,please correct me.


 
the following is tested steps
1. created iscsi server(have at least two NICs in this box,)
1.1 #tgtadm --lld iscsi --op new --mode target --tid=1 --targetname iqn.2001-04.com.example:storage.disk1.amiens.sys1.multipath

1.2 #tgtadm --lld iscsi --op new --mode logicalunit --tid 1 --lun 1 -b /dev/sda4
tgtadm --lld iscsi --op bind --mode target --tid 1 -I ALL
1.3 #chkconfig tgtd on

2. In host
2.1# service iscsi restart
2.2#iscsiadm -m discovery -t sendtargets -p 10.66.72.184
2.3#iscsiadm -m node -T iscsiadm -m node -T iqn.2001-04.com.example:storage.disk1.amiens.sys1.zhangjunyi -p 10.66.72.184 -l
2.4#iscsiadm -m discovery -t sendtargets -p 10.66.72.174
2.5#iscsiadm -m node -T iscsiadm -m node -T iqn.2001-04.com.example:storage.disk1.amiens.sys1.zhangjunyi -p 10.66.72.174 -l
2.6#multipath -ll
mpathb (1IET     00010001) dm-3 IET,VIRTUAL-DISK
size=40G features='0' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=0 status=enabled
| `- 9:0:0:1  sdb 8:16 failed faulty running
`-+- policy='round-robin 0' prio=1 status=active
  `- 10:0:0:1 sdc 8:32 active ready  running
2.7 #cd /dev/mapper
2.8 #pvcreate mpathb
2.9 #vgcreate volgroup0 mpathb
2.10 #lvcreate -n mpiscsi1 -L +30G volgroup0
2.11 #mkfs.ext3 volgroup0-mpiscsi1
2.12 #mount VolGroup00-mpiscsi1 /mnt
2.13 cp a raw img to /mnt
2.14 boot linux guest with ide interface
#/usr/libexec/qemu-kvm -m 4G -smp 4 -drive file=/mnt/RHEL-Server-5.5-64.raw,if=none,id=test,boot=on,cache=none,format=raw -device ide-drive,drive=test -cpu qemu64 -monitor stdio -boot order=cdn,menu=on -netdev tap,id=hostnet0,vhost=on -device rtl8139,netdev=hostnet0,id=net0,mac=22:11:22:45:66:93 -vnc :9 -qmp tcp:0:4445,server,nowait
2.15.cutoff one path
2.16.in guest,run dd command inoder to generate io.
2.17. cutoff the other path,after about 1/2 mitues
2.18.restore  one path

Results
After step2.17,qemu-kvm hangs with 100% cpu.also monitor is not responding.
After step2.18,the vm is continueing.

Comment 7 Kevin Wolf 2010-07-23 09:56:53 UTC
According to the original description you need to use Linux AIO to reproduce the crash, i.e. -drive ...,aio=native

Comment 8 juzhang 2010-07-26 10:05:05 UTC
(In reply to comment #7)
> According to the original description you need to use Linux AIO to reproduce
> the crash, i.e. -drive ...,aio=native    

Using the commands 6 step6.the only difference is add "aio=native"

Reproduced on qemu-kvm-0.12.1.2-2.53.el6

Command Line:

#/usr/libexec/qemu-kvm -m 4G -smp 4 -drive aio=native,file=/mnt/rhel5.5_64_ide.raw,if=none,id=test,boot=on,cache=none,format=raw -device ide-drive,drive=test -cpu qemu64 -monitor stdio -boot order=cdn,menu=on -netdev tap,id=hostnet0,vhost=on -device rtl8139,netdev=hostnet0,id=net0,mac=22:11:22:45:66:93 -vnc :9 -qmp tcp:0:4445,server,nowait
QEMU 0.12.1 monitor - type 'help' for more information
(qemu) qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/hw/ide/internal.h:509: bmdma_active_if: Assertion `bmdma->unit != (uint8_t)-1' failed.
Aborted (core dumped)

Results:
bmdma_active_if: Assertion `bmdma->unit != (uint8_t)-1' failed.
Aborted (core dumped)

Verified on qemu-kvm-0.12.1.2-2.99.el6,passed.

Comment 9 juzhang 2010-07-26 10:06:08 UTC
according to comment8,close this issue.