Bug 1209793

Summary: migration: 7.1->7.2 error while loading state for instance 0x0 of device '0000:00:04.0/intel-hda'
Product: Red Hat Enterprise Linux 7 Reporter: Dr. David Alan Gilbert <dgilbert>
Component: qemu-kvm-rhevAssignee: Dr. David Alan Gilbert <dgilbert>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.2CC: dgilbert, huding, juzhang, mrezanin, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-rhev-2.3.0-8.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-12-04 16:33:57 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1191307    
Attachments:
Description Flags
xml file for failed vm none

Description Dr. David Alan Gilbert 2015-04-08 08:40:40 UTC
Created attachment 1012108 [details]
xml file for failed vm

Description of problem:
Failed migration 7.1->7.2

Version-Release number of selected component (if applicable):
source: qemu-kvm-rhev-2.1.2-23.el7_1.1.x86_64
dest: qemu-kvm-rhev-2.3.0-0.el7.dgilbert20150407a.x86_64

How reproducible:
100%

Steps to Reproduce:
1. See xml
2.
3.

Actual results:

2015-04-08T08:34:41.543221Z qemu-kvm: error while loading state for instance 0x0 of device '0000:00:04.0/intel-hda'
2015-04-08T08:34:41.543411Z qemu-kvm: load of migration failed: Invalid argument

Expected results:
Working migration

Additional info:
/usr/libexec/qemu-kvm -name f20-414 -S -machine pc-i440fx-rhel7.1.0,accel=kvm,usb=off -cpu SandyBridge -m 2048 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid eb42c9ed-4c2b-496d-b415-e1e1cc2a917e -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/f20-414.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x5.0x7 -device ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 -device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x5.0x1 -device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x5.0x2 -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x6 -drive file=/home/vms/f20.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -netdev tap,fd=23,id=hostnet0,vhost=on,vhostfd=24 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:bc:51:1a,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/f20-414.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel1,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=com.redhat.spice.0 -device usb-tablet,id=input0 -spice port=5900,addr=127.0.0.1,disable-ticketing,seamless-migration=on -device qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=charredir0,name=usbredir -device usb-redir,chardev=charredir0,id=redir0 -chardev spicevmc,id=charredir1,name=usbredir -device usb-redir,chardev=charredir1,id=redir1 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on

Comment 2 huiqingding 2015-04-08 09:21:39 UTC
I test migration from rhel7.1->rhel7.2 with sound device, also hit this problem.

When the migration failed, I check "info qtree"
For qemu-kvm-rhev-2.1.2-23.el7 on src host:
(qemu) info qtree
    bus: pci.0
      type PCI
      dev: intel-hda, id "sound0"
        debug = 0 (0x0)
        msi = 1 (0x1)
        addr = 04.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class Audio controller, addr 00:04.0, pci id 8086:2668 (sub 1af4:1100)
        bar 0: mem at 0xfebf0000 [0xfebf3fff]
        bus: sound0.0
          type HDA
          dev: hda-duplex, id "sound0-codec0"
            debug = 0 (0x0)
            mixer = true
            cad = 0 (0x0)

For qemu-kvm-rhev-2.2.0-8.el7 on dst host:
(qemu) info qtree
    bus: pci.0
      type PCI
      dev: intel-hda, id "sound0"
        debug = 0 (0x0)
        msi = 1 (0x1)
        old_msi_addr = false
        addr = 04.0
        romfile = ""
        rombar = 1 (0x1)
        multifunction = false
        command_serr_enable = true
        class Audio controller, addr 00:04.0, pci id 8086:2668 (sub 1af4:1100)
        bar 0: mem at 0xfebf0000 [0xfebf3fff]
        bus: sound0.0
          type HDA
          dev: hda-duplex, id "sound0-codec0"
            debug = 0 (0x0)
            mixer = true
            cad = 0 (0x0)

Add "old_msi_addr=true" to the dst commandline as following, the migration can be finished normally:
-device intel-hda,id=sound0,bus=pci.0,addr=0x4, old_msi_addr=true

Comment 3 Dr. David Alan Gilbert 2015-04-08 09:27:22 UTC
Ah thank you; yes that should be easy to fix then; we just need to tie that to a new machine type.

Comment 4 Dr. David Alan Gilbert 2015-05-05 16:22:27 UTC
qemu-kvm-rhev-2.3.0-1.el7 should fix this using the new machine type from bz 1210050.

Comment 5 Miroslav Rezanina 2015-06-18 07:54:54 UTC
Fixed by rebase...

Comment 7 huiqingding 2015-07-01 08:38:45 UTC
I do migration RHEL7.1 -> RHEL7.2 with the device ich9-intel-hda, also hit this bug.

RHEL7.1 host:
kernel-3.10.0-229.7.2.el7.x86_64
qemu-kvm-rhev-2.1.2-23.el7_1.4.x86_64

RHEL7.2 host:
kernel-3.10.0-287.el7.x86_64
qemu-kvm-rhev-2.3.0-6.el7.x86_64

Command line:
/usr/libexec/qemu-kvm -cpu Opteron_G2 -name test3 -S -machine pc-i440fx-rhel7.1.0,accel=kvm,usb=off -m 1000 -realtime mlock=off -smp 2,maxcpus=4,sockets=4,cores=1,threads=1 -uuid 7347d748-f7ce-448f-8d49-3d29c9bcac30 -no-user-config -nodefaults  -rtc base=utc -no-shutdown -boot strict=on  -drive file=/mnt/rhel7.1.qcow2,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1  -spice port=5901,addr=0.0.0.0,disable-ticketing,seamless-migration=on -k en-us -device qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -msg timestamp=on -monitor stdio -device ich9-intel-hda,id=sound3,bus=pci.0,addr=0x6 -device hda-duplex,id=sound3-codec0,bus=sound3.0,cad=0

After migration from RHEL7.1 to RHEL7.2, the dst qemu-kvm quits with error info:
2015-07-01T08:36:13.194995Z qemu-kvm: error while loading state for instance 0x0 of device '0000:00:06.0/intel-hda'
2015-07-01T08:36:13.195181Z qemu-kvm: load of migration failed: Invalid argument

Hi, Dave,
Should we modify the status of this bug to "ASSIGNED" or open a new bug?

Best regards,
Huiqing

Comment 8 Dr. David Alan Gilbert 2015-07-01 08:46:00 UTC
(In reply to huiqingding from comment #7)
> I do migration RHEL7.1 -> RHEL7.2 with the device ich9-intel-hda, also hit
> this bug.
> 
> RHEL7.1 host:
> kernel-3.10.0-229.7.2.el7.x86_64
> qemu-kvm-rhev-2.1.2-23.el7_1.4.x86_64
> 
> RHEL7.2 host:
> kernel-3.10.0-287.el7.x86_64
> qemu-kvm-rhev-2.3.0-6.el7.x86_64
> 
> Command line:
> /usr/libexec/qemu-kvm -cpu Opteron_G2 -name test3 -S -machine
> pc-i440fx-rhel7.1.0,accel=kvm,usb=off -m 1000 -realtime mlock=off -smp
> 2,maxcpus=4,sockets=4,cores=1,threads=1 -uuid
> 7347d748-f7ce-448f-8d49-3d29c9bcac30 -no-user-config -nodefaults  -rtc
> base=utc -no-shutdown -boot strict=on  -drive
> file=/mnt/rhel7.1.qcow2,if=none,id=drive-ide0-0-0,format=qcow2 -device
> ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1  -spice
> port=5901,addr=0.0.0.0,disable-ticketing,seamless-migration=on -k en-us
> -device
> qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.
> 0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -msg
> timestamp=on -monitor stdio -device
> ich9-intel-hda,id=sound3,bus=pci.0,addr=0x6 -device
> hda-duplex,id=sound3-codec0,bus=sound3.0,cad=0
> 
> After migration from RHEL7.1 to RHEL7.2, the dst qemu-kvm quits with error
> info:
> 2015-07-01T08:36:13.194995Z qemu-kvm: error while loading state for instance
> 0x0 of device '0000:00:06.0/intel-hda'
> 2015-07-01T08:36:13.195181Z qemu-kvm: load of migration failed: Invalid
> argument
> 
> Hi, Dave,
> Should we modify the status of this bug to "ASSIGNED" or open a new bug?

Put it back to ASSIGNED; it looks like I need to fix that.

Dave

> Best regards,
> Huiqing

Comment 9 Dr. David Alan Gilbert 2015-07-02 10:00:16 UTC
Confirmed still broken; simplest case is :

/usr/libexec/qemu-kvm -nographic -machine pc-i440fx-rhel7.1.0,accel=kvm -device ich9-intel-hda,id=sound3,bus=pci.0,addr=0x6

Comment 11 Miroslav Rezanina 2015-07-08 10:54:00 UTC
Fix included in qemu-kvm-rhev-2.3.0-8.el7

Comment 13 huiqingding 2015-07-21 07:40:28 UTC
Reproduce this bug using the following version:

RHEL7.1 host:
kernel-3.10.0-229.7.2.el7.x86_64
qemu-kvm-rhev-2.1.2-23.el7_1.4.x86_64

RHEL7.2 host:
kernel-3.10.0-297.el7.x86_64
qemu-kvm-rhev-2.3.0-5.el7.x86_64

Command line:
/usr/libexec/qemu-kvm -cpu Penryn -name test3 -S -machine pc-i440fx-rhel7.1.0,accel=kvm,usb=off -m 1000 -realtime mlock=off -smp 2,maxcpus=4,sockets=4,cores=1,threads=1 -uuid 7347d748-f7ce-448f-8d49-3d29c9bcac30 -no-user-config -nodefaults  -rtc base=utc -no-shutdown -boot strict=on  -drive file=/mnt/rhel7.1.qcow2,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1  -spice port=5901,addr=0.0.0.0,disable-ticketing,seamless-migration=on -k en-us -device qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -msg timestamp=on -monitor stdio -device intel-hda,id=sound3,bus=pci.0,addr=0x6 -device hda-duplex,id=sound3-codec0,bus=sound3.0,cad=0 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,mac=52:54:00:c6:4b:15,id=test,addr=0xa

After migration from RHEL7.1 to RHEL7.2, the dst qemu-kvm quits with error info:
qemu: warning: error while loading state for instance 0x0 of device '0000:00:06.0/intel-hda'
2015-07-21T07:32:02.350250Z qemu-kvm: load of migration failed: Invalid argument

Comment 14 huiqingding 2015-07-21 07:50:29 UTC
Test this bug using the following version:

RHEL7.1 host:
kernel-3.10.0-229.7.2.el7.x86_64
qemu-kvm-rhev-2.1.2-23.el7_1.4.x86_64

RHEL7.2 host:
kernel-3.10.0-297.el7.x86_64
qemu-kvm-rhev-2.3.0-12.el7.x86_64

Command line:
/usr/libexec/qemu-kvm -cpu Penryn -name test3 -S -machine pc-i440fx-rhel7.1.0,accel=kvm,usb=off -m 1000 -realtime mlock=off -smp 2,maxcpus=4,sockets=4,cores=1,threads=1 -uuid 7347d748-f7ce-448f-8d49-3d29c9bcac30 -no-user-config -nodefaults  -rtc base=utc -no-shutdown -boot strict=on  -drive file=/mnt/rhel7.1.qcow2,if=none,id=drive-ide0-0-0,format=qcow2 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1  -spice port=5901,addr=0.0.0.0,disable-ticketing,seamless-migration=on -k en-us -device qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3 -msg timestamp=on -monitor stdio -device intel-hda,id=sound3,bus=pci.0,addr=0x6 -device hda-duplex,id=sound3-codec0,bus=sound3.0,cad=0 -netdev tap,id=hostnet0,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=hostnet0,mac=52:54:00:c6:4b:15,id=test,addr=0xa

After migration from RHEL7.1 to RHEL7.2, the migration can be finished normally. Inside guest, can play audio normally.

I also test -device ich9-intel-hda, the result is pass, same as the above.

Comment 15 huiqingding 2015-07-21 08:19:02 UTC
Update RHEL7.1.z to the latest version:
kernel-3.10.0-229.10.1.el7.x86_64
qemu-kvm-rhev-2.1.2-23.el7_1.6.x86_64

RHEL7.2 host uses the same version and do same test as comment #14, the result is pass. After migration from RHEL7.1 to RHEL7.2, the migration can be finished normally. Inside guest, can play audio normally.

Comment 16 juzhang 2015-07-27 04:39:24 UTC
According to comment13 to comment15, set this issue as verified.

Comment 18 errata-xmlrpc 2015-12-04 16:33:57 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2546.html