Bug 1174920 - 7.2->earlier compatibility issues
Summary: 7.2->earlier compatibility issues
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: qemu-kvm-rhev
Version: 7.2
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: rc
: ---
Assignee: Dr. David Alan Gilbert
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On: 1209300 1209798 1210715 1215087 1215088 1215091 1215092 1242032
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-12-16 18:25 UTC by Dr. David Alan Gilbert
Modified: 2017-06-13 03:56 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-06-13 03:56:26 UTC
Target Upstream Version:


Attachments (Terms of Use)

Description Dr. David Alan Gilbert 2014-12-16 18:25:59 UTC
This is a place holder to keep track of things we need to deal with for 7.2->earlier migration issues

Paolo's:
Serial thr_ipending
APIC wait_for_sipi

Comment 3 Amit Shah 2015-04-09 09:05:05 UTC
Things the static checker flagged that will be a problem for 7.2->7.1 migration:

$ ./scripts/vmstate-static-checker.py -s /var/tmp/rhev-7.1z.txt -d /var/tmp/rhev-7.2-rc2.txt --reverse
Section "ich9-ahci" Description "ide_bus/error": version error: 2 > 1
Section "ich9-ahci", Description "ide_bus/error": expected field "retry_sector_num", while dest has no further fields
Section "ich9-ahci", Description "ahci port": expected field "port.ifs[0]", got "port_state"; skipping rest
Section "sdhci-pci" does not exist in dest
Section "pci-serial-2x" Description "serial": Entry "Subsections" missing
Section "base-sysbus-fdc" Description "fdc": Entry "Subsections" missing
Section "base-sysbus-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "i8042" Description "pckbd": Entry "Subsections" missing
Section "isa-serial" Description "serial": Entry "Subsections" missing
Section "sysbus-ahci" Description "ide_bus/error": version error: 2 > 1
Section "sysbus-ahci", Description "ide_bus/error": expected field "retry_sector_num", while dest has no further fields
Section "sysbus-ahci", Description "ahci port": expected field "port.ifs[0]", got "port_state"; skipping rest
Section "generic-sdhci" does not exist in dest
Section "fw_cfg_io" does not exist in dest
Section "intel-iommu" does not exist in dest
Section "SUNW,fdtwo" Description "fdc": Entry "Subsections" missing
Section "SUNW,fdtwo", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "mc146818rtc" Description "mc146818rtc": Entry "Subsections" missing
Section "fw_cfg_mem" does not exist in dest
Section "pci-serial" Description "serial": Entry "Subsections" missing
Section "isa-fdc" Description "fdc": Entry "Subsections" missing
Section "isa-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "apic" Description "apic": Entry "Subsections" missing
Section "sysbus-fdc" Description "fdc": Entry "Subsections" missing
Section "sysbus-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "pci-serial-4x" Description "serial": Entry "Subsections" missing
Section "kvm-apic" Description "apic": Entry "Subsections" missing
Section "apic-common" Description "apic": Entry "Subsections" missing

Comment 4 Amit Shah 2015-04-09 09:07:11 UTC
From the list above, we can ignore:

ich9-ahci (ahci, migration was unsupported)
*sysbus*
SUNW*

Comment 5 Paolo Bonzini 2015-04-10 21:10:20 UTC
Serial should be okay now.  Failures to migrate due to subsections should really be cases where it's safer to abort migration rather than continue with invalid state on 7.1.

apic/kvm-apic/apic-common was tuned to never generate the subsection in practice for TCG, and never at all for KVM.

fw_cfg - Section "fw_cfg_io" does not exist in dest. The device type changed from fw_cfg to fw_cfg_io, but vmstate_fw_cfg still has .name="fw_cfg" in 7.2.  Should be good unless I'm missing something.

mc146818rtc can cause failures where it's overall fine to continue on the destination.  We can disable the subsection altogether (return 0 from rtc_irq_reinject_on_ack_count_needed) with small or no loss.  Same for floppy.  I would disable them for simplicity instead of wasting time debugging later.

pckbd was tuned to never generate the subsection in practice.  But we can disable the subsection altogether just to err on the side of safety.

Some other differences seem to be missing from the list:

- cpu_common/exception_index which is okay (only produced if tcg_enabled())

- ide_bus/error for the PIIX4 IDE - known breakage, but only if migrating a VM paused due to I/O error, where the IDE disks are the ones having the error state.  Fix hinted in commit message d12b9ff2a4e21587e5b180008c88a0e6e1ec8ac5, but I would just ignore it as it's really a corner case.

- parallel: we do not support parallel so it's okay to have it break 7.2->7.1 migration.

Comment 6 Dr. David Alan Gilbert 2015-04-13 10:00:29 UTC
(In reply to Paolo Bonzini from comment #5)
> Serial should be okay now.  Failures to migrate due to subsections should
> really be cases where it's safer to abort migration rather than continue
> with invalid state on 7.1.

I'm not sure about that.  I don't yet understand the detail of all of the subsections in  the serial code (hw/char/serial.c) but some seem to be as simple as the xmit/receive fifos.  That seems to say that even just outputting some text on a console and migrating at just the wrong point would trigger a failed migration, and that doesn't seem acceptable.

Dave

Comment 7 Paolo Bonzini 2015-04-13 15:53:58 UTC
The alternative would be data loss on the receiving side, which seems worse.  It can be documented in the release notes.

I'm more worried about why some differences are missing from the report, even though the ones I remembered are okay.

Comment 8 Amit Shah 2015-04-14 09:11:51 UTC
(In reply to Paolo Bonzini from comment #5)

> Some other differences seem to be missing from the list:
> 
> - cpu_common/exception_index which is okay (only produced if tcg_enabled())

cpu_common was not visited by the code which dumps the json; I'll have to check why that is.

> - ide_bus/error for the PIIX4 IDE - known breakage, but only if migrating a
> VM paused due to I/O error, where the IDE disks are the ones having the
> error state.  Fix hinted in commit message
> d12b9ff2a4e21587e5b180008c88a0e6e1ec8ac5, but I would just ignore it as it's
> really a corner case.

This has been flagged:

Section "ich9-ahci" Description "ide_bus/error": version error: 2 > 1
Section "ich9-ahci", Description "ide_bus/error": expected field "retry_sector_num", while dest has no further fields

> - parallel: we do not support parallel so it's okay to have it break
> 7.2->7.1 migration.

parallel didn't even get compiled.

Comment 9 Paolo Bonzini 2015-04-15 11:42:46 UTC
Thanks for the explanation.  Regarding ide_bus/error, I didn't understand that the subsection is flagged only once.  In this case we don't support ich9-ahci migration, but piix-ide also has the same subsection and we do support it.

Do you guys agree on comment 5 (apart from David's mentioned doubts regarding the serial port)?

Comment 10 Amit Shah 2015-04-15 12:25:43 UTC
Juan told me cpustate isn't registered via vmstate, but via the old method.  I'm guessing it's the same for ide.

The ide one didn't show up not because it's duplicated, btw.  e.g. sysbus has ide too, and that is reported.

My view on this is that we'll just have to audit each device specifically to figure whether it'll be safe to migrate by fudging the subsection's 'needed' state.

We could also fail migration once and report that the failure was due to a missing subsection, and that a subsequent migration may not have such a state.  However, that's cruel to everyone.

Failing migration if we could have let it go on without any side effects isn't also optimal -- but then the subsection would not have been introduced in the first place.

If we determine that a subsection won't be emitted at all for our use-cases (x86, kvm only), we can proceed, but I think we still keep ourselves open to the possibility of surprises where migration doesn't succeed once in a while due to the state in a new subsection.

Comment 11 Paolo Bonzini 2015-04-15 14:55:26 UTC
> My view on this is that we'll just have to audit each device specifically to
> figure whether it'll be safe to migrate by fudging the subsection's 'needed'
> state.

Note that there is no new device state.  This is all old device state that we happened not to migrate.

In all cases it's okay to drop the subsection altogether (.needed returns false).  We have never migrated the state and the sky won't fall down if we keep not migrating it.

In some cases, however (I think serial only) it may be better to have "surprises where migration doesn't succeed once in a while due to the state in a new subsection".  In fact, this was the reason why subsections were introduced in the first place, to have backwards migration fail only in very specific and hopefully rare cases:

    commit 5ee84c3375f9dff8fdee7209195c8c5327a92551
    Author: Juan Quintela <quintela@redhat.com>
    Date:   Mon Jul 26 21:38:45 2010 +0200

    ide: fix migration in the middle of a bmdma transfer
    
    It reintroduces
        Revert "ide save/restore pio/atapi cmd transfer fields and io buffer"
    
    but using subsections.  Added bonus is the addition of 
    ide_dummy_transfer_stop to transfer_end_table, that was missing.
    
    Signed-off-by: Juan Quintela <quintela@redhat.com>
    Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

    ---

    commit 50641c5ce258220fc63ba5be1afd1a37f023aba9
    Author: Juan Quintela <quintela@redhat.com>
    Date:   Mon Jul 26 21:38:44 2010 +0200

    ide: fix migration in the middle of pio operation
    
    Signed-off-by: Juan Quintela <quintela@redhat.com>
    Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

Comment 12 Paolo Bonzini 2015-04-15 14:57:13 UTC
BTW, I checked 2.1 ... 2.3 and didn't find anything else than the three items above.  Also, PIIX IDE indeed uses vmstate_register.

Comment 13 Dr. David Alan Gilbert 2015-04-16 19:02:33 UTC
(In reply to Paolo Bonzini from comment #7)
> The alternative would be data loss on the receiving side, which seems worse.
> It can be documented in the release notes.

But that receive side loss already happens when migrating at the moment?
If so then we just document that if you use the new machine type that serial is better protected against it.

Doesn't our emulated serial port already fail like a real serial port in that a slow guest will lose data if it doesn't pluck stuff off the fifo quickly enough?

And anyway, what device are you going to connect to your emulated serial port that's going to survive migration without being interrupted.

I'm happy that failing on a migration problem with block storage is the right thing to do; we should never return corrupt memory or corrupt data on storage;  serial I'm a bit less fussy about, since serial itself isn't exactly built for reliability.

Comment 14 Paolo Bonzini 2015-04-16 20:27:47 UTC
Fair enough!  Then we can tweak the .needed functions for serial as well, with a version comparison against 7.2.0.

Comment 15 Dr. David Alan Gilbert 2015-04-24 09:37:15 UTC
I've created bz's for each of the components described above we think are a problem:

     serial: bug 1215087
     mc146818rtc: 1215088
     floppy: 1215091
     pckbd: 1215092

Comment 16 Amit Shah 2015-05-13 10:30:18 UTC
Now with a brew build for 2.3.0-based qemu available, I re-ran the static checker against 7.0, 7.1 and 7.2.

These are the results for 7.0 <-> 7.2 checks.


qemu 1.5.3 -> 2.3.0 (rhel / rhev 7.0 -> rhev 7.2) with machine type 7.0

$ ~/src/qemu-kvm/scripts/vmstate-static-checker.py -s 1.5.3-86.el7/rhel70-70.json -d 2.3.0-1.el7/rhev72-70.json 

Section "ich9-ahci", Description "ahci port": expected field "port_state", got "port.ifs[0]"; skipping rest
Section "xlnx,ps7-usb" does not exist in dest
Section "sysbus-ahci", Description "ahci port": expected field "port_state", got "port.ifs[0]"; skipping rest
Section "sysbus-ehci-usb" does not exist in dest
Section "exynos4210-ehci-usb" does not exist in dest

--------------

qemu 2.3.0 -> 1.5.3 (rhev 7.2 -> rhel / rhev 7.0) with machine type 7.0

$ ~/src/qemu-kvm/scripts/vmstate-static-checker.py -s 1.5.3-86.el7/rhel70-70.json -d 2.3.0-1.el7/rhev72-70.json  --reverse

Section "ich9-ahci" Description "ide_bus/error": version error: 2 > 1
Section "ich9-ahci", Description "ide_bus/error": expected field "retry_sector_num", while dest has no further fields
Section "ich9-ahci", Description "ahci port": expected field "port.ifs[0]", got "port_state"; skipping rest
Section "e1000-82544gc" does not exist in dest
Section "secondary-vga" does not exist in dest
Section "sdhci-pci" does not exist in dest
Section "pci-serial-2x" Description "serial": Entry "Subsections" missing
Section "base-sysbus-fdc" does not exist in dest
Section "hda-audio" does not exist in dest
Section "qxl" Description "vga": Entry "Subsections" missing
Section "PIIX4_PM" Description "piix4_pm": Entry "Subsections" missing
Section "PIIX4_PM", Description "piix4_pm": expected field "acpi_pci_hotplug.hotplug_select", while dest has no further fields
Section "intel-hda-generic" does not exist in dest
Section "i8042" Description "pckbd": Entry "Subsections" missing
Section "isa-serial" Description "serial": Entry "Subsections" missing
Section "usb-redir", Description "usb-redir-ep": Subsection "usb-redir-ep/stream-state" not found
Section "sysbus-ahci" Description "ide_bus/error": version error: 2 > 1
Section "sysbus-ahci", Description "ide_bus/error": expected field "retry_sector_num", while dest has no further fields
Section "sysbus-ahci", Description "ahci port": expected field "port.ifs[0]", got "port_state"; skipping rest
Section "generic-sdhci" does not exist in dest
Section "fw_cfg_io" does not exist in dest
Section "VGA" Description "vga": Entry "Subsections" missing
Section "intel-iommu" does not exist in dest
Section "SUNW,fdtwo" Description "fdc": Entry "Subsections" missing
Section "SUNW,fdtwo", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "mc146818rtc" Description "mc146818rtc": Entry "Subsections" missing
Section "e1000-82545em" does not exist in dest
Section "e1000-82540em" does not exist in dest
Section "fw_cfg_mem" does not exist in dest
Section "pci-serial" Description "serial": Entry "Subsections" missing
Section "isa-fdc" Description "fdc": Entry "Subsections" missing
Section "isa-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "apic" Description "apic": Entry "Subsections" missing
Section "qxl-vga" Description "vga": Entry "Subsections" missing
Section "sysbus-fdc" Description "fdc": Entry "Subsections" missing
Section "sysbus-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "ICH9-LPC" Description "ich9_pm": Entry "Subsections" missing
Section "pci-serial-4x" Description "serial": Entry "Subsections" missing
Section "kvm-apic" Description "apic": Entry "Subsections" missing
Section "e1000" Description "e1000": Entry "Subsections" missing
Section "apic-common" Description "apic": Entry "Subsections" missing

Comment 17 Amit Shah 2015-05-13 10:31:04 UTC
Results of 7.1 <-> 7.2 checks with 7.0 machine type:


qemu 2.1.2 -> 2.3.0 (rhev 7.1 -> rhev 7.2) with machine type 7.0

$ ~/src/qemu-kvm/scripts/vmstate-static-checker.py -s 2.1.2-23.el7_1.3/rhev71-70.json -d 2.3.0-1.el7/rhev72-70.json 
Section "ich9-ahci", Description "ahci port": expected field "port_state", got "port.ifs[0]"; skipping rest
Section "xlnx,ps7-usb" does not exist in dest
Section "sysbus-ahci", Description "ahci port": expected field "port_state", got "port.ifs[0]"; skipping rest
Section "fusbh200-ehci-usb" does not exist in dest
Section "tegra2-ehci-usb" does not exist in dest
Section "sysbus-ehci-usb" does not exist in dest
Section "exynos4210-ehci-usb" does not exist in dest

--------------

qemu 2.3.0 -> 2.1.2 (rhev 7.2 -> rhev 7.1) with machine type 7.0

$ ~/src/qemu-kvm/scripts/vmstate-static-checker.py -s 2.1.2-23.el7_1.3/rhev71-70.json -d 2.3.0-1.el7/rhev72-70.json  --reverse

Section "ich9-ahci" Description "ide_bus/error": version error: 2 > 1
Section "ich9-ahci", Description "ide_bus/error": expected field "retry_sector_num", while dest has no further fields
Section "ich9-ahci", Description "ahci port": expected field "port.ifs[0]", got "port_state"; skipping rest
Section "sdhci-pci" does not exist in dest
Section "pci-serial-2x" Description "serial": Entry "Subsections" missing
Section "base-sysbus-fdc" Description "fdc": Entry "Subsections" missing
Section "base-sysbus-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "i8042" Description "pckbd": Entry "Subsections" missing
Section "isa-serial" Description "serial": Entry "Subsections" missing
Section "sysbus-ahci" Description "ide_bus/error": version error: 2 > 1
Section "sysbus-ahci", Description "ide_bus/error": expected field "retry_sector_num", while dest has no further fields
Section "sysbus-ahci", Description "ahci port": expected field "port.ifs[0]", got "port_state"; skipping rest
Section "generic-sdhci" does not exist in dest
Section "fw_cfg_io" does not exist in dest
Section "intel-iommu" does not exist in dest
Section "SUNW,fdtwo" Description "fdc": Entry "Subsections" missing
Section "SUNW,fdtwo", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "mc146818rtc" Description "mc146818rtc": Entry "Subsections" missing
Section "fw_cfg_mem" does not exist in dest
Section "pci-serial" Description "serial": Entry "Subsections" missing
Section "isa-fdc" Description "fdc": Entry "Subsections" missing
Section "isa-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "apic" Description "apic": Entry "Subsections" missing
Section "sysbus-fdc" Description "fdc": Entry "Subsections" missing
Section "sysbus-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "pci-serial-4x" Description "serial": Entry "Subsections" missing
Section "kvm-apic" Description "apic": Entry "Subsections" missing
Section "apic-common" Description "apic": Entry "Subsections" missing

--------------

Comment 18 Amit Shah 2015-05-13 10:31:31 UTC
Finally, results of 7.1 <-> 7.2 checks with 7.1 machine type:


qemu 2.1.2 -> 2.3.0 (rhev 7.1 -> rhev 7.2) with machine type 7.1

$ ~/src/qemu-kvm/scripts/vmstate-static-checker.py -s 2.1.2-23.el7_1.3/rhev71-71.json -d 2.3.0-1.el7/rhev72-71.json 

Section "ich9-ahci", Description "ahci port": expected field "port_state", got "port.ifs[0]"; skipping rest
Section "xlnx,ps7-usb" does not exist in dest
Section "sysbus-ahci", Description "ahci port": expected field "port_state", got "port.ifs[0]"; skipping rest
Section "fusbh200-ehci-usb" does not exist in dest
Section "tegra2-ehci-usb" does not exist in dest
Section "sysbus-ehci-usb" does not exist in dest
Section "exynos4210-ehci-usb" does not exist in dest

--------------

qemu 2.3.0 -> 2.1.2 (rhev 7.2 -> rhev 7.1) with machine type 7.1

$ ~/src/qemu-kvm/scripts/vmstate-static-checker.py -s 2.1.2-23.el7_1.3/rhev71-71.json -d 2.3.0-1.el7/rhev72-71.json  --reverse

Section "ich9-ahci" Description "ide_bus/error": version error: 2 > 1
Section "ich9-ahci", Description "ide_bus/error": expected field "retry_sector_num", while dest has no further fields
Section "ich9-ahci", Description "ahci port": expected field "port.ifs[0]", got "port_state"; skipping rest
Section "sdhci-pci" does not exist in dest
Section "pci-serial-2x" Description "serial": Entry "Subsections" missing
Section "base-sysbus-fdc" Description "fdc": Entry "Subsections" missing
Section "base-sysbus-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "i8042" Description "pckbd": Entry "Subsections" missing
Section "isa-serial" Description "serial": Entry "Subsections" missing
Section "sysbus-ahci" Description "ide_bus/error": version error: 2 > 1
Section "sysbus-ahci", Description "ide_bus/error": expected field "retry_sector_num", while dest has no further fields
Section "sysbus-ahci", Description "ahci port": expected field "port.ifs[0]", got "port_state"; skipping rest
Section "generic-sdhci" does not exist in dest
Section "fw_cfg_io" does not exist in dest
Section "intel-iommu" does not exist in dest
Section "SUNW,fdtwo" Description "fdc": Entry "Subsections" missing
Section "SUNW,fdtwo", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "mc146818rtc" Description "mc146818rtc": Entry "Subsections" missing
Section "fw_cfg_mem" does not exist in dest
Section "pci-serial" Description "serial": Entry "Subsections" missing
Section "isa-fdc" Description "fdc": Entry "Subsections" missing
Section "isa-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "apic" Description "apic": Entry "Subsections" missing
Section "sysbus-fdc" Description "fdc": Entry "Subsections" missing
Section "sysbus-fdc", Description "fdrive": Subsection "fdrive/perpendicular" not found
Section "pci-serial-4x" Description "serial": Entry "Subsections" missing
Section "kvm-apic" Description "apic": Entry "Subsections" missing
Section "apic-common" Description "apic": Entry "Subsections" missing

--------------

Comment 19 Dr. David Alan Gilbert 2015-06-26 16:48:40 UTC
OK, so I think I've posted all the simple PC devices:
   rtc
   serial
   kbd
   floppy

that leaves the rdma (1210715) code that I'm waiting to go in upstream.
The APIC changes are belived to only be with kernel_irqchip=off which doesn't normally happen.

So other than waiting for RDMA I think we're good.

Comment 21 huiqingding 2015-08-18 10:10:40 UTC
I test rdma migration:
1) boot guest on 7.2 host, do ping-pong migration 7.2<->7.1 for 5 times
2) boot guest on 7.1 host, do ping-pong migration 7.1<->7.2 for 5 times

RHEL7.2 host:
kernel-3.10.0-304.el7.x86_64
qemu-kvm-rhev-2.3.0-18.el7.x86_64

RHEL7.1 host:
kernel-3.10.0-229.13.1.el7.x86_64
qemu-kvm-rhev-2.1.2-23.el7_1.8.x86_64

RHEL7.2 guest:
kernel-3.10.0-302.el7.x86_64

The result is as following:
Host              Machine type          Result
------------------------------------------------
7.2<->7.1         rhel7.1.0            pass
7.2<->7.1         rhel7.0.0            pass
7.2<->7.1         rhel6.6.0            pass
7.2<->7.1         rhel6.5.0            pass
7.1<->7.2         rhel7.1.0            pass
7.1<->7.2         rhel7.0.0            pass
7.1<->7.2         rhel6.6.0            pass
7.1<->7.2         rhel6.5.0            pass

Steps to Test:
1. boot rhel7.2 guest on src host
2. boot the guest on dst host with "-incoming rdma:0:5800"
3. inside guest, do some operation:
# dd if=/dev/random of=/home/file bs=1M count=4096
# netperf -H 10.66.9.152 -l 600
4. do rdma migration:
(qemu) migrate_set_speed 40G
(qemu) migrate_set_capability rdma-pin-all on
(qemu) migrate rdma:192.168.0.21:5800
5. inside guest, ping other host, do reboot and shutdown.

The command line is:
/usr/libexec/qemu-kvm \
-name rhel7 \
-S -machine pc-i440fx-rhel7.1.0,accel=kvm,usb=off \
-realtime mlock=off \
-cpu SandyBridge \
-sandbox off \
-m 3840   \
-smp 8,cores=1,threads=1,sockets=8  \
-object memory-backend-ram,policy=bind,id=mem-0,size=256M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-0,cpus=0,cpus=1 \
-object memory-backend-ram,policy=bind,id=mem-1,size=512M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-1,cpus=2-3 \
-object memory-backend-ram,policy=bind,id=mem-2,size=1024M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-2,cpus=4 \
-object memory-backend-ram,policy=bind,id=mem-3,size=2048M,prealloc=yes,host-nodes=0 -numa node,memdev=mem-3,cpus=5-7 \
-uuid 49a3438a-70a3-4ba8-92ce-3a05e0934608 \
-no-user-config \
-nodefaults \
-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/rhel7.monitor,server,nowait \
-mon chardev=charmonitor,id=monitor,mode=control \
-rtc base=utc,driftfix=slew \
-global kvm-pit.lost_tick_policy=discard \
-no-hpet \
-no-shutdown \
-global PIIX4_PM.disable_s3=1 \
-global PIIX4_PM.disable_s4=1 \
-boot order=c,menu=on,splash-time=3000,strict=on \
-device ich9-usb-ehci1,id=usb0,bus=pci.0,addr=0x5.0x7 \
-device ich9-usb-uhci1,masterbus=usb0.0,firstport=0,bus=pci.0,multifunction=on,addr=0x5 \
-device ich9-usb-uhci2,masterbus=usb0.0,firstport=2,bus=pci.0,addr=0x5.0x1 \
-device ich9-usb-uhci3,masterbus=usb0.0,firstport=4,bus=pci.0,addr=0x5.0x2 \
-device virtio-scsi-pci,id=scsi0,cmd_per_lun=234,bus=pci.0,addr=0x8 \
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 \
-drive file=/usr/share/virtio-win/virtio-win.iso,if=none,id=drive-ide0-1-0,readonly=on,format=raw,serial= \
-device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 \
-drive file=/usr/share/virtio-win/virtio-win_amd64.vfd,if=none,id=drive-fdc0-0-0,format=raw,cache=none \
-global isa-fdc.driveA=drive-fdc0-0-0 \
-drive file=/home/rhel7.2.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,serial=f65effa5-90a6-47f2-8487-a9f64c95d4f5,cache=none,discard=unmap,werror=stop,rerror=stop,aio=threads \
-device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x9,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 \
-drive file=/home/virtio-scsi-blk,if=none,id=drive-scsi-disk,format=qcow2,cache=writethrough,werror=stop,rerror=stop \
-device virtio-scsi-pci,id=scsi1,addr=0x13 \
-device scsi-hd,drive=drive-scsi-disk,bus=scsi1.0,id=data-disk2 \
-chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 \
-chardev spicevmc,id=charchannel0,name=vdagent \
-device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 \
-chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channel/target/rhel7.org.qemu.guest_agent.0,server,nowait \
-device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 \
-device usb-tablet,id=input0 \
-device intel-hda,id=sound0,bus=pci.0,addr=0x4 \
-device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
-device intel-hda,id=sound1,bus=pci.0,addr=0x6 \
-device hda-micro,id=sound1-codec0,bus=sound1.0 \
-device intel-hda,id=sound2,bus=pci.0,addr=0x17 \
-device hda-output,id=sound2-codec0,bus=sound2.0,cad=0 \
-device ich9-intel-hda,id=sound3,bus=pci.0,addr=0xc \
-device hda-duplex,id=sound3-codec0,bus=sound3.0,cad=0 \
-chardev spicevmc,id=charredir0,name=usbredir \
-device usb-redir,chardev=charredir0,id=redir0 \
-chardev spicevmc,id=charredir1,name=usbredir \
-device usb-redir,chardev=charredir1,id=redir1 \
-chardev spicevmc,id=charredir2,name=usbredir \
-device usb-redir,chardev=charredir2,id=redir2 \
-chardev spicevmc,id=charredir3,name=usbredir \
-device usb-redir,chardev=charredir3,id=redir3 \
-device usb-host,id=hostdev0 \
-device pvpanic,ioport=1285 -msg timestamp=on \
-netdev tap,id=hostnet0,id=hostnet0,script=/etc/qemu-ifup \
-device virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=b6:af:42:b8:46:18,bus=pci.0,addr=0x14 \
-netdev tap,id=hostnet1,script=/etc/qemu-ifup  \
-device e1000,netdev=hostnet1,id=virtio-net-pci1,mac=b6:2f:a8:85:72:6c,bus=pci.0,addr=0x15,multifunction=off \
-netdev tap,id=hostnet2,vhost=on,script=/etc/qemu-ifup \
-device rtl8139,netdev=hostnet2,id=virtio-net-pci2,mac=4e:63:28:bc:b1:25,bus=pci.0,addr=0x16,multifunction=off \
-netdev tap,id=hostnet3,vhost=on,script=/etc/qemu-ifup \
-device e1000-82540em,netdev=hostnet3,id=virtio-net-pci3,mac=4e:63:28:bc:b1:35,bus=pci.0,addr=0x18,multifunction=off \
-netdev tap,id=hostnet4,vhost=on,script=/etc/qemu-ifup \
-device e1000-82544gc,netdev=hostnet4,id=virtio-net-pci4,mac=4e:63:28:bc:b1:45,bus=pci.0,addr=0x19,multifunction=off \
-netdev tap,id=hostnet5,vhost=on,script=/etc/qemu-ifup \
-device e1000-82545em,netdev=hostnet5,id=virtio-net-pci5,mac=4e:63:28:bc:b1:55,bus=pci.0,addr=0x1a,multifunction=off \
-drive file=/home/ide-disk,if=none,id=drive-data-disk,format=raw,cache=none,aio=native,werror=stop,rerror=stop,copy-on-read=off,serial=fux-ide,media=disk \
-device ide-drive,drive=drive-data-disk,id=system-disk,logical_block_size=512,physical_block_size=512,min_io_size=32,opt_io_size=64,discard_granularity=512,ver=fuxc-ver,bus=ide.0,unit=0  \
-device ich9-usb-uhci6,id=uhci6,bus=pci.0,addr=0xa \
-device usb-kbd,id=kdb0,bus=uhci6.0 \
-device ich9-usb-uhci5,id=uhci5,bus=pci.0,addr=0xb \
-device usb-mouse,id=mouse0,bus=uhci5.0 \
-device nec-usb-xhci,id=xhci,bus=pci.0,addr=0xd \
-device usb-bot,id=bot1,bus=xhci.0 \
-drive file=/usr/share/virtio-win/virtio-win.iso,if=none,id=usb-cdrom1,format=raw \
-device scsi-cd,bus=bot1.0,scsi-id=0,lun=1,drive=usb-cdrom1,id=usb-cdrom1 \
-drive file=/home/bot-disk1,id=usb-disk1,if=none,format=qcow2 \
-device scsi-hd,bus=bot1.0,scsi-id=0,lun=0,drive=usb-disk1,id=usb-disk1 \
-device usb-ehci,id=ehci,bus=pci.0,addr=0xe \
-device usb-bot,id=bot2,bus=ehci.0 \
-drive file=/usr/share/virtio-win/virtio-win.iso,if=none,id=usb-cdrom2,format=raw \
-device scsi-cd,bus=bot2.0,scsi-id=0,lun=1,drive=usb-cdrom2,id=usb-cdrom2 \
-drive file=/home/bot-disk2,id=usb-disk2,if=none,format=qcow2 \
-device scsi-hd,bus=bot2.0,scsi-id=0,lun=0,drive=usb-disk2,id=usb-disk2 \
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0xf \
-device usb-bot,id=bot3,bus=usb.0 \
-drive file=/usr/share/virtio-win/virtio-win.iso,if=none,id=usb-cdrom3,format=raw \
-device scsi-cd,bus=bot3.0,scsi-id=0,lun=1,drive=usb-cdrom3,id=usb-cdrom3 \
-drive file=/home/bot-disk3,id=usb-disk3,if=none,format=qcow2 \
-device scsi-hd,bus=bot3.0,scsi-id=0,lun=0,drive=usb-disk3,id=usb-disk3 \
-device ich9-usb-uhci3,id=uhci,bus=pci.0,addr=0x10 \
-device usb-storage,drive=drive-usb-0,id=usb-0,removable=on,bus=uhci.0,port=1 \
-drive file=/home/usb-uhci,if=none,id=drive-usb-0,media=disk,format=qcow2 \
-device ich9-usb-ehci1,id=ehci1,bus=pci.0,addr=0x11 \
-device usb-storage,drive=drive-usb-1,id=usb-1,removable=on,bus=ehci1.0,port=1 \
-drive file=/home/usb-ehci,if=none,id=drive-usb-1,media=disk,format=qcow2 \
-device nec-usb-xhci,id=xhci1,bus=pci.0,addr=0x12 \
-device usb-storage,drive=drive-usb-2,id=usb-2,removable=on,bus=xhci1.0,port=1 \
-drive file=/home/usb-xhci,if=none,id=drive-usb-2,media=disk,format=qcow2 \
-device usb-ccid,id=ccid0 \
-watchdog ib700 -watchdog-action reset \
-device virtio-rng-pci,id=rng0,bus=pci.0,addr=0x1c \
-device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x1d \
-monitor stdio \
-qmp tcp:0:4466,server,nowait -serial unix:/tmp/ttym,server,nowait \
-spice port=5910,addr=0.0.0.0,disable-ticketing,seamless-migration=on \
-k en-us \
-device qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2

Comment 22 Dr. David Alan Gilbert 2015-08-18 10:15:48 UTC
(In reply to huiqingding from comment #21)
> I test rdma migration:
> 1) boot guest on 7.2 host, do ping-pong migration 7.2<->7.1 for 5 times
> 2) boot guest on 7.1 host, do ping-pong migration 7.1<->7.2 for 5 times
> 
> RHEL7.2 host:
> kernel-3.10.0-304.el7.x86_64
> qemu-kvm-rhev-2.3.0-18.el7.x86_64
> 
> RHEL7.1 host:
> kernel-3.10.0-229.13.1.el7.x86_64
> qemu-kvm-rhev-2.1.2-23.el7_1.8.x86_64
> 
> RHEL7.2 guest:
> kernel-3.10.0-302.el7.x86_64
> 
> The result is as following:
> Host              Machine type          Result
> ------------------------------------------------
> 7.2<->7.1         rhel7.1.0            pass
> 7.2<->7.1         rhel7.0.0            pass
> 7.2<->7.1         rhel6.6.0            pass
> 7.2<->7.1         rhel6.5.0            pass
> 7.1<->7.2         rhel7.1.0            pass
> 7.1<->7.2         rhel7.0.0            pass
> 7.1<->7.2         rhel6.6.0            pass
> 7.1<->7.2         rhel6.5.0            pass
> 
> Steps to Test:
> 1. boot rhel7.2 guest on src host
> 2. boot the guest on dst host with "-incoming rdma:0:5800"
> 3. inside guest, do some operation:
> # dd if=/dev/random of=/home/file bs=1M count=4096
> # netperf -H 10.66.9.152 -l 600
> 4. do rdma migration:
> (qemu) migrate_set_speed 40G
> (qemu) migrate_set_capability rdma-pin-all on
> (qemu) migrate rdma:192.168.0.21:5800
> 5. inside guest, ping other host, do reboot and shutdown.
> 
> The command line is:
> /usr/libexec/qemu-kvm \
> -name rhel7 \
> -S -machine pc-i440fx-rhel7.1.0,accel=kvm,usb=off \
> -realtime mlock=off \
> -cpu SandyBridge \
> -sandbox off \
> -m 3840   \
> -smp 8,cores=1,threads=1,sockets=8  \
> -object
> memory-backend-ram,policy=bind,id=mem-0,size=256M,prealloc=yes,host-nodes=0
> -numa node,memdev=mem-0,cpus=0,cpus=1 \
> -object
> memory-backend-ram,policy=bind,id=mem-1,size=512M,prealloc=yes,host-nodes=0
> -numa node,memdev=mem-1,cpus=2-3 \
> -object
> memory-backend-ram,policy=bind,id=mem-2,size=1024M,prealloc=yes,host-nodes=0
> -numa node,memdev=mem-2,cpus=4 \
> -object
> memory-backend-ram,policy=bind,id=mem-3,size=2048M,prealloc=yes,host-nodes=0
> -numa node,memdev=mem-3,cpus=5-7 \
> -uuid 49a3438a-70a3-4ba8-92ce-3a05e0934608 \
> -no-user-config \
> -nodefaults \
> -chardev
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/rhel7.monitor,server,nowait
> \
> -mon chardev=charmonitor,id=monitor,mode=control \
> -rtc base=utc,driftfix=slew \
> -global kvm-pit.lost_tick_policy=discard \
> -no-hpet \
> -no-shutdown \
> -global PIIX4_PM.disable_s3=1 \
> -global PIIX4_PM.disable_s4=1 \
> -boot order=c,menu=on,splash-time=3000,strict=on \
> -device ich9-usb-ehci1,id=usb0,bus=pci.0,addr=0x5.0x7 \
> -device
> ich9-usb-uhci1,masterbus=usb0.0,firstport=0,bus=pci.0,multifunction=on,
> addr=0x5 \
> -device ich9-usb-uhci2,masterbus=usb0.0,firstport=2,bus=pci.0,addr=0x5.0x1 \
> -device ich9-usb-uhci3,masterbus=usb0.0,firstport=4,bus=pci.0,addr=0x5.0x2 \
> -device virtio-scsi-pci,id=scsi0,cmd_per_lun=234,bus=pci.0,addr=0x8 \
> -device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x7 \
> -drive
> file=/usr/share/virtio-win/virtio-win.iso,if=none,id=drive-ide0-1-0,
> readonly=on,format=raw,serial= \
> -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 \
> -drive
> file=/usr/share/virtio-win/virtio-win_amd64.vfd,if=none,id=drive-fdc0-0-0,
> format=raw,cache=none \
> -global isa-fdc.driveA=drive-fdc0-0-0 \
> -drive
> file=/home/rhel7.2.qcow2,if=none,id=drive-virtio-disk0,format=qcow2,
> serial=f65effa5-90a6-47f2-8487-a9f64c95d4f5,cache=none,discard=unmap,
> werror=stop,rerror=stop,aio=threads \
> -device
> virtio-blk-pci,scsi=off,bus=pci.0,addr=0x9,drive=drive-virtio-disk0,
> id=virtio-disk0,bootindex=1 \
> -drive
> file=/home/virtio-scsi-blk,if=none,id=drive-scsi-disk,format=qcow2,
> cache=writethrough,werror=stop,rerror=stop \
> -device virtio-scsi-pci,id=scsi1,addr=0x13 \
> -device scsi-hd,drive=drive-scsi-disk,bus=scsi1.0,id=data-disk2 \
> -chardev pty,id=charserial0 -device
> isa-serial,chardev=charserial0,id=serial0 \
> -chardev spicevmc,id=charchannel0,name=vdagent \
> -device
> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,
> name=com.redhat.spice.0 \
> -chardev
> socket,id=charchannel1,path=/var/lib/libvirt/qemu/channel/target/rhel7.org.
> qemu.guest_agent.0,server,nowait \
> -device
> virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,
> name=org.qemu.guest_agent.0 \
> -device usb-tablet,id=input0 \
> -device intel-hda,id=sound0,bus=pci.0,addr=0x4 \
> -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 \
> -device intel-hda,id=sound1,bus=pci.0,addr=0x6 \
> -device hda-micro,id=sound1-codec0,bus=sound1.0 \
> -device intel-hda,id=sound2,bus=pci.0,addr=0x17 \
> -device hda-output,id=sound2-codec0,bus=sound2.0,cad=0 \
> -device ich9-intel-hda,id=sound3,bus=pci.0,addr=0xc \
> -device hda-duplex,id=sound3-codec0,bus=sound3.0,cad=0 \
> -chardev spicevmc,id=charredir0,name=usbredir \
> -device usb-redir,chardev=charredir0,id=redir0 \
> -chardev spicevmc,id=charredir1,name=usbredir \
> -device usb-redir,chardev=charredir1,id=redir1 \
> -chardev spicevmc,id=charredir2,name=usbredir \
> -device usb-redir,chardev=charredir2,id=redir2 \
> -chardev spicevmc,id=charredir3,name=usbredir \
> -device usb-redir,chardev=charredir3,id=redir3 \
> -device usb-host,id=hostdev0 \
> -device pvpanic,ioport=1285 -msg timestamp=on \
> -netdev tap,id=hostnet0,id=hostnet0,script=/etc/qemu-ifup \
> -device
> virtio-net-pci,netdev=hostnet0,id=virtio-net-pci0,mac=b6:af:42:b8:46:18,
> bus=pci.0,addr=0x14 \
> -netdev tap,id=hostnet1,script=/etc/qemu-ifup  \
> -device
> e1000,netdev=hostnet1,id=virtio-net-pci1,mac=b6:2f:a8:85:72:6c,bus=pci.0,
> addr=0x15,multifunction=off \
> -netdev tap,id=hostnet2,vhost=on,script=/etc/qemu-ifup \
> -device
> rtl8139,netdev=hostnet2,id=virtio-net-pci2,mac=4e:63:28:bc:b1:25,bus=pci.0,
> addr=0x16,multifunction=off \
> -netdev tap,id=hostnet3,vhost=on,script=/etc/qemu-ifup \
> -device
> e1000-82540em,netdev=hostnet3,id=virtio-net-pci3,mac=4e:63:28:bc:b1:35,
> bus=pci.0,addr=0x18,multifunction=off \
> -netdev tap,id=hostnet4,vhost=on,script=/etc/qemu-ifup \
> -device
> e1000-82544gc,netdev=hostnet4,id=virtio-net-pci4,mac=4e:63:28:bc:b1:45,
> bus=pci.0,addr=0x19,multifunction=off \
> -netdev tap,id=hostnet5,vhost=on,script=/etc/qemu-ifup \
> -device
> e1000-82545em,netdev=hostnet5,id=virtio-net-pci5,mac=4e:63:28:bc:b1:55,
> bus=pci.0,addr=0x1a,multifunction=off \
> -drive
> file=/home/ide-disk,if=none,id=drive-data-disk,format=raw,cache=none,
> aio=native,werror=stop,rerror=stop,copy-on-read=off,serial=fux-ide,
> media=disk \
> -device
> ide-drive,drive=drive-data-disk,id=system-disk,logical_block_size=512,
> physical_block_size=512,min_io_size=32,opt_io_size=64,
> discard_granularity=512,ver=fuxc-ver,bus=ide.0,unit=0  \
> -device ich9-usb-uhci6,id=uhci6,bus=pci.0,addr=0xa \
> -device usb-kbd,id=kdb0,bus=uhci6.0 \
> -device ich9-usb-uhci5,id=uhci5,bus=pci.0,addr=0xb \
> -device usb-mouse,id=mouse0,bus=uhci5.0 \
> -device nec-usb-xhci,id=xhci,bus=pci.0,addr=0xd \
> -device usb-bot,id=bot1,bus=xhci.0 \
> -drive
> file=/usr/share/virtio-win/virtio-win.iso,if=none,id=usb-cdrom1,format=raw \
> -device scsi-cd,bus=bot1.0,scsi-id=0,lun=1,drive=usb-cdrom1,id=usb-cdrom1 \
> -drive file=/home/bot-disk1,id=usb-disk1,if=none,format=qcow2 \
> -device scsi-hd,bus=bot1.0,scsi-id=0,lun=0,drive=usb-disk1,id=usb-disk1 \
> -device usb-ehci,id=ehci,bus=pci.0,addr=0xe \
> -device usb-bot,id=bot2,bus=ehci.0 \
> -drive
> file=/usr/share/virtio-win/virtio-win.iso,if=none,id=usb-cdrom2,format=raw \
> -device scsi-cd,bus=bot2.0,scsi-id=0,lun=1,drive=usb-cdrom2,id=usb-cdrom2 \
> -drive file=/home/bot-disk2,id=usb-disk2,if=none,format=qcow2 \
> -device scsi-hd,bus=bot2.0,scsi-id=0,lun=0,drive=usb-disk2,id=usb-disk2 \
> -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0xf \
> -device usb-bot,id=bot3,bus=usb.0 \
> -drive
> file=/usr/share/virtio-win/virtio-win.iso,if=none,id=usb-cdrom3,format=raw \
> -device scsi-cd,bus=bot3.0,scsi-id=0,lun=1,drive=usb-cdrom3,id=usb-cdrom3 \
> -drive file=/home/bot-disk3,id=usb-disk3,if=none,format=qcow2 \
> -device scsi-hd,bus=bot3.0,scsi-id=0,lun=0,drive=usb-disk3,id=usb-disk3 \
> -device ich9-usb-uhci3,id=uhci,bus=pci.0,addr=0x10 \
> -device
> usb-storage,drive=drive-usb-0,id=usb-0,removable=on,bus=uhci.0,port=1 \
> -drive file=/home/usb-uhci,if=none,id=drive-usb-0,media=disk,format=qcow2 \
> -device ich9-usb-ehci1,id=ehci1,bus=pci.0,addr=0x11 \
> -device
> usb-storage,drive=drive-usb-1,id=usb-1,removable=on,bus=ehci1.0,port=1 \
> -drive file=/home/usb-ehci,if=none,id=drive-usb-1,media=disk,format=qcow2 \
> -device nec-usb-xhci,id=xhci1,bus=pci.0,addr=0x12 \
> -device
> usb-storage,drive=drive-usb-2,id=usb-2,removable=on,bus=xhci1.0,port=1 \
> -drive file=/home/usb-xhci,if=none,id=drive-usb-2,media=disk,format=qcow2 \
> -device usb-ccid,id=ccid0 \
> -watchdog ib700 -watchdog-action reset \
> -device virtio-rng-pci,id=rng0,bus=pci.0,addr=0x1c \
> -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x1d \
> -monitor stdio \
> -qmp tcp:0:4466,server,nowait -serial unix:/tmp/ttym,server,nowait \
> -spice port=5910,addr=0.0.0.0,disable-ticketing,seamless-migration=on \
> -k en-us \
> -device
> qxl-vga,id=video0,ram_size=134217728,vram_size=67108864,vgamem_mb=16,bus=pci.
> 0,addr=0x2

Excellent!  Very nice mix of devices.

Comment 23 juzhang 2015-08-24 02:45:21 UTC
According to comment21 and comment22, set this issue as verified.


Note You need to log in before you can comment on or make changes to this bug.