Bug 1298552
| Summary: | migration error for VMs with rhel6 machine types | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 7 | Reporter: | Michal Skrivanek <michal.skrivanek> |
| Component: | qemu-kvm-rhev | Assignee: | Dr. David Alan Gilbert <dgilbert> |
| Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | urgent | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | 7.2 | CC: | bugs, dgilbert, dominique.taffin, fromani, gklein, huding, istein, juzhang, knoel, mavital, mburman, michal.skrivanek, virt-maint, xfu |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | 1293566 | Environment: | |
| Last Closed: | 2016-01-20 11:14:26 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1285700, 1293566, 1302742 | ||
|
Description
Michal Skrivanek
2016-01-14 12:26:59 UTC
> Perhaps a gpxe change in bug 1231931 is related, as this used to work just fine.
sorry, that's not a correct gpxe bug, something else then
Test migration between two RHEL7.2 hosts and the kernel and qemu-kvm version are: kernel-3.10.0-327.8.1.el7.x86_64 qemu-kvm-rhev-2.3.0-31.el7_2.5.x86_64 ipxe-roms-qemu-20130517-7.gitc4bce43.el7.noarch 1. boot vm in src host: # /usr/libexec/qemu-kvm -S -machine rhel6.5.0,accel=kvm,usb=off -cpu Westmere -smbios type=1 -no-user-config -nodefaults -net none -monitor stdio 2. boot vm in dst host: # /usr/libexec/qemu-kvm -S -machine rhel6.5.0,accel=kvm,usb=off -cpu Westmere -smbios type=1 -no-user-config -nodefaults -net none -monitor stdio -incoming tcp:0:5800 3. do migration from src host to dst host: (qemu) migrate -d tcp:dst_ip:5800 after step 3, qemu-kvm in dst host quits with error: copying E and F segments from pc.bios to pc.ram copying C and D segments from pc.rom to pc.ram ERROR: invalid runstate transition: 'inmigrate' -> 'prelaunch' Aborted (core dumped) (gdb) bt #0 0x00007ffff04e45f7 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007ffff04e5ce8 in __GI_abort () at abort.c:90 #2 0x00005555556d4bc7 in runstate_set (new_state=<optimized out>) at vl.c:648 #3 0x0000555555796401 in process_incoming_migration_co (opaque=0x555556a38000) at migration/migration.c:259 #4 0x00005555557ec70a in coroutine_trampoline (i0=<optimized out>, i1=<optimized out>) at coroutine-ucontext.c:80 #5 0x00007ffff04f6110 in ?? () from /lib64/libc.so.6 #6 0x00007fffffffd500 in ?? () #7 0x0000000000000000 in ?? () Also test -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off, hit the same problem. huiqingding: That's a separate issue; that's because you're trying to migrate a source which is in -S and hasn't been started yet. Michal: OK, we need to work out the history of those ROM sizes. We can't just change it back since that will break migration for other combinations.
> 2015-12-22T07:29:05.023584Z qemu-kvm: Length mismatch:
> 0000:00:03.0/virtio-net-pci.rom: 0x20000 in != 0x40000: Invalid argument
> 2015-12-22T07:29:05.023620Z qemu-kvm: error while loading state for instance
> 0x0 of device 'ram'
0x20000 = 128k
0x40000 = 256k
so I think that's saying it's trying to load a 128k ROM image from the migration stream, but it's expecting a 256k ROM.
The RHEL6 /usr/share/gpxe/virtio-net.rom comes from gpxe-roms-qemu, and both the 0.9.7-6.10.el6 package from 2015 and a nice new 0.9.7-6.15 both have that as 53760 bytes.
On RHEL7:
/usr/share/qemu-kvm/rhel6-virtio.rom comes from the qemu-kvm-rhev package itself; and is 53248 bytes in qemu-kvm-rhev-2.3.0-31.el7_2.5 and in the qemu-kvm-ev-2.3.0-31.el7_2.3.1 that I downloaded.
So those ROM sizes look OK - and neither of them are anywhere near 128 or 256k.
strace -e open /usr/libexec/qemu-kvm -M rhel6.5.0 -device virtio-net -S -nographic -nodefaults
shows:
open("/usr/share/qemu-kvm/rhel6-virtio.rom", O_RDONLY) = 10
so it is opening the ROM file we thought it should.
Hmm, we need to reproduce this fully.
Dave
I've dumped migration streams as well, both rhel7 and rhel6 are showing 64k RAM blocks for the virito-net-pci ROM; so I can't see where the sizes are coming from. The only case where we've got someone with a machine that can reproduce this, has a broken package install, and therefore it's not a qemu migration bug. Given that we don't understand how they got the broken installation, it's still possible there's a bug somewhere, and there's still plenty unexplained. So marking as INSUFFICIENT_DATA - please reopen if we can reproduce it or we find someone else who hits it that we can see what's going on in their filesystem. Please see bug - https://bugzilla.redhat.com/show_bug.cgi?id=1293566#c50 (In reply to meital avital from comment #12) > Please see bug - https://bugzilla.redhat.com/show_bug.cgi?id=1293566#c50 I think the needinfo? was intended for Dave. Yes, this bug was fixed; it was a packaging error with the iPXE roms on the RHEV isos. |