| Summary: | Migration fail with heavy mem load on host | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux 6 | Reporter: | zhanghaiyan <yoyzhang> |
| Component: | libvirt | Assignee: | Daniel Veillard <veillard> |
| Status: | CLOSED INSUFFICIENT_DATA | QA Contact: | Virtualization Bugs <virt-bugs> |
| Severity: | medium | Docs Contact: | |
| Priority: | medium | ||
| Version: | 6.1 | CC: | berrange, dallan, dyuan, eblake, llim, weizhan, xen-maint |
| Target Milestone: | rc | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2011-06-21 01:43:03 UTC | Type: | --- |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
If QEMU is crashing, then this is fairly unlikely to be a libvirt bug. Please use GDB to obtain a stack trace from the QEMU coredump. On source host, no useful gdb info is generated On target host, no qemu-kvm process during the migration So, I am sorry I cannot provide gbd debug info :( Could you please give some suggestion ? # gdb /usr/libexec/qemu-kvm coredump GNU gdb (GDB) Red Hat Enterprise Linux (7.2-36.el6) Copyright (C) 2010 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-redhat-linux-gnu". For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>... Reading symbols from /usr/libexec/qemu-kvm...(no debugging symbols found)...done. [New Thread 20609] [New Thread 20630] [New Thread 20631] [New Thread 20632] [New Thread 20629] Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/librt.so.1 Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libpthread.so.0 Reading symbols from /lib64/libaio.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libaio.so.1 Reading symbols from /lib64/libutil.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libutil.so.1 Reading symbols from /lib64/libasound.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libasound.so.2 Reading symbols from /usr/lib64/libpulse.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libpulse.so.0 Reading symbols from /usr/lib64/libpulse-simple.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libpulse-simple.so.0 Reading symbols from /usr/lib64/libsasl2.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libsasl2.so.2 Reading symbols from /usr/lib64/libgnutls.so.26...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libgnutls.so.26 Reading symbols from /usr/lib64/libspice-server.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libspice-server.so.1 Reading symbols from /usr/lib64/libpixman-1.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libpixman-1.so.0 Reading symbols from /usr/lib64/libcelt051.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcelt051.so.0 Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libm.so.6 Reading symbols from /usr/lib64/libssl.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libssl.so.10 Reading symbols from /usr/lib64/libcrypto.so.10...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libcrypto.so.10 Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libdl.so.2 Reading symbols from /lib64/libz.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libz.so.1 Reading symbols from /usr/lib64/libXrandr.so.2...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libXrandr.so.2 Reading symbols from /usr/lib64/libXfixes.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libXfixes.so.3 Reading symbols from /usr/lib64/libXext.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libXext.so.6 Reading symbols from /usr/lib64/libXrender.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libXrender.so.1 Reading symbols from /usr/lib64/libX11.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libX11.so.6 Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done. Loaded symbols for /lib64/libc.so.6 Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/ld-linux-x86-64.so.2 Reading symbols from /usr/lib64/libpulsecommon-0.9.21.so...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libpulsecommon-0.9.21.so Reading symbols from /usr/lib64/libSM.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libSM.so.6 Reading symbols from /usr/lib64/libICE.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libICE.so.6 Reading symbols from /usr/lib64/libXtst.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libXtst.so.6 Reading symbols from /lib64/libwrap.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libwrap.so.0 Reading symbols from /usr/lib64/libsndfile.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libsndfile.so.1 Reading symbols from /usr/lib64/libasyncns.so.0...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libasyncns.so.0 Reading symbols from /lib64/libdbus-1.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libdbus-1.so.3 Reading symbols from /lib64/libresolv.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libresolv.so.2 Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libcrypt.so.1 Reading symbols from /usr/lib64/libtasn1.so.3...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libtasn1.so.3 Reading symbols from /lib64/libgcrypt.so.11...(no debugging symbols found)...done. Loaded symbols for /lib64/libgcrypt.so.11 Reading symbols from /usr/lib64/libjpeg.so.62...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libjpeg.so.62 Reading symbols from /lib64/libgssapi_krb5.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libgssapi_krb5.so.2 Reading symbols from /lib64/libkrb5.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5.so.3 Reading symbols from /lib64/libcom_err.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libcom_err.so.2 Reading symbols from /lib64/libk5crypto.so.3...(no debugging symbols found)...done. Loaded symbols for /lib64/libk5crypto.so.3 Reading symbols from /usr/lib64/libxcb.so.1...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libxcb.so.1 Reading symbols from /lib64/libuuid.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libuuid.so.1 Reading symbols from /usr/lib64/libXi.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libXi.so.6 Reading symbols from /lib64/libnsl.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libnsl.so.1 Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done. Loaded symbols for /lib64/libfreebl3.so Reading symbols from /lib64/libgpg-error.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libgpg-error.so.0 Reading symbols from /lib64/libkrb5support.so.0...(no debugging symbols found)...done. Loaded symbols for /lib64/libkrb5support.so.0 Reading symbols from /lib64/libkeyutils.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libkeyutils.so.1 Reading symbols from /usr/lib64/libXau.so.6...(no debugging symbols found)...done. Loaded symbols for /usr/lib64/libXau.so.6 Reading symbols from /lib64/libselinux.so.1...(no debugging symbols found)...done. Loaded symbols for /lib64/libselinux.so.1 Reading symbols from /lib64/libnss_files.so.2...(no debugging symbols found)...done. Loaded symbols for /lib64/libnss_files.so.2 Core was generated by `/usr/libexec/qemu-kvm -S -M rhel6.1.0 -enable-kvm -m 2048 -smp 4,sockets=4,core'. Program terminated with signal 6, Aborted. #0 0x00000037a58329a5 in raise () from /lib64/libc.so.6 Missing separate debuginfos, use: debuginfo-install qemu-kvm-0.12.1.2-2.147.el6.x86_64 (gdb) bt #0 0x00000037a58329a5 in raise () from /lib64/libc.so.6 #1 0x00000037a5834185 in abort () from /lib64/libc.so.6 #2 0x00000000004827e9 in ?? () #3 0x00000000004ead99 in ?? () #4 0x000000000045225e in ?? () #5 0x000000000040e579 in ?? () #6 0x00000037a581ec5d in __libc_start_main () from /lib64/libc.so.6 #7 0x0000000000407ff9 in ?? () #8 0x00007fff47dbe6e8 in ?? () #9 0x000000000000001c in ?? () #10 0x0000000000000030 in ?? () #11 0x00007fff47dbec22 in ?? () #12 0x00007fff47dbec38 in ?? () #13 0x00007fff47dbec3b in ?? () #14 0x00007fff47dbec3e in ?? () #15 0x00007fff47dbec48 in ?? () #16 0x00007fff47dbec54 in ?? () #17 0x00007fff47dbec57 in ?? () #18 0x00007fff47dbec5c in ?? () #19 0x00007fff47dbec61 in ?? () #20 0x00007fff47dbec7f in ?? () #21 0x00007fff47dbec85 in ?? () #22 0x00007fff47dbec93 in ?? () #23 0x00007fff47dbec99 in ?? () #24 0x00007fff47dbecbe in ?? () #25 0x00007fff47dbeccb in ?? () #26 0x00007fff47dbecd7 in ?? () #27 0x00007fff47dbece0 in ?? () #28 0x00007fff47dbed35 in ?? () #29 0x00007fff47dbed3a in ?? () #30 0x00007fff47dbed66 in ?? () #31 0x00007fff47dbed6b in ?? () #32 0x00007fff47dbed74 in ?? () #33 0x00007fff47dbed7a in ?? () #34 0x00007fff47dbed7c in ?? () #35 0x00007fff47dbed83 in ?? () #36 0x00007fff47dbedd3 in ?? () #37 0x00007fff47dbeddb in ?? () #38 0x00007fff47dbee26 in ?? () #39 0x00007fff47dbee2e in ?? () #40 0x00007fff47dbee44 in ?? () #41 0x00007fff47dbee4c in ?? () #42 0x00007fff47dbee95 in ?? () #43 0x00007fff47dbee9e in ?? () #44 0x00007fff47dbeeb1 in ?? () #45 0x00007fff47dbeeb9 in ?? () #46 0x00007fff47dbeee3 in ?? () #47 0x00007fff47dbeee8 in ?? () #48 0x00007fff47dbeef0 in ?? () #49 0x00007fff47dbef05 in ?? () #50 0x00007fff47dbef0a in ?? () Since RHEL 6.1 External Beta has begun, and this bug remains unresolved, it has been rejected as it is not proposed as exception or blocker. Red Hat invites you to ask your support representative to propose this request, if appropriate and relevant, in the next release of Red Hat Enterprise Linux. I'm closing as INSUFFICIENT DATA; if the behavior reappears, please reopen and change the component to qemu since qemu is crashing. |
Description of problem: Do migration between 2 hosts, one is Dell780 and the other is z400. scenario 1: Migrate a guest from z400 to 780 while loading heavy mem on target 780. ---> Migration pass. scenario 2: Migrate a guest from 780 to z400 while loading heavy mem on target z400 ---> Migration fail. On source 780, report this error error: cannot send monitor command '{"execute":"qmp_capabilities"}': Connection reset by peer Version-Release number of selected component (if applicable): - libvirt-0.8.7-7.el6.x86_64 - kernel-2.6.32-113.el6.x86_64 - qemu-kvm-0.12.1.2-2.146.el6.x86_64 How reproducible: 5/5 Steps to Reproduce: For scenario 2 1. Load heavy mem on target host z400 # stress --vm 27 --vm-bytes 256M --vm-keep --timeout 1000s Check the free mem # free total used free shared buffers cached Mem: 8109588 7685440 424148 0 2684 41604 -/+ buffers/cache: 7641152 468436 Swap: 0 0 0 2. Migrate guest from source (780) to target (400) # time virsh migrate --live rhel61_i386_5 qemu+ssh://10.66.93.59/system error: cannot send monitor command '{"execute":"qmp_capabilities"}': Connection reset by peer Actual results: 2. Migrate fail Expected results: 2. Migrate succeed Additional info: On target host # cat /var/log/message Feb 23 04:01:19 dhcp-93-59 abrt[10442]: saved core dump of pid 10416 (/usr/libexec/qemu-kvm) to /var/spool/abrt/ccpp-1298451679-10416.new/coredump (48615424 bytes)