Bug 1279320
| Summary: | ibm s390x kvm VM gets stuck in a crash loop | ||
|---|---|---|---|
| Product: | [Community] Virtualization Tools | Reporter: | kevinzs |
| Component: | virt-manager | Assignee: | Cole Robinson <crobinso> |
| Status: | CLOSED DEFERRED | QA Contact: | |
| Severity: | high | Docs Contact: | |
| Priority: | unspecified | ||
| Version: | unspecified | CC: | berrange, crobinso, gscrivan, kevinzs, rbalakri |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | Doc Type: | Bug Fix | |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2015-11-11 15:03:01 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
My guess is that something is causing qemu to repeatedly reset. Can you provide /var/log/libvirt/qemu/$vmname.log from the libvirtd server? Hi Cole,
Here is the log information :
/var/log/libvirt/qemu/generic.log, and virt-manager shows that the generic has the states "crashed"
2015-11-11 02:37:08.963+0000: starting up libvirt version: 1.2.20, package: 1.el7_1.kvmibm1_1_1.2 (Koji, 2015-11-02-15:14:28, lnx2), qemu version: 2.4.0
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name generic -S -machine s390-ccw-kvmibm-1.1.1,accel=kvm,usb=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 47653a1e-6726-4ef7-83ab-28acd68800b4 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-generic/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-reboot -boot strict=on -device virtio-scsi-ccw,id=scsi0,devno=fe.0.0002 -drive file=/var/lib/libvirt/images/generic.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-ccw,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2 -drive file=/var/lib/libvirt/images/RHEL-7.0-20140507.0-Server-s390x-dvd1.iso,if=none,id=drive-scsi0-0-0-0,readonly=on,format=raw -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=22,id=hostnet0,vhost=on,vhostfd=23 -device virtio-net-ccw,netdev=hostnet0,id=net0,mac=52:54:00:71:77:4d,devno=fe.0.0001 -chardev pty,id=charconsole0 -device sclpconsole,chardev=charconsole0,id=console0 -device virtio-balloon-ccw,id=balloon0,devno=fe.0.0003 -msg timestamp=on
char device redirected to /dev/pts/1 (label charconsole0)
qemu: terminating on signal 15 from pid 1382
2015-11-11 02:37:09.399+0000: shutting down
2015-11-11 02:37:11.368+0000: starting up libvirt version: 1.2.20, package: 1.el7_1.kvmibm1_1_1.2 (Koji, 2015-11-02-15:14:28, lnx2), qemu version: 2.4.0
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name generic -S -machine s390-ccw-kvmibm-1.1.1,accel=kvm,usb=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 47653a1e-6726-4ef7-83ab-28acd68800b4 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-generic/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device virtio-scsi-ccw,id=scsi0,devno=fe.0.0002 -drive file=/var/lib/libvirt/images/generic.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-ccw,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-scsi0-0-0-0,readonly=on,format=raw -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 -netdev tap,fd=22,id=hostnet0,vhost=on,vhostfd=23 -device virtio-net-ccw,netdev=hostnet0,id=net0,mac=52:54:00:71:77:4d,devno=fe.0.0001 -chardev pty,id=charconsole0 -device sclpconsole,chardev=charconsole0,id=console0 -device virtio-balloon-ccw,id=balloon0,devno=fe.0.0003 -msg timestamp=on
char device redirected to /dev/pts/1 (label charconsole0)
(In reply to Cole Robinson from comment #1) > My guess is that something is causing qemu to repeatedly reset. Can you > provide /var/log/libvirt/qemu/$vmname.log from the libvirtd server? Hi Cole, Thanks for your help~ Here is the log information : /var/log/libvirt/qemu/generic.log, and virt-manager shows that the generic has the states "crashed" 2015-11-11 02:37:08.963+0000: starting up libvirt version: 1.2.20, package: 1.el7_1.kvmibm1_1_1.2 (Koji, 2015-11-02-15:14:28, lnx2), qemu version: 2.4.0 LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name generic -S -machine s390-ccw-kvmibm-1.1.1,accel=kvm,usb=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 47653a1e-6726-4ef7-83ab-28acd68800b4 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-generic/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-reboot -boot strict=on -device virtio-scsi-ccw,id=scsi0,devno=fe.0.0002 -drive file=/var/lib/libvirt/images/generic.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-ccw,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=2 -drive file=/var/lib/libvirt/images/RHEL-7.0-20140507.0-Server-s390x-dvd1.iso,if=none,id=drive-scsi0-0-0-0,readonly=on,format=raw -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,bootindex=1 -netdev tap,fd=22,id=hostnet0,vhost=on,vhostfd=23 -device virtio-net-ccw,netdev=hostnet0,id=net0,mac=52:54:00:71:77:4d,devno=fe.0.0001 -chardev pty,id=charconsole0 -device sclpconsole,chardev=charconsole0,id=console0 -device virtio-balloon-ccw,id=balloon0,devno=fe.0.0003 -msg timestamp=on char device redirected to /dev/pts/1 (label charconsole0) qemu: terminating on signal 15 from pid 1382 2015-11-11 02:37:09.399+0000: shutting down 2015-11-11 02:37:11.368+0000: starting up libvirt version: 1.2.20, package: 1.el7_1.kvmibm1_1_1.2 (Koji, 2015-11-02-15:14:28, lnx2), qemu version: 2.4.0 LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=none /usr/bin/qemu-kvm -name generic -S -machine s390-ccw-kvmibm-1.1.1,accel=kvm,usb=off -m 1024 -realtime mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 47653a1e-6726-4ef7-83ab-28acd68800b4 -nographic -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-generic/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -boot strict=on -device virtio-scsi-ccw,id=scsi0,devno=fe.0.0002 -drive file=/var/lib/libvirt/images/generic.qcow2,if=none,id=drive-virtio-disk0,format=qcow2 -device virtio-blk-ccw,scsi=off,devno=fe.0.0000,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive if=none,id=drive-scsi0-0-0-0,readonly=on,format=raw -device scsi-cd,bus=scsi0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0 -netdev tap,fd=22,id=hostnet0,vhost=on,vhostfd=23 -device virtio-net-ccw,netdev=hostnet0,id=net0,mac=52:54:00:71:77:4d,devno=fe.0.0001 -chardev pty,id=charconsole0 -device sclpconsole,chardev=charconsole0,id=console0 -device virtio-balloon-ccw,id=balloon0,devno=fe.0.0003 -msg timestamp=on char device redirected to /dev/pts/1 (label charconsole0) If the VM is crashing then you need to dig a bit deeper to figure out why. Could be a bug there, or a virt-manager XML issue, or libvirt qemu command line issue, or user error I'd suggest asking the KVM IBM folks, or take that qemu command line and try to manually make it work by hand. Closing as DEFERRED for now but please reopen if you determine virt-manager needs changes (In reply to Cole Robinson from comment #4) > If the VM is crashing then you need to dig a bit deeper to figure out why. > Could be a bug there, or a virt-manager XML issue, or libvirt qemu command > line issue, or user error > > I'd suggest asking the KVM IBM folks, or take that qemu command line and try > to manually make it work by hand. Closing as DEFERRED for now but please > reopen if you determine virt-manager needs changes Hi Cole, Thanks for your advice. I find that , when I don't specify the kernel and initrd , installation process has been skipped , and be treated successful. Then according to the boot xml, virt-manager will try to restart it again and again. Do you think I can debug this by getting the debug information of libvirtd in the server? Also ,I don't know the domain lifecycle event means , is it a event that sent by libvirtd in the server? Sincerely Thanks~ (In reply to Cole Robinson from comment #4) > If the VM is crashing then you need to dig a bit deeper to figure out why. > Could be a bug there, or a virt-manager XML issue, or libvirt qemu command > line issue, or user error > > I'd suggest asking the KVM IBM folks, or take that qemu command line and try > to manually make it work by hand. Closing as DEFERRED for now but please > reopen if you determine virt-manager needs changes Hi Cole, Thanks for your advice. I find that , when I don't specify the kernel and initrd , installation process has been skipped , and be treated successful. Then according to the boot xml, virt-manager will try to restart it again and again. Do you think I can debug this by getting the debug information of libvirtd in the server? Also ,I don't know the domain lifecycle event means , is it a event that sent by libvirtd in the server? Sincerely Thanks~ |
Description of problem: Virt-manager in amd64, connect libvirtd in s390x, install a guest OS redhat 7.0 in s390x. If do not specify kernel and initrd path, We can create the new guest (with minimal capacity and default NAT for network), but as soon as the guest starts, virt-manager tries to open a console connection to it, and hangs. The --debug output from virt-manager looks like this: [Fri, 30 Oct 2015 09:43:50 virt-manager 11927] DEBUG (connection:753) domain lifecycle event: domain=zamsaad_guest event=4 reason=0 [Fri, 30 Oct 2015 09:43:50 virt-manager 11927] DEBUG (connection:753) domain lifecycle event: domain=zamsaad_guest event=8 reason=0 [Fri, 30 Oct 2015 09:43:50 virt-manager 11927] DEBUG (connection:753) domain lifecycle event: domain=zamsaad_guest event=4 reason=0 [Fri, 30 Oct 2015 09:43:50 virt-manager 11927] DEBUG (connection:753) domain lifecycle event: domain=zamsaad_guest event=8 reason=0 Also , when we specify the kernel and initrd image path ,enable direct kernel boot, virt-manager will open guest install interface . But in this time, force off the guest ,disable direct kernel boot , virt-manager will go into loop like above. Version-Release number of selected component (if applicable): git clone from virt-tools . newest version How reproducible: Steps to Reproduce: 1.new connect to remote libvirtd using ssh 2.select Rhel7.0 iso,select image size 8G ,select virtual cpu and memory. 3.Do not specify kernel and initrd path, click finish to begin guest install Actual results: [一, 09 11月 2015 14:16:31 virt-manager 32128] DEBUG (guest:396) Generated install XML: <domain type="kvm"> <name>generic</name> <uuid>d7ed24c8-d7e7-4025-a1d2-2abcc9161599</uuid> <memory>1048576</memory> <currentMemory>1048576</currentMemory> <vcpu>1</vcpu> <os> <type arch="s390x" machine="s390-ccw-virtio">hvm</type> <boot dev="cdrom"/> <boot dev="hd"/> </os> <clock offset="utc"/> <on_poweroff>destroy</on_poweroff> <on_reboot>destroy</on_reboot> <on_crash>destroy</on_crash> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <disk type="file" device="disk"> <driver name="qemu" type="qcow2"/> <source file="/var/lib/libvirt/images/generic-1.qcow2"/> <target dev="vda" bus="virtio"/> </disk> <disk type="file" device="cdrom"> <driver name="qemu" type="raw"/> <source file="/var/lib/libvirt/images/RHEL-7.0-20140507.0-Server-s390x-dvd1.iso"/> <target dev="sda" bus="scsi"/> <readonly/> </disk> <interface type="network"> <source network="default"/> <mac address="52:54:00:c5:5b:ee"/> </interface> <console type="pty"/> </devices> </domain> [一, 09 11月 2015 14:16:31 virt-manager 32128] DEBUG (guest:397) Generated boot XML: <domain type="kvm"> <name>generic</name> <uuid>d7ed24c8-d7e7-4025-a1d2-2abcc9161599</uuid> <memory>1048576</memory> <currentMemory>1048576</currentMemory> <vcpu>1</vcpu> <os> <type arch="s390x" machine="s390-ccw-virtio">hvm</type> <boot dev="hd"/> </os> <clock offset="utc"/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <disk type="file" device="disk"> <driver name="qemu" type="qcow2"/> <source file="/var/lib/libvirt/images/generic-1.qcow2"/> <target dev="vda" bus="virtio"/> </disk> <disk type="block" device="cdrom"> <target dev="sda" bus="scsi"/> <readonly/> </disk> <interface type="network"> <source network="default"/> <mac address="52:54:00:c5:5b:ee"/> </interface> <console type="pty"/> </devices> </domain> [一, 09 11月 2015 14:16:32 virt-manager 32128] DEBUG (connection:754) domain lifecycle event: domain=generic event=4 reason=0 [一, 09 11月 2015 14:16:32 virt-manager 32128] DEBUG (connection:754) domain lifecycle event: domain=generic event=2 reason=0 [一, 09 11月 2015 14:16:32 virt-manager 32128] DEBUG (connection:754) domain lifecycle event: domain=generic event=8 reason=0 [一, 09 11月 2015 14:16:32 virt-manager 32128] DEBUG (connection:754) domain lifecycle event: domain=generic event=5 reason=2 [一, 09 11月 2015 14:16:32 virt-manager 32128] DEBUG (connection:754) domain lifecycle event: domain=generic event=0 reason=0 [一, 09 11月 2015 14:16:33 virt-manager 32128] DEBUG (guest:427) XML fetched from libvirt object: <domain type='kvm'> <name>generic</name> <uuid>d7ed24c8-d7e7-4025-a1d2-2abcc9161599</uuid> <memory unit='KiB'>1048576</memory> <currentMemory unit='KiB'>1048576</currentMemory> <vcpu placement='static'>1</vcpu> <os> <type arch='s390x' machine='s390-ccw-kvmibm-1.1.1'>hvm</type> <boot dev='hd'/> </os> <clock offset='utc'/> <on_poweroff>destroy</on_poweroff> <on_reboot>restart</on_reboot> <on_crash>restart</on_crash> <devices> <emulator>/usr/bin/qemu-kvm</emulator> <disk type='file' device='disk'> <driver name='qemu' type='qcow2'/> <source file='/var/lib/libvirt/images/generic-1.qcow2'/> <target dev='vda' bus='virtio'/> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0000'/> </disk> <disk type='block' device='cdrom'> <driver name='qemu' type='raw'/> <target dev='sda' bus='scsi'/> <readonly/> <address type='drive' controller='0' bus='0' target='0' unit='0'/> </disk> <controller type='scsi' index='0'> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0002'/> </controller> <controller type='virtio-serial' index='0'> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0003'/> </controller> <interface type='network'> <mac address='52:54:00:c5:5b:ee'/> <source network='default'/> <model type='virtio'/> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0001'/> </interface> <console type='pty'> <target type='virtio' port='0'/> </console> <memballoon model='virtio'> <address type='ccw' cssid='0xfe' ssid='0x0' devno='0x0004'/> </memballoon> </devices> </domain> [一, 09 11月 2015 14:16:33 virt-manager 32128] DEBUG (create:2291) Install completed [一, 09 11月 2015 14:16:36 virt-manager 32128] DEBUG (connection:1050) domain=generic status=Shutoff added [一, 09 11月 2015 14:16:36 virt-manager 32128] DEBUG (connection:1044) New domain=generic requested, but it's already tracked. [一, 09 11月 2015 14:16:36 virt-manager 32128] DEBUG (connection:1044) New domain=generic requested, but it's already tracked. [一, 09 11月 2015 14:16:36 virt-manager 32128] DEBUG (connection:1044) New domain=generic requested, but it's already tracked. Expected results: Do not try to restart again and again ,or try restart several times then stop. Additional info: