Bug 689768

My guess is that the underlying error is related to bug 689761, however libvirt shouldn't be trying hard to report a better error than that. Reassigning to libvirt.

I assume we are racing with qemu: qemu closes the monitor, we are disconnected, but detect the process as still alive since it is still shutting down. Maybe we should always append the log output for startup errors, if any actually exists.

Here's another example of suboptimal error reporting when the host is under load:

https://bugzilla.redhat.com/show_bug.cgi?id=634069

Neil, can you reproduce the qmp_capabilities error semi-reliably?

Edit an existent guest, seems it can't be reproduced every time. I tried to start the guest many times, but got the different results.

# virsh dumpxml foo
<domain type='kvm'>
  <name>foo</name>
  <uuid>f403a96b-e521-fedf-d615-4a42b928a79b</uuid>
  <memory>524288</memory>
  <currentMemory>524288</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='x86_64' machine='rhel6.1.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/libexec/qemu-kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2' cache='none' io='threads'/>
      <source file='/var/lib/libvirt/images/foo.qcow2'/>
      <target dev='vda' bus='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </disk>
    <disk type='block' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
      <address type='drive' controller='0' bus='1' unit='0'/>
    </disk>
    <controller type='ide' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:0b:49:44'/>
      <source network='default'/>
      <model type='virtio'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='udp'>
      <source mode='connect' service='4444'/>
      <target port='0'/>
    </serial>
    <console type='udp'>
      <source mode='connect' service='4444'/>
      <target type='serial' port='0'/>
    </console>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes'/>
    <sound model='ich6'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </memballoon>
  </devices>
</domain>

# virsh start foo
error: Failed to start domain foo
error: cannot send monitor command '{"execute":"qmp_capabilities"}': Connection reset by peer

# virsh start foo
error: Failed to start domain foo
error: internal error process exited while connecting to monitor: getaddrinfo((null),4444): Name or service not known
inet_dgram_opts failed
chardev: opening backend "udp" failed

I caught the qmp_capabilities error in libvirtd log.

16:01:59.408: 1261: debug : qemudStartVMDaemon:3038 : Beginning VM startup process
16:01:59.408: 1261: debug : qemudStartVMDaemon:3050 : Setting current domain def as transient
16:01:59.409: 1261: debug : qemudStartVMDaemon:3057 : Preparing host devices
16:01:59.409: 1261: debug : qemudStartVMDaemon:3061 : Preparing chr devices
16:01:59.409: 1261: debug : qemudStartVMDaemon:3070 : Generating domain security label (if required)
16:01:59.409: 1261: debug : qemudStartVMDaemon:3080 : Generating setting domain security labels (if required)
16:01:59.409: 1261: debug : qemudStartVMDaemon:3110 : Ensuring no historical cgroup is lying around
16:01:59.410: 1261: debug : qemudStartVMDaemon:3156 : Creating domain log file
16:01:59.410: 1261: debug : qemudStartVMDaemon:3160 : Determining emulator version
16:01:59.483: 1261: debug : qemudStartVMDaemon:3166 : Setting up domain cgroup (if required)
16:01:59.485: 1261: debug : qemudStartVMDaemon:3175 : Preparing monitor state
16:01:59.485: 1261: debug : qemudStartVMDaemon:3210 : Assigning domain PCI addresses
16:01:59.485: 1261: debug : qemudStartVMDaemon:3229 : Building emulator command line
16:01:59.494: 1261: debug : qemudStartVMDaemon:3275 : Clear emulator capabilities: 1
16:01:59.494: 1261: warning : qemudStartVMDaemon:3279 : Executing /usr/libexec/qemu-kvm
16:01:59.498: 1261: warning : qemudStartVMDaemon:3289 : Executing done /usr/libexec/qemu-kvm
16:01:59.498: 1261: debug : qemudStartVMDaemon:3328 : Waiting for monitor to show up
16:01:59.498: 1261: debug : qemudWaitForMonitor:2244 : Connect monitor to 0x1e0a670 'foo'
16:01:59.701: 1261: debug : qemuMonitorJSONCommandWithFd:218 : Send command '{"execute":"qmp_capabilities"}' for write with FD -1
16:02:00.166: 1255: debug : qemuHandleMonitorEOF:737 : Received EOF on 0x1e0a670 'foo'
16:02:00.166: 1261: debug : qemuMonitorJSONCommandWithFd:223 : Receive command reply ret=-1 errno=104 0 bytes '(null)'
16:02:00.166: 1255: debug : qemudShutdownVMDaemon:3402 : Shutting down VM 'foo' pid=1360 migrated=0
16:02:00.306: 1261: debug : qemudShutdownVMDaemon:3402 : Shutting down VM 'foo' pid=-1 migrated=0
16:02:00.306: 1261: debug : qemudShutdownVMDaemon:3405 : VM 'foo' not active

Since RHEL 6.1 External Beta has begun, and this bug remains
unresolved, it has been rejected as it is not proposed as
exception or blocker.

Red Hat invites you to ask your support representative to
propose this request, if appropriate and relevant, in the
next release of Red Hat Enterprise Linux.

This bug is very close related to BZ#68971 in not the same. Therefore patch which fixed that bug has fixed this as well:

commit d30d5726507fd59a87127eb22075cfcb4482f9df
Author:     Cole Robinson <crobinso>
AuthorDate: Thu Aug 11 15:19:51 2011 +0800
Commit:     Daniel Veillard <veillard>
CommitDate: Thu Aug 11 15:19:51 2011 +0800

    qemu: Fix -chardev udp if parameters are omitted
    
    The following XML:
    
        <serial type='udp'>
          <source mode='connect' service='9999'/>
        </serial>
    
    is accepted by domain_conf.c but maps to the qemu command line:
    
    -chardev udp,host=127.0.0.1,port=2222,localaddr=(null),localport=(null)
    
    qemu can cope with everything omitting except the connection port, which
    seems to also be the intent of domain_conf validation, so let's not
    generate bogus command lines for that case.
    The defaults are empty strings for addresses and 0 for the localport
    
    Additionally, tweak the qemu cli parsing to handle omitted host
    parameters
    for -serial udp

v0.9.4-23-gd30d572

However, moving to modified directly so we can QA it.

Reproduce this issue with:
libvirt-0.8.7-18.el6.x86_64
libvirt-python-0.8.7-18.el6.x86_64
libvirt-client-0.8.7-18.el6.x86_64
1.# virt-install -n test1 -r 512 -f /var/lib/libvirt/images/test1.img -c /home/share/en_windows_xp_professional_with_service_pack_3_x86_cd_x14-80428.iso -s 6 --vnc --serial udp,host=:2222

Starting install...
ERROR    internal error process exited while connecting to monitor: getaddrinfo((null),(null)): Servname not supported for ai_socktype
inet_dgram_opts failed
chardev: opening backend "udp" failed

Domain installation does not appear to have been successful.
If it was, you can restart your domain by running:
  virsh --connect qemu:///system start test1
otherwise, please restart your installation.

2.Use the guest xml of comment 2
# virsh start foo
error: Failed to start domain foo
error: cannot send monitor command '{"execute":"qmp_capabilities"}': Connection reset by peer


Verified this issue with:
libvirt-client-0.9.4-23.el6.x86_64
libvirt-0.9.4-23.el6.x86_64
libvirt-python-0.9.4-23.el6.x86_64
1.# virt-install -n test1 -r 512 -f /var/lib/libvirt/images/test1.img -c /home/share/en_windows_xp_professional_with_service_pack_3_x86_cd_x14-80428.iso -s 6 --vnc --serial udp,host=:2222

Starting install...
Creating domain...                                       |    0 B     00:00     

Virt-viewer pop up,and the guest begin to installation.

2.Use the guest xml of comment 2.
# virsh start foo
Domain foo started

# virsh list
 Id Name                 State
----------------------------------
  1 foo                  running
  3 test1                running

According to Comment 7, move it to VERIFIED.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHSA-2012-0748.html