Bug 1196934

Summary: libvirt sometimes output useless error when qemu failed to start
Product: Red Hat Enterprise Linux 7 Reporter: Luyao Huang <lhuang>
Component: libvirtAssignee: Ján Tomko <jtomko>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: low Docs Contact:
Priority: low    
Version: 7.1CC: dyuan, jtomko, mzhan, rbalakri, zhwang
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.2.14-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-19 06:18:29 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Luyao Huang 2015-02-27 03:15:00 UTC
Description of problem:
libvirt sometimes output useless error when qemu failed to start/migrate/restore

Version-Release number of selected component (if applicable):
libvirt-1.2.8-16.el7_1.1.x86_64

How reproducible:
100%

Steps to Reproduce:
1.prepare a vm have a settings will make qemu fail to start (i use -m rhel6.5.0 with -numa memdev as a example)

# virsh dumpxml test3
  <memoryBacking>
    <hugepages>
      <page size='2048' unit='KiB' nodeset='0'/>
    </hugepages>
  </memoryBacking>
...
  <os>
    <type arch='x86_64' machine='rhel6.5.0'>hvm</type>
    <boot dev='hd'/>
  </os>
...
  <cpu>
    <numa>
      <cell id='0' cpus='0,2' memory='512000'/>
      <cell id='1' cpus='1,3' memory='512000'/>
    </numa>
  </cpu>
...
2.start it multi times, will get different error:
[root@lhuang ~]# virsh start test3
error: Failed to start domain test3
error: internal error: early end of file from monitor: possible problem:
2015-02-27T03:03:08.570226Z qemu-kvm: -numa memdev is not supported by machine rhel6.5.0


[root@lhuang ~]# virsh start test3
error: Failed to start domain test3
error: operation failed: domain is no longer running

[root@lhuang ~]# virsh start test3
error: Failed to start domain test3
error: operation failed: domain is no longer running

[root@lhuang ~]# virsh start test3
error: Failed to start domain test3
error: operation failed: domain is no longer running

[root@lhuang ~]# virsh start test3
error: Failed to start domain test3
error: internal error: early end of file from monitor: possible problem:
2015-02-27T03:03:16.985494Z qemu-kvm: -numa memdev is not supported by machine rhel6.5.0

...

3.

Actual results:
libvirt sometimes output useless error when qemu failed to start/migrate/restore
try a lot of times we will get two kind of error:
error: Failed to start domain test3
error: internal error: early end of file from monitor: possible problem:
2015-02-27T03:03:16.985494Z qemu-kvm: -numa memdev is not supported by machine rhel6.5.0

error: Failed to start domain test3
error: operation failed: domain is no longer running

and the first one is catch from $vmname.log
and the second one is introduced in libvirt-1.2.8-14.el7

Expected results:
if qemu process failed to start we will catch a log from /var/log/libvirt/qemu/vmname.log (if there is a log), this will be useful to the users, however after libvirt-1.2.8-14.el7 we have a new error, which will cover the clearly error sometimes when we start a qemu process.

Additional info:

we have fixed useless error output when start vm fail before(in bug 765733 for rhel7.1).

Comment 1 Ján Tomko 2015-03-19 09:28:50 UTC
Proposed patch:
https://www.redhat.com/archives/libvir-list/2015-March/msg00978.html

Pushed as:
commit 4f068209387118f9b67c7cfe1a6c0097d7f573e7
Author:     Luyao Huang <lhuang>
AuthorDate: 2015-03-19 11:14:39 +0800
Commit:     Ján Tomko <jtomko>
CommitDate: 2015-03-19 10:17:27 +0100

    qemu: do not overwrite the error in qemuDomainObjExitMonitor
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1196934
    
    When qemu exits during startup, libvirt includes the error from
    /var/log/libvirt/qemu/vm.log in the error message:
    
    $ virsh start test3
    error: Failed to start domain test3
    error: internal error: early end of file from monitor: possible problem:
    2015-02-27T03:03:16.985494Z qemu-kvm: -numa memdev is not supported by
    machine rhel6.5.0
    
    The check for domain liveness added to qemuDomainObjExitMonitor
    in commit dc2fd51f sometimes overwrites this error:
    $ virsh start test3
    error: Failed to start domain test3
    error: operation failed: domain is no longer running
    
    Fix the check to only report an error if there is none set.
    
    Signed-off-by: Luyao Huang <lhuang>
    Signed-off-by: Ján Tomko <jtomko>

git describe: v1.2.13-246-g4f06820

Comment 3 vivian zhang 2015-06-26 02:07:34 UTC
I can produce this with build libvirt-1.2.8-16.el7_1.1.x86_64

Verify this with build libvirt-1.2.16-1.el7.x86_64

1. prepare a vm have a settings will make qemu fail to start (i use -m rhel6.5.0 with -numa memdev as a example)
<memoryBacking>
<hugepages>
<page size='2048' unit='KiB' nodeset='0'/>
</hugepages>

<os>
<type arch='x86_64' machine='rhel6.5.0'>hvm</type>
<boot dev='hd'/>
</os>

<cpu mode='custom' match='exact'>
<model fallback='allow'>Penryn</model>
<numa>
<cell id='0' cpus='0,2' memory='512000' unit='KiB'/>
<cell id='1' cpus='1,3' memory='512000' unit='KiB'/>
</numa>
</cpu>

2. start guest, always start guest failed with right error
[root@client 1.2.16-1.el7]# virsh start rh7
error: Failed to start domain rh7
error: internal error: early end of file from monitor: possible problem:
2015-06-18T06:07:04.056584Z qemu-kvm: -numa memdev is not supported by machine rhel6.5.0

[root@client 1.2.16-1.el7]# virsh start rh7
error: Failed to start domain rh7
error: internal error: early end of file from monitor: possible problem:
2015-06-18T06:07:05.203086Z qemu-kvm: -numa memdev is not supported by machine rhel6.5.0

[root@client 1.2.16-1.el7]# virsh start rh7
error: Failed to start domain rh7
error: internal error: early end of file from monitor: possible problem:
2015-06-18T06:07:06.443335Z qemu-kvm: -numa memdev is not supported by machine rhel6.5.0

[root@client 1.2.16-1.el7]# virsh start rh7
error: Failed to start domain rh7
error: internal error: early end of file from monitor: possible problem:
2015-06-18T06:07:07.733511Z qemu-kvm: -numa memdev is not supported by machine rhel6.5.0

[root@client 1.2.16-1.el7]# virsh start rh7
error: Failed to start domain rh7
error: internal error: early end of file from monitor: possible problem:
2015-06-18T06:07:09.071946Z qemu-kvm: -numa memdev is not supported by machine rhel6.5.0

Comment 4 vivian zhang 2015-06-29 08:55:41 UTC
When qemu failed to migrate/restore, can not produce libvirt output useless error. So change this bug summary, and move to verified

Comment 6 errata-xmlrpc 2015-11-19 06:18:29 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html