Bug 892273

i have tried several test run, and now met this problem quite often, like below,


01/10 21:06:30 INFO |guestfs_su:0686| GS:lvm partition...
01/10 21:06:31 INFO |guestfs_su:0686| GS:libguestfs: trace: add_drive "/usr/local/staf/test/RHEV/libguestfs-autotest/autotest/client/tests/libguestfs/images/fs_mount.btrfs.qcow2"
01/10 21:06:31 INFO |guestfs_su:0686| GS:libguestfs: trace: add_drive = 0
01/10 21:06:31 INFO |guestfs_su:0686| GS:libguestfs: trace: launch
01/10 21:06:31 INFO |guestfs_su:0686| GS:libguestfs: trace: get_tmpdir
01/10 21:06:31 INFO |guestfs_su:0686| GS:libguestfs: trace: get_tmpdir = "/tmp"
01/10 21:06:31 INFO |guestfs_su:0686| GS:libvir: XML-RPC error : Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused
01/10 21:06:31 INFO |guestfs_su:0686| GS:*stdin*:1: libguestfs: error: could not connect to libvirt (URI = NULL): Failed to connect socket to '/var/run/libvirt/libvirt-sock': Connection refused [code=38 domain=7]
01/10 21:06:31 INFO |guestfs_su:0686| GS:libguestfs: trace: launch = -1 (error)
01/10 21:06:31 INFO |guestfs_su:0686| GS:libguestfs: trace: close

I'm pretty sure this has got to be a libvirt bug.  After all,
we send libvirt the same XML every time!  Or perhaps qemu, if
for some reason there is a race/delay between libvirt and qemu
creating the monitor.

the libvirt version, libvirt-1.0.1-1.el7.x86_64

Hi,moli 

I can not reproduce this bug with our ENV, so if it can be fixed ,please help us to verify it ,thanks very much .


Wenlong

This problem is thought to be quite rare, so I don't think we can assume it's been fixed yet.

If libvirt's socket is not showing up, we need some libvirt logs to determine where the problem might reside.  Could you attach full debug logs from the time libvirt started without the socket?  The best thing would be trying to reproduce such start with cleared logs, so there are no other unnecessary starts logged in there.  Thank you in advance.

(In reply to comment #6)
> If libvirt's socket is not showing up, we need some libvirt logs to
> determine where the problem might reside.  Could you attach full debug logs
> from the time libvirt started without the socket?  The best thing would be
> trying to reproduce such start with cleared logs, so there are no other
> unnecessary starts logged in there.  Thank you in advance.

sorry for miss this thread, i will provide the log once i could reproduced,

Closing as INSUFFICIENT_DATA for now, feel free to reopen if there is any new info (logs etc.).

This occurs in Fedora 20, and therefore will happen in RHEL 7
too.  More information added to the upstream bug (bug 895901).

I can't reproduce this bug with steps of reporter, only hit it by the steps of bug 1048818. 

Which way is better to handle this bug, leave it here or close it and reopen again if we find the reproducer 

What's your suggestion? Rich

This is a real bug, we should leave it open.

I don't really know if this bug is the same as bug 1048818,
or different.

This is really a matter of setting a qemu monitor socket timeout.  There is a discussion in the upstream thread [1] about making the timeout configurable, but even if that's not accepted, the default might be probably changed.

Martin

[1] https://www.redhat.com/archives/libvir-list/2014-January/msg00408.html

Hopefully fixed upstream by v1.2.1-11-gfe89b68:

commit fe89b687a02d1a8e1dce695a67b4f9d2c254d7b9
Author: Martin Kletzander <mkletzan>
Date:   Thu Jan 9 07:57:59 2014 +0100

    qemu: Change the default unix monitor timeout

Reproduce this bug on libvirt.x86_64 0:1.1.1-18.el7.
1. Write a wrapper for qemu to waits several seconds (>3):
# cat /tmp/qemu-wrapper.py 
#!/usr/bin/env python

import os
import sys
import time

TIMEOUT = 5
QEMU_PATH = "/usr/libexec/qemu-kvm"

time.sleep(TIMEOUT)

sys.argv[0] = QEMU_PATH
os.execv(QEMU_PATH, sys.argv)

#ll -Z /tmp/qemu-wrapper.py 
-rwxr-xr-x. qemu qemu system_u:object_r:qemu_exec_t:s0 /tmp/qemu-wrapper.py

2. Use this wrapper as an emulator for a guest
# virsh dumpxml test|grep emulator
    <emulator>/tmp/qemu-wrapper.py</emulator>
3. Start the guest by virsh
#virsh start test
error: Failed to start domain test
error: monitor socket did not show up: No such file or directory

#tail /var/log/libvirt/libvirtd.log
2014-01-23 07:23:17.949+0000: 3842: error : qemuMonitorOpenUnix:311 : monitor socket did not show up: No such file or directory

After update libvirt to  libvirt.x86_64 0:1.1.1-19.el7
Step 3, guest can start successfully.
# virsh start test
Domain test started

The bug fix patch change the default unix monitor timeout from 3 seconds to 30 seconds, if change the TIMEOUT in the qemu wrapper to more than 30 seconds, it will fail as expected.
Move this bug to VERIFIED.

This request was resolved in Red Hat Enterprise Linux 7.0.

Contact your manager or support representative in case you have further questions about the request.