Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 825992

Summary: Guest stuck at "Wait For Launch"
Product: Red Hat Enterprise Linux 6 Reporter: Chao Yang <chayang>
Component: vdsmAssignee: Dan Kenigsberg <dkenigsb>
Status: CLOSED NOTABUG QA Contact: Haim <hateya>
Severity: high Docs Contact:
Priority: unspecified    
Version: 6.3CC: abaron, bazulay, danken, hateya, iheim, yeylon, ykaul
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard: virt
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2012-06-07 09:48:39 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
libvirt and vdsm log none

Description Chao Yang 2012-05-29 09:37:34 UTC
Created attachment 587369 [details]
libvirt and vdsm log

Description of problem:
I created two VMs from existing templates, then click on start, but they get stuck at "Wait for Launch". Threre is io error as well as libvirt error in vdsm.log:
IOError: [Errno 2] No such file or directory: '/dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06' but I checked, this file exists:
# ls -l /dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06
lrwxrwxrwx. 1 root root 8 May 29 14:13 /dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06 -> ../dm-10


Version-Release number of selected component (if applicable):
qemu-kvm-rhev-0.12.1.2-2.294.el6.x86_64
libvirt-0.9.10-20.el6.x86_64
vdsm-4.9.6-13.el6.x86_64


How reproducible:


Steps to Reproduce:
1.
2.
3.
  
Actual results:


Expected results:


Additional info:

Comment 2 Haim 2012-05-29 11:31:11 UTC
it appears that vdsm fails to communicate with libvirt; what is the status of the service ? does virhs -r capabilities works ? does libvirt service restart works for you ?

Comment 3 Chao Yang 2012-05-29 11:51:31 UTC
(In reply to comment #2)
> it appears that vdsm fails to communicate with libvirt; what is the status
> of the service ? does virhs -r capabilities works ? does libvirt service
> restart works for you ?

vdsmd and libvirtd are running. virsh -r capabilities works. restart libvirt service works, the guests in "Wait For Launch" status previously are now running.

Comment 4 Haim 2012-05-29 12:58:10 UTC
(In reply to comment #3)
> (In reply to comment #2)
> > it appears that vdsm fails to communicate with libvirt; what is the status
> > of the service ? does virhs -r capabilities works ? does libvirt service
> > restart works for you ?
> 
> vdsmd and libvirtd are running. virsh -r capabilities works. restart libvirt
> service works, the guests in "Wait For Launch" status previously are now
> running.

we need to understand what caused vdsm to loose connection to libvirt service, do you have a reproducer ?

Comment 5 Dan Kenigsberg 2012-05-29 21:32:25 UTC
What's the version of libvirt? I recall that there was a recenet rpc bug in libvirt that blocked SI4 build.

Comment 6 Chao Yang 2012-05-30 05:21:49 UTC
(In reply to comment #5)
> What's the version of libvirt? I recall that there was a recenet rpc bug in
> libvirt that blocked SI4 build.

libvirt-0.9.10-20.el6.x86_64

Comment 7 Chao Yang 2012-05-30 05:38:35 UTC
(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > it appears that vdsm fails to communicate with libvirt; what is the status
> > > of the service ? does virhs -r capabilities works ? does libvirt service
> > > restart works for you ?
> > 
> > vdsmd and libvirtd are running. virsh -r capabilities works. restart libvirt
> > service works, the guests in "Wait For Launch" status previously are now
> > running.
> 
> we need to understand what caused vdsm to loose connection to libvirt
> service, do you have a reproducer ?

Not sure if it is a reliable reproducer, I just started VM from a template, then it happened.

Comment 8 Itamar Heim 2012-05-30 11:51:06 UTC
> lrwxrwxrwx. 1 root root 8 May 29 14:13 /dev/f84b4e6a-f186-490c-9478-483261e15682
> /2438c346-9769-48f6-b861-372f94f91a06 -> ../dm-10"

permissions here are root/root. iirc they should be vdsm.
how did you get to this?
though it probably should have failed runnning the guest, rather than hang it.

Comment 10 Chao Yang 2012-06-01 09:07:12 UTC
(In reply to comment #8)
> > lrwxrwxrwx. 1 root root 8 May 29 14:13 /dev/f84b4e6a-f186-490c-9478-483261e15682
> > /2438c346-9769-48f6-b861-372f94f91a06 -> ../dm-10"
> 
> permissions here are root/root. iirc they should be vdsm.
> how did you get to this?
> though it probably should have failed runnning the guest, rather than hang
> it.

I don't know, I just created an iscsi storage from WPFClient, then New VMs on this storage.  ls tells this file is owned by root but getfact tells vdsm. 

# ls -l /dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06lrwxrwxrwx. 1 root root 8 May 29 14:13 /dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06 -> ../dm-10

# getfacl /dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06
getfacl: Removing leading '/' from absolute path names
# file: dev/f84b4e6a-f186-490c-9478-483261e15682/2438c346-9769-48f6-b861-372f94f91a06
# owner: vdsm
# group: qemu
user::rw-
group::rw-
other::---

Comment 11 Dan Kenigsberg 2012-06-03 15:06:20 UTC
(In reply to comment #6)
> (In reply to comment #5)
> > What's the version of libvirt? I recall that there was a recenet rpc bug in
> > libvirt that blocked SI4 build.
> 
> libvirt-0.9.10-20.el6.x86_64

please try -21, I suspect a dup of bug 821468

Comment 12 Chao Yang 2012-06-04 01:55:40 UTC
(In reply to comment #11)
> (In reply to comment #6)
> > (In reply to comment #5)
> > > What's the version of libvirt? I recall that there was a recenet rpc bug in
> > > libvirt that blocked SI4 build.
> > 
> > libvirt-0.9.10-20.el6.x86_64
> 
> please try -21, I suspect a dup of bug 821468

I have already upgraded to -21. If still reproducible, will update here.

Comment 13 Itamar Heim 2012-06-07 09:48:39 UTC
chayang - please re-open if reproduced with -21 libvirt