Bug 429403

Summary: virsh shutdown fails
Product: [Fedora] Fedora Reporter: Son To <son.c.to>
Component: xenAssignee: Xen Maintainance List <xen-maint>
Status: CLOSED WONTFIX QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: low Docs Contact:
Priority: low    
Version: 8CC: cowan, crobinso, williamt
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: i686   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-01-09 02:36:00 EST Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
Attachments:
Description Flags
xm list --long|
none
/var/log/xen/xend.log
none
xm list --long (when funked, for comment #7)
none
xend.log from comment #7 when virt-{manager,install} hung
none
doh... corrected content type for xend.log attachment for #7 none

Description Son To 2008-01-19 05:28:15 EST
Description of problem:
trying to shutdown a vm using virsh shutdown vmName
the vm shutdown but the virtual machine manager still shows that its running
thought it doesn't take up any resource.

virsh list shows that its shutdown, but
xm list shows that its still running

xm shutdown results in 

[root@clone xen]# xm shutdown skywalker
Error: Domain cannot be shutdown
Usage: xm shutdown <Domain> [-waRH]

the vm cannot be restarted using virsh start without restarting xend

Version-Release number of selected component (if applicable):
kernel-xen-2.6.21-2952.fc8

How reproducible:
always reproducible 

Steps to Reproduce:
1. virsh shutdown <vmName>

  
Actual results:
vm shutsdown but appears in the virtual machine manager that its running.

/var/log/xen/xend-debug.log has the following exception

Traceback (most recent call last):
  File "/usr/lib/python2.5/site-packages/xen/web/httpserver.py", line 140, in
process
    resource = self.getResource()
  File "/usr/lib/python2.5/site-packages/xen/web/httpserver.py", line 172, in
getResource
    return self.getServer().getResource(self)
  File "/usr/lib/python2.5/site-packages/xen/web/httpserver.py", line 351, in
getResource
    return self.root.getRequestResource(req)
  File "/usr/lib/python2.5/site-packages/xen/web/resource.py", line 39, in
getRequestResource
    return findResource(self, req)
  File "/usr/lib/python2.5/site-packages/xen/web/resource.py", line 26, in
findResource
    next = resource.getPathResource(pathElement, request)
  File "/usr/lib/python2.5/site-packages/xen/web/resource.py", line 49, in
getPathResource
    val = self.getChild(path, request)
  File "/usr/lib/python2.5/site-packages/xen/web/SrvDir.py", line 71, in getChild
    val = self.get(x)
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDomainDir.py", line
52, in get
    return self.domain(x)
  File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDomainDir.py", line
44, in domain
    dom = self.xd.domain_lookup(x)
  File "/usr/lib/python2.5/site-packages/xen/xend/XendDomain.py", line 524, in
domain_lookup
    raise XendInvalidDomain(str(domid))
XendInvalidDomain: <Fault 3: '2'>


Expected results:


Additional info:
Comment 1 Daniel Berrange 2008-01-19 13:23:33 EST
Please attach output of 'xm list --long', and /var/log/xen/xend.log

Also, what version of the 'xen' RPM is installed, and what kernel is running in
the guest OS.
Comment 2 Son To 2008-01-19 13:46:57 EST
Created attachment 292268 [details]
xm list --long|
Comment 3 Son To 2008-01-19 13:47:35 EST
Created attachment 292269 [details]
/var/log/xen/xend.log
Comment 4 Son To 2008-01-19 13:48:46 EST
I'm running fedora8 and winxp. same problem shutting down winxp so it appears to
be guest independent
Comment 5 Daniel Berrange 2008-01-19 15:58:33 EST
It is something wrong related to fullyvirtualized guests. You can't do a
graceful shutdown of fullyvirt guests unless there are special drivers installed
in the guest. Unfortunately there are no open source drivers for Windows. XenD
has fallback code to simply power-off the guest if it is missing drivers, and
that's what appears to be causing the error you see from 'virsh shutdown'. You
ought to be able to do a 'virsh destroy' instead.
Comment 6 William Taylor 2008-05-09 19:01:22 EDT
I am having the same problems.
Running FC8 as the host OS
and centos 5 for the guest
If I shutdown one with virsh shutdown domainname
They shutdown or poweroff but can not be restarted.
virsh list --all:
 Id Name                 State
----------------------------------
  0 Domain-0             running
  - voip1                shut off
  - voip2                no state
  - voip3                shut off
  - voip4                no state

Why does it say no state? The virtmanager shows them as running.
This is for voip2 and voip4
I tried doing a virsh destroy but I still can not restart them.
virsh start voip4
error: Domain is already active

What should I do?
Comment 7 matt cowan 2008-06-17 09:47:12 EDT
While not quite the same as what I'm experiencing, this bug seems pretty close.

When a paravirt f9 guest is shutdown on an f8 dom0 it leaves things in a funky
state: 'xm list' shows just blank space for the state of the f9 guest, 'virsh
list --all' shows the domain as shut off.

I'm trying to do testing of f9: lots of virt-install, shutdown, delete,
repeat... but whenever the guest is shutdown via seemingly any mechanism:
   -xm/virsh shutdown <dom>
   -hitting "ok" on a failed kickstart, as soon as it says "Guest installation
complete... restarting guest." (domain successfully restarts)
   -upon completion of a successful kickstart, as soon as it says "Guest
installation complete... restarting guest." (domain successfully restarts anyway)
   -'shutdown -[hr] now' inside the guest, as soon as it says "Restarting
system." (domain successfully restarts anyway (with -r))
the connection hangs in virt-manager (with it displaying the domain as
"Shutoff"), attempts to reconnect hang at "Connecting", and further attempts to
run virt-install barf (as below).

ok, gotta get rid of the remnants before I can run virt-install again with the
same mac/disk/etc...
  xm shutdown <dom> -> "Error: Domain cannot be shutdown" (ok, sure, it's
already down) 
  virsh destroy <dom> -> "error: Failed to destroy domain <dom>"
  xm destroy <dom> -> no output, but domain still shows up with empty state in
"xm list"
  'xm delete <dom>' or 'virsh undefine <dom>' both succeed in removing the
domain from the output of xm/virsh list, but that doesn't help virt-manager or
virt-install

# virt-install -p --nographics -n <dom> -r 256 -m <mac> -f <logvol> -l
http://<installhost/f[89]>/ -x "ip=<ip> netmask=<mask> gateway=<gw>
dns=<dns1[,dns2...]> kssendmac ksdevice=bootif noipv6 lang=en_US ks=<kssrc>
keymap=us"
libvir: Xen Daemon error : GET operation failed: 


Starting install...
libvir: Xen Daemon error : GET operation failed: 
Retrieving file .treeinfo 100% |=========================|  386 B    00:00     
Retrieving file vmlinuz.. 100% |=========================| 2.4 MB    00:00     
Retrieving file initrd.im 100% |=========================| 9.2 MB    00:00     
libvir: Xen Daemon error : GET operation failed: 
libvir: Xen Daemon error : GET operation failed: 
virDomainLookupByID() failed GET operation failed: 
Domain installation may not have been
 successful.  If it was, you can restart your domain
 by running 'virsh start <dom>'; otherwise, please
 restart your installation.
Thu, 12 Jun 2008 16:04:58 ERROR    virDomainLookupByID() failed GET operation
failed: 
Traceback (most recent call last):
  File "/usr/sbin/virt-install", line 502, in <module>
    main()
  File "/usr/sbin/virt-install", line 462, in main
    dom = guest.start_install(conscb,progresscb)
  File "/usr/lib/python2.5/site-packages/virtinst/Guest.py", line 813, in
start_install
    return self._do_install(consolecb, meter)
  File "/usr/lib/python2.5/site-packages/virtinst/Guest.py", line 829, in
_do_install
    self._create_devices(meter)
  File "/usr/lib/python2.5/site-packages/virtinst/Guest.py", line 727, in
_create_devices
    nic.setup(self.conn)
  File "/usr/lib/python2.5/site-packages/virtinst/Guest.py", line 281, in setup
    vm = conn.lookupByID(id)
  File "/usr/lib/python2.5/site-packages/libvirt.py", line 928, in lookupByID
    if ret is None:raise libvirtError('virDomainLookupByID() failed', conn=self)
libvirtError: virDomainLookupByID() failed GET operation failed: 

tried restarting xend, libvirtd, no luck; only solution I've found is to reboot
dom0!  I'm guessing xen is fine as any already running domains are fine, could
probably even still install new guests the oldschool xen way instead of with
virt-install/virt-manager but that's painful!  I can even still 'virsh/xm start
<dom>' any prexisting but not running domains.

With no f9 dom0, it will be important for f8 to be able to run f9 guests!  And
this is a killer for anyone who doesn't try f9 on f8 on a test box first! 
priority should be high.

how reproducable: always

steps to reproduce: make a virt-install, f9 guest on f8 dom0, once the f9 guest
shutsdown or reboots at the end of installation, tada, no more
virt-manager/install for you until you reboot.  Same thing happens after
rebooting dom0 and starting up/shutting down the f9 guest (which otherwise runs
fine).  I do not have this problem with f8 guests on this same system, they
reboot/shutdown just fine.

Using all the latest f8 dom0 stuff:
kernel-xen-2.6.21.7-3.fc8.i686.rpm
libvirt-0.4.2-1.fc8.i386.rpm
libvirt-python-0.4.2-1.fc8.i386.rpm
python-virtinst-0.300.2-4.fc8.noarch.rpm
virt-manager-0.5.3-2.fc8.i386.rpm
xen-3.1.2-2.fc8.i386.rpm
xen-libs-3.1.2-2.fc8.i386.rpm

paravirtualized guests
guest is stock f9 installer stuff
same exact issue on 2 different systems with different hardware

The only obvious errors in
/var/log/xen/{xen-hotplug,domain-builder-ng,xend,xend-debug}.log are in
xend-debug; 86 of these:
> Traceback (most recent call last):
>   File "/usr/lib/python2.5/site-packages/xen/web/httpserver.py", line 140, in
process
>     resource = self.getResource()
>   File "/usr/lib/python2.5/site-packages/xen/web/httpserver.py", line 172, in
getResource
>     return self.getServer().getResource(self)
>   File "/usr/lib/python2.5/site-packages/xen/web/httpserver.py", line 351, in
getResource
>     return self.root.getRequestResource(req)
>   File "/usr/lib/python2.5/site-packages/xen/web/resource.py", line 39, in
getRequestReso
urce
>     return findResource(self, req)
>   File "/usr/lib/python2.5/site-packages/xen/web/resource.py", line 26, in
findResource
>     next = resource.getPathResource(pathElement, request)
>   File "/usr/lib/python2.5/site-packages/xen/web/resource.py", line 49, in
getPathResourc
e
>     val = self.getChild(path, request)
>   File "/usr/lib/python2.5/site-packages/xen/web/SrvDir.py", line 71, in getChild
>     val = self.get(x)
>   File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDomainDir.py",
line 52, in ge
t
>     return self.domain(x)
>   File "/usr/lib/python2.5/site-packages/xen/xend/server/SrvDomainDir.py",
line 44, in do
main
>     dom = self.xd.domain_lookup(x)
>   File "/usr/lib/python2.5/site-packages/xen/xend/XendDomain.py", line 524, in
domain_loo
kup
>     raise XendInvalidDomain(str(domid))

The first ending with:
> XendInvalidDomain: <Fault 3: '<uuid>'>
the next with the dom name instead of uuid, and
the subsequent 84 with the domid

Comment 8 matt cowan 2008-06-17 10:38:00 EDT
Created attachment 309619 [details]
xm list --long (when funked, for comment #7)
Comment 9 matt cowan 2008-06-17 11:30:48 EDT
Created attachment 309626 [details]
xend.log from comment #7 when virt-{manager,install} hung

this chunk of the xend.log should cover running virt-install, the kickstart
failing and rebooting (bad partition info), at which point
virt-{install,manager} are toast, and me 'xm shutdown <dom>'ing the domain (it
successfully resets and starts to boot from the image installed during an
earlier trial)
Comment 10 matt cowan 2008-06-17 13:55:56 EDT
Created attachment 309648 [details]
doh... corrected content type for xend.log attachment for #7
Comment 11 matt cowan 2008-06-27 15:11:01 EDT
I still have the same problem after upgrading libvirt to:
libvirt-0.4.3-1.fc8.i386.rpm
libvirt-python-0.4.3-1.fc8.i386.rpm
Comment 12 matt cowan 2008-07-01 10:15:15 EDT
I still have the same problem after upgrading libvirt to:
Jul 01 10:01:36 Updated: libvirt - 0.4.4-1.fc8.i386
Jul 01 10:01:42 Updated: libvirt-python - 0.4.4-1.fc8.i386
Comment 13 Bug Zapper 2008-11-26 04:29:07 EST
This message is a reminder that Fedora 8 is nearing its end of life.
Approximately 30 (thirty) days from now Fedora will stop maintaining
and issuing updates for Fedora 8.  It is Fedora's policy to close all
bug reports from releases that are no longer maintained.  At that time
this bug will be closed as WONTFIX if it remains open with a Fedora 
'version' of '8'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version prior to Fedora 8's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that 
we may not be able to fix it before Fedora 8 is end of life.  If you 
would still like to see this bug fixed and are able to reproduce it 
against a later version of Fedora please change the 'version' of this 
bug to the applicable version.  If you are unable to change the version, 
please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events.  Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

The process we are following is described here: 
http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Comment 14 Cole Robinson 2008-11-26 10:22:03 EST
Hi, this bug has been sitting still for quite a while. If anyone can find a way to reliably report the initial error, please file a bug against xen. I'm pretty certain this isn't a virt-manager issue: virt-manager does all it's work through libvirt or xen. So setting to CLOSED. If I got this wrong, please reopen.
Comment 15 Cole Robinson 2008-11-26 10:41:11 EST
Ugh, I fail. Wrong bug. Moving this back to ASSIGNED. :(
Comment 16 Bug Zapper 2009-01-09 02:36:00 EST
Fedora 8 changed to end-of-life (EOL) status on 2009-01-07. Fedora 8 is 
no longer maintained, which means that it will not receive any further 
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of 
Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.