Bug 198866

Summary: xend service hangs at startup
Product: [Fedora] Fedora Reporter: James Hunt <jamesodhunt>
Component: xenAssignee: Xen Maintainance List <xen-maint>
Status: CLOSED INSUFFICIENT_DATA QA Contact: Brian Brock <bbrock>
Severity: medium Docs Contact:
Priority: medium    
Version: 5CC: bstein, clalance, katzj
Target Milestone: ---   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2007-03-09 14:13:05 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description James Hunt 2006-07-14 09:47:21 UTC
Description of problem:

Having installed xen, and rebooted into xen kernel, the system hangs at startup
on the "xend" service.

Version-Release number of selected component (if applicable):

bridge-utils-1.0.6-1.2
kernel-xen0-2.6.17-1.2145_FC5
python-2.4.3-8.FC5
vnc-4.1.1-39.fc5
xen-3.0.2-3.FC5

How reproducible:

Every time.

Steps to Reproduce:
1. Boot kernel-xen0-2.6.17-1.2145_FC5

Actual results:

Hangs at the "Starting xend:" stage.

Expected results:

A full boot, and xend starting.

Additional info:

Booting interactively, bypassing xend and xendomains services, and then
attempting to start xend manually results in a hang too.

The file /var/log/xend-debug.log is interesting:

_____________________________________________________________
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap
    self.run()
  File "/usr/lib/python2.4/threading.py", line 422, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.4/site-packages/xen/xend/server/XMLRPCServer.py", line
94, in run
    self.server = UnixXMLRPCServer(XML_RPC_SOCKET, False)
  File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line 152, in
__init__
    os.makedirs(parent, stat.S_IRWXU)
  File "/usr/lib/python2.4/os.py", line 159, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/var/run/xend'
_____________________________________________________________

OK, so it cannot create /var/run/xend. If you do a mkdir as root, and even a
chmod 777 on /var/run/xend, running "service xend start" again, you get:

_____________________________________________________________

Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap
    self.run()
  File "/usr/lib/python2.4/threading.py", line 422, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.4/site-packages/xen/xend/server/XMLRPCServer.py", line
94, in run
    self.server = UnixXMLRPCServer(XML_RPC_SOCKET, False)
  File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line 147, in
__init__
    os.chown(parent, os.geteuid(), os.getegid())
OSError: [Errno 13] Permission denied: '/var/run/xend'

name, mode)
OSError: [Errno 13] Permission denied: '/var/run/xend'
_____________________________________________________________

Comment 1 Jeremy Katz 2006-07-18 20:28:33 UTC
Can you also do 'restorecon /var/run/xend'? 

Comment 2 James Hunt 2006-08-12 10:19:04 UTC
Did that, and then booted kernel 2.6.17-1.2174_FC5xen0. I started the xend
service manually, and got the following in /var/log/xend-debug.log:

___________________________________________
Exception in thread Thread-3:
Traceback (most recent call last):
  File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap
    self.run()
  File "/usr/lib/python2.4/threading.py", line 422, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/lib/python2.4/site-packages/xen/xend/server/XMLRPCServer.py", line
94, in run
    self.server = UnixXMLRPCServer(XML_RPC_SOCKET, False)
  File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line 152, in
__init__
    os.makedirs(parent, stat.S_IRWXU)
  File "/usr/lib/python2.4/os.py", line 159, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/var/run/xend'

 MiB, but dom0_min_mem is 256 and shrinking to 256 MiB would leave only 2263 MiB
free.
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py", line
77, in op_create
    dominfo = self.xd.domain_create(config)
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line 226, in
domain_create
    dominfo = XendDomainInfo.create(config)
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 182,
in create
    vm.initDomain()
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1203,
in initDomain
    balloon.free(m + fudge)
  File "/usr/lib/python2.4/site-packages/xen/xend/balloon.py", line 114, in free
    raise VmError(
VmError: I need 513 MiB, but dom0_min_mem is 256 and shrinking to 256 MiB would
leave only 2263 MiB free.
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py", line
77, in op_create
    dominfo = self.xd.domain_create(config)
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line 226, in
domain_create
    dominfo = XendDomainInfo.create(config)
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 182,
in create
    vm.initDomain()
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1203,
in initDomain
    balloon.free(m + fudge)
  File "/usr/lib/python2.4/site-packages/xen/xend/balloon.py", line 114, in free
    raise VmError(
VmError: I need 513 MiB, but dom0_min_mem is 256 and shrinking to 256 MiB would
leave only 2263 MiB free.
Traceback (most recent call last):
  File "/usr/lib/python2.4/site-packages/xen/xend/server/SrvDomainDir.py", line
77, in op_create
    dominfo = self.xd.domain_create(config)
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line 226, in
domain_create
    dominfo = XendDomainInfo.create(config)
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 182,
in create
    vm.initDomain()
  File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1203,
in initDomain
    balloon.free(m + fudge)
  File "/usr/lib/python2.4/site-packages/xen/xend/balloon.py", line 114, in free
    raise VmError(
VmError: I need 513 MiB, but dom0_min_mem is 256 and shrinking to 256 MiB would
leave only 2263 MiB free.
___________________________________________

Note, however that /var/run/xend/xmlrpc.sock *was* created. Also note the
potentially dubious "balloon.free(m + fudge)".

Note also that we have that ol' chestnut bug 189328 rearing its head at the end
of the log output (my machine has 2.5Gb RAM, so the error is clearly wrong).

I don't know what is going on, but I have never got Xen to work on any of my
machines. I really hope Xen improves, but I'm starting to concur with Mr
Pinchev's recent comments on Xen - this is definately not ready for
primetime/RHEL... :-(

Comment 3 Chris Lalancette 2006-10-01 15:52:56 UTC
As a quick test, can you try disabling SELinux completely (selinux=0 on the
kernel command line) and try starting xend again?  The policy on FC5 may not be
completely right (which is a bit of an understatement).

Thanks!
Chris Lalancette

Comment 4 Stephen Tweedie 2007-03-09 14:13:05 UTC
Closed due to lack of information; please reopen if this is still an issue.