Red Hat Bugzilla – Bug 251669
Updating dom0 userland from 5.0 to 5.1 beta problems
Last modified: 2010-03-14 17:31:18 EDT
I've updated some RHEL5 x86_64 systems to 5.1 beta, without rebooting. They're
still running kernel 2.6.18-8.1.8.el5xen.
After the updates have been applied, xend doesn't seem to have been restarted
automatically, but some things already fail :
- xentop reports "Failed to retrieve statistics from libxenstat"
- xm commands seem to still work (console, shutdown, create)
After restarting xend, everything I normally use starts to fail :
- xm commands report "Error: Unable to connect to xend: Connection refused. Is
- xentop still reports the same error
The xend python processes are running, but xend.log contains :
[2007-08-10 12:59:42 xend 3241] INFO (SrvDaemon:190) Xend stopped due to signal 15.
[2007-08-10 12:59:46 xend 15807] INFO (SrvDaemon:283) Xend Daemon started
[2007-08-10 12:59:46 xend 15807] INFO (SrvDaemon:287) Xend changeset: unavailable .
[2007-08-10 12:59:46 xend 15807] ERROR (SrvDaemon:297) Exception starting xend
((13, 'Permission denied'))
Traceback (most recent call last):
File "/usr/lib64/python2.4/site-packages/xen/xend/server/SrvDaemon.py", line
291, in run
servers = SrvServer.create()
File "/usr/lib64/python2.4/site-packages/xen/xend/server/SrvServer.py", line
108, in create
File "/usr/lib64/python2.4/site-packages/xen/xend/server/SrvRoot.py", line 40,
File "/usr/lib64/python2.4/site-packages/xen/web/SrvDir.py", line 82, in get
val = val.getobj()
File "/usr/lib64/python2.4/site-packages/xen/web/SrvDir.py", line 52, in getobj
self.obj = klassobj()
line 39, in __init__
self.xd = XendDomain.instance()
File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomain.py", line 655, in
File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomain.py", line 76, in init
File "/usr/lib64/python2.4/site-packages/xen/xend/XendDomain.py", line 139, in
domlist = xc.domain_getinfo()
Error: (13, 'Permission denied')
And xend-debug.log contains something other than "Nothing to flush." for the
first time ever :
domctl operation failed -- need to rebuild the user-space tool set?
Exception starting xend: (13, 'Permission denied')
Rebooting dom0 to a 5.1 beta kernel (2.6.18-36xen here) fixed the issues,
unfortunately once I get to the situation above, I can no longer "save" the
running domUs nor have xendomains do it automatically, there is no other
solution than to shutdown cleanly domUs by logging into them individually, or
have them destroyed upon dom0's shutdown.
There is no mention of this in the 5.1 beta RELEASE-NOTES-U1-en file, so I would
think it either needs to be fixed/prevented somehow, or at least be mentioned in
the final release notes.
FWIW, rebooting to the old 2.6.18-8.1.8.el5xen kernel doesn't work, only
rebooting to the newest kernels. Something needs to be done to :
- Enforce the userland/kernel sync
- Make error messages clearer about the incompatibility
- Document as required in the 5.1 release notes
As the xen version stayed 3.0.3, I wouldn't have expected this to happen...
Updating from RHEL-5.0 to 5.1 requires a reboot if running Xen. This is because
the hypervisor in 5.1 is not ABI compatible with the hypervisor in 5.0. This in
turn requires that your running kernel match the corresponding Xen RPM.
Its impossible to enforce this in the RPM dependancies because we you can
obviously reboot between 5.0 and 5.1 kernels at will. So this needs to be in the
Reassigning for release-noting.
I did reboot.
However, the grub.conf had default=1, pointing to the old
xen.gz-2.6.18-8.1.4.el5, not to the new xen.gz-2.6.18-53.el5.
So the release note should also note that the user should make sure that the new
kernel was really booted.
This "bites" quite often for systems installed as normal servers, then changed
to use a Xen kernel. The /etc/sysconfig/kernel file still references the plain
"kernel" instead of "kernel-xen" to be the default new kernel.
Thanks Jan, Matthias. Editing as follows:
If you are using the Virtualized kernel when upgrading from Red Hat Enterprise
Linux 5 to 5.1, you must reboot after completing the upgrade. You should then
boot the system using the updated Virtualized kernel.
The hypervisors of Red Hat Enterprise Linux 5 and 5.1 are not ABI-compatible. If
you do not boot the system after upgrading using the updated Virtualized kernel,
the upgraded Virtualization RPMs will not match the running kernel.
closing thsi bug, since RHEL5.1 release notes already released.
Tracking this bug for the Red Hat Enterprise Linux 5.3 Release Notes.
Release note added. If any revisions are required, please set the
"requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.