Bug 249887 - xen kernel dies on F7 BUG: spinlock lockup on CPU#0, swapper/0, c1335c00 (not tainted)
Summary: xen kernel dies on F7 BUG: spinlock lockup on CPU#0, swapper/0, c1335c00 (not...
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel-xen
Version: 7
Hardware: i386
OS: Linux
low
low
Target Milestone: ---
Assignee: Eduardo Habkost
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2007-07-27 17:45 UTC by Paul Van Allsburg
Modified: 2009-12-14 20:41 UTC (History)
2 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2008-06-17 01:59:23 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
console messages containing crash information (39.28 KB, application/octet-stream)
2007-09-27 18:23 UTC, Paul Van Allsburg
no flags Details
crash log of origional F7 released xen kernel (62.66 KB, text/plain)
2007-09-28 18:34 UTC, Paul Van Allsburg
no flags Details
xen successful boot log (23.86 KB, text/plain)
2007-10-02 14:58 UTC, Paul Van Allsburg
no flags Details
yum updates today caused xen kernel to crash (35.41 KB, text/plain)
2007-10-08 16:33 UTC, Paul Van Allsburg
no flags Details

Description Paul Van Allsburg 2007-07-27 17:45:40 UTC
Description of problem:
Performed a yum update, after reboot Xen kernel halts on with: BUG: spinlock
lockup on CPU#0, swapper/0, c1335c00 (not tainted)

Version-Release number of selected component (if applicable):
Trying to boot to these kernels:
2.6.22.1-27.fc7       boots successfully
2.6.20-2925.11.f7xen  fails with error msg above
2.6.21-1.3228.fc7     boots successfully
2.6.20-2925.9.fc7xen  fails with error msg above


How reproducible:
The xen kernels were booting fine until I did a yum update on 7/26 of:
Jul 26 07:46:16 cream yum: Updated: cairo.i386 1.4.10-1.fc7
Jul 26 07:46:17 cream yum: Updated: fuse-libs.i386 2.7.0-3.fc7
Jul 26 07:46:18 cream yum: Updated: bind-libs.i386 9.4.1-7.P1.fc7
Jul 26 07:46:19 cream yum: Updated: libXi.i386 1.1.1-1.fc7
Jul 26 07:46:53 cream yum: Installed: kernel.i686 2.6.22.1-27.fc7
Jul 26 07:46:55 cream yum: Updated: fuse.i386 2.7.0-3.fc7
Jul 26 07:46:55 cream yum: Updated: gnome-python2-extras.i386 2.14.3-4.fc7
Jul 26 07:47:36 cream yum: Updated: gnome-utils.i386 2.18.1-2.fc7
Jul 26 07:47:43 cream yum: Updated: evolution-data-server.i386 1.10.3.1-2.fc7
Jul 26 07:47:46 cream yum: Updated: selinux-policy.noarch 2.6.4-28.fc7
Jul 26 07:47:48 cream yum: Updated: gnome-python2-libegg.i386 2.14.3-4.fc7
Jul 26 07:47:49 cream yum: Updated: ntfs-3g.i386 1.710-1.fc7
Jul 26 07:47:50 cream yum: Updated: mdadm.i386 2.6.2-4.fc7
Jul 26 07:47:50 cream yum: Updated: iwlwifi-firmware.noarch 2.14.4-1
Jul 26 07:47:52 cream yum: Updated: ed.i386 0.7-1.fc7
Jul 26 07:48:01 cream yum: Updated: gdm.i386 2.18.3-1.fc7
Jul 26 07:48:46 cream yum: Updated: selinux-policy-targeted.noarch 2.6.4-28.fc7
Jul 26 07:48:48 cream yum: Updated: autofs.i386 5.0.1-20
Jul 26 07:48:48 cream yum: Updated: bind-utils.i386 9.4.1-7.P1.fc7
Jul 26 07:49:20 cream yum: Updated: gedit.i386 2.18.2-1.fc7
Jul 26 07:49:21 cream yum: Updated: gnome-python2-gtkhtml2.i386 2.14.3-4.fc7

Steps to Reproduce:
1. Xen kernels fail at every boot.  nothing is captured in the message log.
2.
3.
  
Actual results:


Expected results:


Additional info:
I tried adding to the boot linux ide=nodma with no success at boot time.

   Product: ProLiant DL380 G2
Processor 1 (Pentium III(R) at 1.133 GHz)
  Numeric Coprocessor (387 at 1.133 GHz)
  Secondary Cache (512K)
Processor 2 (Pentium III(R) at 1.133 GHz)
  Numeric Coprocessor (387 at 1.133 GHz)
  Secondary Cache (512K)
Memory - 1280 Megabytes
Keyboard (101-102 Key)
Pointing Device Interface
  with Pointing Device attached
Graphics Controller(s)
 ATI RAGE XL Graphics Controller
Diskette Drive A: - 1.44 Megabyte (3.5 inch)
1 Serial Interface(s)
  COM1 (Address 3F8) at IRQ4
Embedded IDE Controller, Base Address 0x1f0
  Compaq 24X CD-ROM, Drive position 1
    (COMPAQ  CD-ROM CRN-8241B, REV 2.23)
Drive Controller 1, Compaq SMART Array 5i
Drive Controller 2, Compaq SMART Array 5300
   Hard Disk Drive 1 - (36.4G)  Raid 5
   Hard Disk Drive 2 - (36.4G)  Raid 5
   Hard Disk Drive 3 - (36.4G)  Raid 5

Comment 1 Eduardo Habkost 2007-09-25 14:27:07 UTC
Could you attach or paste the full BUG message output?

If no log file is generated at the time of the crash, a picture of the crash 
screen would be useful, too.

Also, could you confirm if the latest kernel-xen that went on Fedora 7 updates 
still have the problem?

Comment 2 Paul Van Allsburg 2007-09-27 17:29:14 UTC
Updated kernel 2.6.20-2934.fc7xen fails with 

Bug: unable to handle kernel paging request at virtual address
recursive die()failure
Oops! 0002[#5]
<1> Fixing recursive fault but reboot is needed

I setup tty console hoping to trap console messages but it does not work with
xen kernel.  console=tty0 works fine with 2.6.22.7-85.fc7 and server boots
successfully.  

How can I get console messages out of xen kernel?

grub.conf is:


default=0
timeout=9
#splashimage=(hd0,0)/grub/splash.xpm.gz
serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1
terminal --timeout=15 serial console
hiddenmenu
title Fedora (2.6.22.7-85.fc7)
        root (hd0,0)
        kernel /vmlinuz-2.6.22.7-85.fc7 ro root=/dev/VolGroup00/LogVol00
console=tty0 console=ttyS0,115200n8 rhgb quiet
        initrd /initrd-2.6.22.7-85.fc7.img
title Fedora (2.6.20-2934.fc7xen)
        root (hd0,0)
        kernel /xen.gz-2.6.20-2934.fc7
        module /vmlinuz-2.6.20-2934.fc7xen ro root=/dev/VolGroup00/LogVol00
console=tty0 console=ttyS0,115200n8 rhgb quiet
        module /initrd-2.6.20-2934.fc7xen.img


Comment 3 Eduardo Habkost 2007-09-27 17:50:47 UTC
The serial port is managed by Xen. To get a serial console, this should work:

title Fedora (2.6.20-2934.fc7xen)
        root (hd0,0)
        kernel /xen.gz-2.6.20-2934.fc7 com1=115200,8n1
        module /vmlinuz-2.6.20-2934.fc7xen ro root=/dev/VolGroup00/LogVol00
console=tty0 console=ttyS0,115200 rhgb quiet
        module /initrd-2.6.20-2934.fc7xen.img

Comment 4 Paul Van Allsburg 2007-09-27 18:23:56 UTC
Created attachment 208881 [details]
console messages containing crash information

Comment 5 Eduardo Habkost 2007-09-27 19:06:41 UTC
The log has:
4gb seg fixup, process init (pid 1), cs:ip 73:0096c9bd


The warning isn't supposed to cause problems, but it is better to be sure that 
the problem isn't caused by this. Your initrd was not generated with the 
nosegeg libraries (because bug #244730).

I will clone bug #244730 for Fedora 7, but while it is not fixed, you can 
regenerate your initrd, but do it while running a Xen kernel (you can use the 
previous xen kernel that was working on your system) to make sure the nosegneg 
libraries are copied:

# mkinitrd -v -f /boot/initrd-2.6.20-2934.fc7xen.img 2.6.20-2934.fc7xen

You should see a line like this on the mkinitrd output:
/lib/i686/nosegneg/libc.so.6 -> /tmp/initrd.GH5960/lib/libc.so.6


If you see this instead:
/lib/libc.so.6 -> /tmp/initrd.GH5960/lib/libc.so.6

Then the wrong libc file is being copied to initrd. This shouldn't happen.

Comment 6 Paul Van Allsburg 2007-09-28 18:34:17 UTC
Created attachment 210781 [details]
crash log of origional F7 released xen kernel

Comment 7 Paul Van Allsburg 2007-09-28 18:39:56 UTC
I don's have a xen kernel that will boot, both

2.6.20-2934.fc7xen and
2.6.20-2931.fc7xen

both fail.  
So I used my kickstart file and reloaded F7 from the origional dvd
setup the console log and booted - the xen kernel crashed  and rebooted itself,
the next time xen loaded it crashed and halted with the spinlock error I reported.
see the attachment curie-f7.txt


Comment 8 Paul Van Allsburg 2007-10-02 14:57:00 UTC
I performed a full update and got 2.6.20-2936.fc7xen along with other updates. 
This xen kernel boots(attached cream_xen_oct2.txt) Successfully! The log still
shows:
4gb seg fixup, process init (pid 1), cs:ip 73:004249bd
do I still need to be concerned with the mkinitrd shown above?

Comment 9 Paul Van Allsburg 2007-10-02 14:58:16 UTC
Created attachment 213541 [details]
xen successful boot log

Comment 10 Paul Van Allsburg 2007-10-08 16:33:50 UTC
Created attachment 219871 [details]
yum updates today caused xen kernel to crash

I allowed the following suggested updates today and my xen kernels will not
boot. Updates were were:

Oct  8 08:48:55 cream yum: Updated: dbus-glib.i386 0.73-3.fc7
Oct  8 08:49:02 cream yum: Updated: libpurple.i386 2.2.1-1.fc7
Oct  8 08:49:04 cream yum: Updated: libselinux.i386 2.0.14-9.fc7
Oct  8 08:49:05 cream yum: Updated: xen-libs.i386 3.1.0-6.fc7
Oct  8 08:49:06 cream yum: Updated: fuse-libs.i386 2.7.0-6.fc7
Oct  8 08:49:11 cream yum: Updated: selinux-policy.noarch 2.6.4-45.fc7
Oct  8 08:49:12 cream yum: Updated: libnetfilter_conntrack.i386 0.0.81-1.fc7
Oct  8 08:49:19 cream yum: Updated: xen.i386 3.1.0-6.fc7
Oct  8 08:49:20 cream yum: Updated: fuse.i386 2.7.0-6.fc7
Oct  8 08:49:45 cream yum: Updated: fast-user-switch-applet.i386 2.17.4-5.fc7
Oct  8 08:50:11 cream yum: Updated: pidgin.i386 2.2.1-1.fc7
Oct  8 08:50:15 cream yum: Updated: qemu.i386 0.9.0-3.fc7
Oct  8 08:50:42 cream yum: Updated: selinux-policy-targeted.noarch 2.6.4-45.fc7

Oct  8 08:50:51 cream yum: Updated: tzdata.noarch 2007h-1.fc7
Oct  8 08:50:52 cream yum: Updated: wodim.i386 1.1.6-5.fc7
Oct  8 08:50:53 cream yum: Updated: kvm.i386 35-3.fc7
Oct  8 08:50:54 cream yum: Updated: libselinux-python.i386 2.0.14-9.fc7
Oct  8 08:50:55 cream yum: Updated: alsa-utils.i386 1.0.14-2.fc7
Oct  8 08:50:58 cream yum: Updated: genisoimage.i386 1.1.6-5.fc7
Oct  8 08:50:58 cream yum: Updated: rsync.i386 2.6.9-3.fc7
Oct  8 08:50:59 cream yum: Updated: xorg-x11-drv-amd.i386 0.0-22.20070625.fc7
I attached grub.conf / this update list & boot log in that atachment
Thanks
Paul

Comment 11 Bug Zapper 2008-05-14 13:42:43 UTC
This message is a reminder that Fedora 7 is nearing the end of life. Approximately 30 (thirty) days from now Fedora will stop maintaining and issuing updates for Fedora 7. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '7'.

Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 7's end of life.

Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 7 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora please change the 'version' of this bug. If you are unable to change the version, please add a comment here and someone will do it for you.

Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. If possible, it is recommended that you try the newest available Fedora distribution to see if your bug still exists.

Please read the Release Notes for the newest Fedora distribution to make sure it will meet your needs:
http://docs.fedoraproject.org/release-notes/

The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping

Comment 12 Bug Zapper 2008-06-17 01:59:21 UTC
Fedora 7 changed to end-of-life (EOL) status on June 13, 2008. 
Fedora 7 is no longer maintained, which means that it will not 
receive any further security or bug fix updates. As a result we 
are closing this bug. 

If you can reproduce this bug against a currently maintained version 
of Fedora please feel free to reopen this bug against that version.

Thank you for reporting this bug and we are sorry it could not be fixed.


Note You need to log in before you can comment on or make changes to this bug.