Bug 252301 - kernel-xen 2.6.21-2925.15.fc8 regressions
Summary: kernel-xen 2.6.21-2925.15.fc8 regressions
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel-xen
Version: rawhide
Hardware: i686
OS: Linux
low
low
Target Milestone: ---
Assignee: Eduardo Habkost
QA Contact: Virtualization Bugs
URL:
Whiteboard:
: 254941 335961 (view as bug list)
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2007-08-15 07:29 UTC by Adam Tkac
Modified: 2013-04-30 23:37 UTC (History)
4 users (show)

Fixed In Version: 2.6.20-2934.fc7
Clone Of:
Environment:
Last Closed: 2007-09-25 08:28:50 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)
common ops (from dmesg immediately after boot) (1.64 KB, text/plain)
2007-08-15 07:38 UTC, Adam Tkac
no flags Details
errors during paravirt start (8.81 KB, text/plain)
2007-08-23 19:57 UTC, Adam Tkac
no flags Details
dmesg from 2925.14.fc8 (22.93 KB, text/plain)
2007-08-30 10:39 UTC, Adam Tkac
no flags Details
2925.14.fc8 xm dmesg (4.66 KB, text/plain)
2007-08-30 10:40 UTC, Adam Tkac
no flags Details
2931.fc8 dmesg (22.15 KB, text/plain)
2007-08-30 10:40 UTC, Adam Tkac
no flags Details
2931.fc8 xm dmesg (4.66 KB, text/plain)
2007-08-30 10:41 UTC, Adam Tkac
no flags Details
fresh ops from up2date paravirt fc7 guest (2.66 KB, text/plain)
2007-09-06 09:08 UTC, Adam Tkac
no flags Details

Description Adam Tkac 2007-08-15 07:29:39 UTC
Description of problem:
I've updated to 2.6.21-2925.15.fc8 from 2.6.21-2925.14.fc8 and some things get
broken.
- Xserver restarts sometimes
- installation of RHEL4 fullvirt machine results in kernel faults (rhel4 guest
kernel fail)

Version-Release number of selected component (if applicable):
2.6.21-2925.15.fc8

How reproducible:
always

Comment 1 Adam Tkac 2007-08-15 07:38:40 UTC
Created attachment 161334 [details]
common ops (from dmesg immediately after boot)

Comment 2 Adam Tkac 2007-08-15 07:42:16 UTC
I will provide more debuginfo after next crashes

Comment 3 Adam Tkac 2007-08-15 10:45:15 UTC
Hm, kernel now completely hangs. After reboot nothing interesting in log. Any
hints how debug this?

Comment 4 Adam Tkac 2007-08-15 10:45:47 UTC
(In reply to comment #3)
> Hm, kernel now completely hangs. After reboot nothing interesting in log. Any
> hints how debug this?

Of course, during creating paravirt machine

Comment 5 Eduardo Habkost 2007-08-20 13:45:49 UTC
(In reply to comment #1)
> Created an attachment (id=161334) [edit]
> common ops (from dmesg immediately after boot)
> 


Do you have the remaining of the dmesg contents? Especially the ACPI and 
IRQ-related messages.

Is the TG3 network interface working, even with the warnings?


If you have the full contents of dmesg and 'xm dmesg' of both 
2.6.21-2925.14.fc8 and 2.6.21-2925.15.fc8, they will be useful for checking 
what changed from one version to another.

Comment 6 Eduardo Habkost 2007-08-20 20:39:59 UTC
A new kernel-xen was submitted including a fix on the kernel IRQ handling. 
This may help to solve some of the issues, especially the TG3 
synchronize_irq() warning.

Could you test kernel-xen-2.6-2.6.21-2931.fc8 (that is building right now and 
should go to Rawhide soon) and report if the problems remain?

Comment 7 Adam Tkac 2007-08-23 19:57:58 UTC
Created attachment 172373 [details]
errors during paravirt start

I don't have direct access to that machine now so I will test next kernel and
attach dmesg outputs during monday

Comment 8 Eduardo Habkost 2007-08-23 20:11:47 UTC
(In reply to comment #7)
> Created an attachment (id=172373) [edit]
> errors during paravirt start

Are those errors from the guest or from the dom0 kernel?

Comment 9 Eduardo Habkost 2007-08-23 20:53:39 UTC
(In reply to comment #7)
> Created an attachment (id=172373) [edit]
> errors during paravirt start

Do you have the output of 'xm dmesg' of when this error was generated? The 
hypervisor seem to have returned an error to the kernel, and probably 
additional info is on xen dmesg.

Comment 10 Adam Tkac 2007-08-30 10:38:57 UTC
I did some play with kernels 2925.14.fc8 and 2931.fc8. Kernel always hangs when
multiple domains are under load (for example multiple domains are started, yum
update on multiple domains etc). About comment #7 : If I could remember
correctly I started machine remotely (ssh + virsh) and remote machine (means
dom0, not domU) has to be rebooted so I guess problem could be with dom0.

Comment 11 Adam Tkac 2007-08-30 10:39:44 UTC
Created attachment 180541 [details]
dmesg from 2925.14.fc8

Comment 12 Adam Tkac 2007-08-30 10:40:27 UTC
Created attachment 180561 [details]
2925.14.fc8 xm dmesg

Comment 13 Adam Tkac 2007-08-30 10:40:46 UTC
Created attachment 180581 [details]
2931.fc8 dmesg

Comment 14 Adam Tkac 2007-08-30 10:41:02 UTC
Created attachment 180601 [details]
2931.fc8 xm dmesg

Comment 15 Adam Tkac 2007-09-06 09:07:16 UTC
I've removed Regression keyword because it seems problems were also in older kernel

Comment 16 Adam Tkac 2007-09-06 09:08:20 UTC
Created attachment 188501 [details]
fresh ops from up2date paravirt fc7 guest

Comment 17 Eduardo Habkost 2007-09-13 18:12:19 UTC
(In reply to comment #7)
> Created an attachment (id=172373) [edit]
> errors during paravirt start
>

The log contains:

> kernel: EIP is at xen_l1_entry_update+0x95/0xa1

I just got a xen_l2_entry_update() Oops on my test machine,
and it is related CONFIG_HIGHPTE, that is not really reliable under Xen, but 
it was enabled on Fedora. I have disabled CONFIG_HIGHPTE on Rawhide kernel-xen 
(kernel-xen-2.6-2.6.21-2937.fc8) and it will be disabled on the next F7 
kernel-xen update (kernel-xen-2.6-2.6.20-2934.fc7 or higher).

The oops from comment #16 is also a symptom of the same bug related HIGHPTE.


Could you check if kernel-xen-2.6-2.6.21-2937.fc8 is more stable, for you?

If the stability problems are solved, then we have only the tg3 
synchronize_irq() problems.

Comment 18 Adam Tkac 2007-09-17 13:15:14 UTC
I tried 2.6.21-2940.fc8 and it looks fine. Kernel has no problems under heavy
load. Only tg3 issue will be seen in dmesg output but I have no problems related
to this

Comment 19 Eduardo Habkost 2007-09-17 16:36:53 UTC
Thanks.
I will create a separated bug for the tg3 synchronize_irq() warning.

Comment 20 Eduardo Habkost 2007-09-17 16:46:15 UTC
Bug #293451 created for the tg3 synchronize_irq() warning.

I am closing this bug, as the stability problems caused by HIGHPTE are gone.

Comment 21 Fedora Update System 2007-09-18 03:13:58 UTC
kernel-xen-2.6-2.6.20-2934.fc7 has been pushed to the Fedora 7 testing repository.  If problems still persist, please make note of it in this bug report.

Comment 22 Fedora Update System 2007-09-25 08:28:47 UTC
kernel-xen-2.6-2.6.20-2934.fc7 has been pushed to the Fedora 7 stable repository.  If problems still persist, please make note of it in this bug report.

Comment 23 Eduardo Habkost 2007-10-17 12:55:04 UTC
*** Bug 335961 has been marked as a duplicate of this bug. ***

Comment 24 Eduardo Habkost 2007-10-22 14:33:34 UTC
*** Bug 254941 has been marked as a duplicate of this bug. ***


Note You need to log in before you can comment on or make changes to this bug.