Bug 610911

Summary: Host KVM MMU bug causes crashes in guest Anaconda / python programs
Product: [Fedora] Fedora Reporter: Scott Dowdle <dowdle>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: low    
Version: 13CC: amit.shah, amyagi, anton, berrange, bjrosen, clalance, dougsland, dwmw2, ehabkost, extras-orphan, gansalmon, gcosta, itamar, jaswinder, jclift, jforbes, jonathan, jpirko, kernel-maint, knoel, madhu.chinakonda, markmc, mishu, notting, ondrejj, quintela, scottt.tw, virt-maint
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: kernel-2.6.33.6-147.2.4.fc13 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-08-03 01:11:38 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
Fedora 13 patch (2.6.33 kernel) none

Description Scott Dowdle 2010-07-02 18:08:22 UTC
I've tried installing the Red Hat Enterprise Linux 6 beta 2 under KVM on Fedora 13... completely updated system.

For some reason the RHEL6 installer crashes at various places across multiple VM boots / attempts... with no specific, repeatable operation-to-bug relation.  I use Fedora 12, 13, and Windows 7 in KVM without any problems and am baffled why RHEL6 is so error prone.

At first I thought it was RHEL6 but I've seen reports from those who have witnessed this problem switch to VirtualBox and/or physical hardware and the problems disappeared.

How to repeat:

Download RHEL6 Beta 2 media (x86_64 workstation is what I was using but I've heard it pertains to the i686 release as well) and boot it up in an attempt to create a VM with it.  I have created hundreds of VMs with the same setup and RHEL6 is the only thing I've had a problem with.

I'm not even sure where to start with providing data points on this "bug"... but it is pretty easy to trigger just by trying to create a RHEL6beta2 VM.

Comment 1 Justin Clift 2010-07-02 20:55:44 UTC
This also affects me.  Using a F13 workstation, running RHEL 6 beta 2 (either x86_64 or i386) is unusably flaky under KVM. :(

Running it under VirtualBox works, as does running other guest OS's under KVM, so this appears to be a bug in KVM + RHEL 6 beta 2 not working correctly together.

I suspect it may be related to BZ# 609071 or BZ# 607650.

Comment 2 Amit Shah 2010-07-14 12:58:55 UTC
607650 talks of a host kernel problem that's resolved by upstream commit 3be2264be3c00865116f997dc53ebcc90fe7fc4b

That commit should soon find its way to stable kernels and to Fedora (or be backported).

Comment 3 Scott Dowdle 2010-07-14 16:14:47 UTC
Wow.  More and more bugs I try to browse to give me the "You are not authorized to access bug #xxxxxxx"... and that's what happens to bug #607650 that you mention. :(

Comment 4 Amit Shah 2010-07-15 03:47:39 UTC
(In reply to comment #3)
> Wow.  More and more bugs I try to browse to give me the "You are not authorized
> to access bug #xxxxxxx"... and that's what happens to bug #607650 that you
> mention. :(    

A few bugs get reported during the regular testing of upcoming Red Hat releases and are not visible to non-Red Hat people. IIRC these bugs are normally opened to all once some release milestones are met.

In any case, we've identified the bug and the commit that's needed to fix it.

I'll also update the subject line to better describe the bug.

Comment 5 Justin Clift 2010-07-16 12:27:51 UTC
Hi Scott,

Avi Kivity, the Engineer who has been fixing this problem, has created a kernel rpm for testing with.

On my F13 workstation it's working fine and RHEL 6 beta 2 inside kvm is happy.

Are you able to give this a shot as well, and let us know how it goes?

  http://justinclift.fedorapeople.org/bz610911/kernel-2.6.33.6-147.avi.fc13.x86_64.rpm
  http://justinclift.fedorapeople.org/bz610911/kernel-devel-2.6.33.6-147.avi.fc13.x86_64.rpm
  http://justinclift.fedorapeople.org/bz610911/kernel-headers-2.6.33.6-147.avi.fc13.x86_64.rpm

Comment 7 Justin Clift 2010-07-16 15:56:50 UTC
Hi Chuck, not sure.  Ask Avi or Andrea maybe?

Comment 8 Scott Dowdle 2010-07-16 16:54:46 UTC
(In reply to comment #5)
> Hi Scott,
> 
> Avi Kivity, the Engineer who has been fixing this problem, has created a kernel
> rpm for testing with.
> 
> On my F13 workstation it's working fine and RHEL 6 beta 2 inside kvm is happy.
> 
> Are you able to give this a shot as well, and let us know how it goes?
> 
>  
> http://justinclift.fedorapeople.org/bz610911/kernel-2.6.33.6-147.avi.fc13.x86_64.rpm
>  
> http://justinclift.fedorapeople.org/bz610911/kernel-devel-2.6.33.6-147.avi.fc13.x86_64.rpm
>  
> http://justinclift.fedorapeople.org/bz610911/kernel-headers-2.6.33.6-147.avi.fc13.x86_64.rpm    

Works for me.  I was able to successfully install RHEL6Beta x86_64 Workstation and ran it for a while without any problems.  I could only get 800x600 in virt-manager selecting RHEL6 as the distro profile when creating but I don't know if that is a kernel thing or not.  With other OSes, I'm able to get 1024x768.

So, the problem appears to be fixed!

Comment 9 Avi Kivity 2010-07-16 16:55:08 UTC
Those commits are important, but they don't fix the problem reported here.  I'll post patches for the Fedora kernel.

Comment 10 Avi Kivity 2010-07-16 16:55:39 UTC
Scott, thanks for testing.

Comment 11 Amit Shah 2010-07-21 12:53:46 UTC
*** Bug 616011 has been marked as a duplicate of this bug. ***

Comment 12 Amit Shah 2010-07-21 12:59:53 UTC
Avi has identified upstream commit 06f334e2b509b4c9f6c4cec7e0e56444a2730922 from kvm.git as fixing this issue:

http://git.kernel.org/?p=linux/kernel/git/avi/kvm.git;a=commitdiff;h=06f334e2b509b4c9f6c4cec7e0e56444a2730922

Please backport to the F13 kernel.

Comment 13 Avi Kivity 2010-07-21 14:36:33 UTC
Created attachment 433437 [details]
Fedora 13 patch (2.6.33 kernel)

Comment 14 Chuck Ebbert 2010-07-22 12:13:23 UTC
Patch went in 2.6.34.1-25.fc13

Will this fix make it to 2.6.35?

Comment 15 Joshua Rosen 2010-07-22 12:35:36 UTC
2.6.34.1-25.fc13 isn't in testing, is it going to be put into the F13 repos or is it just going to be available from the rawhide repo?

Comment 16 Avi Kivity 2010-07-22 14:05:18 UTC
(In reply to comment #14)
> Will this fix make it to 2.6.35?    

Yes.  I'll submit it tomorrow, with -stable updates to follow.

Comment 17 Chuck Ebbert 2010-07-22 19:29:04 UTC
(In reply to comment #15)
> 2.6.34.1-25.fc13 isn't in testing, is it going to be put into the F13 repos or
> is it just going to be available from the rawhide repo?    

You can get it from koji for now -- there are still too many bugs to keep it from going into F13 updates-testing just yet.

Comment 18 Avi Kivity 2010-07-23 05:04:20 UTC
Chuck, is it possible to push a 2.6.33 update with just the patch added on top, to avoid the long 2.6.34 cycle?  This bug is hitting many people.

Comment 19 Chuck Ebbert 2010-07-23 16:04:04 UTC
Patch added to 2.6.33.6-147.2.2.fc13

Comment 20 Fedora Update System 2010-07-24 14:11:24 UTC
kernel-2.6.33.6-147.2.4.fc13 has been submitted as an update for Fedora 13.
http://admin.fedoraproject.org/updates/kernel-2.6.33.6-147.2.4.fc13

Comment 21 Fedora Update System 2010-07-27 02:47:56 UTC
kernel-2.6.33.6-147.2.4.fc13 has been pushed to the Fedora 13 testing repository.  If problems still persist, please make note of it in this bug report.
 If you want to test the update, you can install it with 
 su -c 'yum --enablerepo=updates-testing update kernel'.  You can provide feedback for this update here: http://admin.fedoraproject.org/updates/kernel-2.6.33.6-147.2.4.fc13

Comment 22 Chuck Ebbert 2010-07-27 14:52:54 UTC
*** Bug 618227 has been marked as a duplicate of this bug. ***

Comment 23 Akemi Yagi 2010-07-29 15:19:00 UTC
I believe the problem I am/was having stems from the same cause. My RHEL6beta x86_64 KVM guest was quite unstable on a Fedora12/13 host but ran fine on a CentOS-5.5 host.  I installed the test kernel-2.6.33.6-147.2.4.fc13 on the Fedora13 host. This did not fix the problem.

Then I applied a workaround [1] (offered for RHEL6beta) to the 6beta guest and this seems to have resolved the instability issue. I am now testing the patched kernel-2.6.32-44.2.el6 on this guest (with the workaround removed).

[1] Boot/install the guest with transparent huge pages off using the kernel command line parameter:  'transparent_hugepage=never'

Comment 24 Fedora Update System 2010-08-03 01:11:15 UTC
kernel-2.6.33.6-147.2.4.fc13 has been pushed to the Fedora 13 stable repository.  If problems still persist, please make note of it in this bug report.