Bug 1112360 - kernel panic when nova launches rhel-guest-image-6-6.5 with --no-kvm
Summary: kernel panic when nova launches rhel-guest-image-6-6.5 with --no-kvm
Keywords:
Status: CLOSED WORKSFORME
Alias: None
Product: Red Hat Quickstart Cloud Installer
Classification: Red Hat
Component: Installation - RHELOSP
Version: 1.0
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
: 1.0
Assignee: Matthew Booth
QA Contact:
Dan Macpherson
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2014-06-23 17:30 UTC by Judd Maltin
Modified: 2016-04-29 16:17 UTC (History)
6 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2014-06-30 15:54:30 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
nova compute log - normal output (no debug) (17.67 KB, text/plain)
2014-06-23 18:55 UTC, Judd Maltin
no flags Details

Description Judd Maltin 2014-06-23 17:30:44 UTC
All scenarios work OK with Cirros image.  Only rhel-guest-image-6-6.5-20140422.0.el6_5.noarch fails with following kernel panic


Version-Release number of selected component (if applicable):


How reproducible:


Steps to Reproduce:
1. Install OSP with Foreman on VMWare nodes
2. Add the rhel-guest-image with glance

# glance image-create --name='rhel-image' --is-public=true --container-format=bare --disk-format=qcow2 < /usr/share/rhel-guest-image-6/rhel-guest-image-6-6.5-20140422.0-1.qcow2

3. Launch an m1.tiny RHEL image and watch it fail with error message "too small"

4. Launch an m1.small REHL image and watch it kernel-panic 

Actual results:

?Kernel panic - not syncing: Attempted to kill init!
Pid: 1, comm: init Not tainted 2.6.32-431.17.1.el6.x86_64 #1
Call Trace:
 [<ffffffff815274ef>] ? panic+0xa7/0x16f
 [<ffffffff81077292>] ? do_exit+0x862/0x870
 [<ffffffff810772f8>] ? do_group_exit+0x58/0xd0
 [<ffffffff81077387>] ? sys_exit_group+0x17/0x20
 [<ffffffff8100b072>] ? system_call_fastpath+0x16/0x1b
general protection fault: fff2 [#1] SMP 
last sysfs file: 
CPU 0 
Modules linked in:

Pid: 1, comm: init Not tainted 2.6.32-431.17.1.el6.x86_64 #1 Red Hat RHEV Hypervisor
RIP: 0010:[<ffffffff81527594>]  [<ffffffff81527594>] panic+0x14c/0x16f
RSP: 0018:ffff88007e497e48  EFLAGS: 00000246
RAX: 0000000000000000 RBX: ffffffff817b17c7 RCX: 0000000000000000
RDX: ffffffff81e31ebc RSI: 0000000000000000 RDI: 0000000000000046
RBP: ffff88007e497eb8 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000001 R11: 0000000000000000 R12: 0000000000000001
R13: ffffffff81aa4500 R14: ffff88007e495500 R15: 0000000000000000
FS:  00007f977f754700(0000) GS:ffff880002200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00007f977f251ff0 CR3: 0000000001a85000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 0000000000000000 DR7: 0000000000000000
Process init (pid: 1, threadinfo ffff88007e496000, task ffff88007e495500)
Stack:
 ffff88007e495500 0000000000000000 ffff880000000008 ffff88007e497ec8
<d> ffff88007e497e78 ffffffff81edb498 ffff88007e497eb8 0000000000000000
<d> ffff88007e480840 ffff88007e495e78 ffff88007e495e78 0000000000000000
Call Trace:
 [<ffffffff81077292>] do_exit+0x862/0x870
 [<ffffffff810772f8>] do_group_exit+0x58/0xd0
 [<ffffffff81077387>] sys_exit_group+0x17/0x20
 [<ffffffff8100b072>] system_call_fastpath+0x16/0x1b
Code: e8 e2 5f d6 ff 48 8b 45 98 48 8d 5c 03 01 69 05 97 a4 90 00 e8 03 00 00 48 98 48 39 d8 7f ca e8 93 b6 b6 ff fb 66 0f 1f 44 00 00 <31> db e8 65 ea bb ff 48 89 df ff 15 6c a4 90 00 bf 58 89 41 00 
RIP  [<ffffffff81527594>] panic+0x14c/0x16f
 RSP <ffff88007e497e48>
---[ end trace b6d4b02c0cc523eb ]---

Expected results:


Additional info:

VMWare VCenter 5.5.0 (with latest patches)

OpenStack nodes:

# rpm -qa | grep openstack
openstack-ceilometer-common-2013.2.3-1.el6ost.noarch
openstack-nova-common-2013.2.3-7.el6ost.noarch
openstack-ceilometer-compute-2013.2.3-1.el6ost.noarch
openstack-utils-2013.2-3.el6ost.noarch
openstack-nova-api-2013.2.3-7.el6ost.noarch
openstack-nova-compute-2013.2.3-7.el6ost.noarch
openstack-nova-network-2013.2.3-7.el6ost.noarch

# rpm -qa | grep qemu
qemu-img-rhev-0.12.1.2-2.415.el6_5.8.x86_64
gpxe-roms-qemu-0.9.7-6.10.el6.noarch
qemu-kvm-rhev-0.12.1.2-2.415.el6_5.8.x86_64

# uname -a
Linux openshift2.rcbd.lab 2.6.32-431.17.1.el6.x86_64 #1 SMP Fri Apr 11 17:27:00 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux

Comment 1 Stephen Gordon 2014-06-23 17:57:16 UTC
Are the logs from Nova available? Also just to confirm the scenario here is that RHELOSP incl. both compute and control planes is running *on top* of VMware?

Comment 2 Judd Maltin 2014-06-23 18:55:58 UTC
Created attachment 911547 [details]
nova compute log - normal output (no debug)

Comment 3 Judd Maltin 2014-06-23 18:59:04 UTC
Correct, compute and control planes are (RHEL 6.5) running *on top* of VMWare.  That RHEL 6.5 have OpenStack installed.  That OpenStack has the RHEL-guest-image loaded into glance, and I am attempting to launch QEMU --no-kvm with that image.  The image is indeed marked QCOW2 in Glance.

The console output seems abbreviated to me.  I'd expect more on the console than just the kernel panic.  I'm not finding a way around this.

Comment 4 Stephen Gordon 2014-06-23 20:30:55 UTC
Matt can you try and reproduce? I suspect the issue will actually need to be looked at by one of the RHEL teams but want to confirm we can reliably reproduce before handing it off.

Comment 5 Matthew Booth 2014-06-24 08:09:49 UTC
Sure. Where can I grab that image from?

Comment 6 Andrew Cathrow 2014-06-24 16:41:43 UTC
Are we trying to run a QCOW image on VMware ?

Comment 7 Judd Maltin 2014-06-24 16:43:33 UTC
No sir.  We're trying to run a QCOW2 image on QEMU on RHEL on VMWARE.

Comment 8 Judd Maltin 2014-06-24 16:44:13 UTC
rhel-guest-image is a package in the RHN.

Comment 9 Stephen Gordon 2014-06-25 12:33:24 UTC
I gave Matt the link to the guest image via IRC, apologies for the confusion.

Comment 10 Matthew Booth 2014-06-26 11:17:10 UTC
I have recreated this environment locally and I can't reproduce the kernel panic. How do you want to proceed?

Comment 11 Stephen Gordon 2014-06-26 13:22:11 UTC
Hi Judd,

Can you confirm the details of the nova boot command being used to initiate the VM?

Thanks,

Steve

Comment 12 Judd Maltin 2014-06-30 15:54:30 UTC
Please close this bug.

I redeployed my environment with precisely the same tools, re-installing RHEL on the VMware VMs in precisely the same manner, and now the QEMU --no-kvm nodes created by OpenStack work just fine.

Thanks very much for the considered attention.

Hopefully, next time a real bug.

-judd

Comment 13 Judd Maltin 2014-06-30 15:57:11 UTC
Working config:

[root@openshift2 nova]# rpm -qa | grep openstack
openstack-nova-compute-2013.2.3-7.el6ost.noarch
openstack-utils-2013.2-3.el6ost.noarch
openstack-ceilometer-common-2013.2.3-1.el6ost.noarch
openstack-ceilometer-compute-2013.2.3-1.el6ost.noarch
openstack-nova-api-2013.2.3-7.el6ost.noarch
openstack-nova-common-2013.2.3-7.el6ost.noarch
openstack-nova-network-2013.2.3-7.el6ost.noarch


rhel-guest-image-6-6.5-20140422.0.el6_5.noarch


Note You need to log in before you can comment on or make changes to this bug.