614272 – reboot guest cause call trace

RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 614272 - reboot guest cause call trace

Summary: reboot guest cause call trace

Keywords:
Status:	CLOSED DUPLICATE of bug 608613
Alias:	None
Product:	Red Hat Enterprise Linux 6
Classification:	Red Hat
Component:	qemu-kvm
Sub Component:
Version:	6.0
Hardware:	All
OS:	Linux
Priority:	low
Severity:	medium
Target Milestone:	rc
Target Release:	---
Assignee:	Marcelo Tosatti
QA Contact:	Virtualization Bugs
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	580953
TreeView+	depends on / blocked

Reported:	2010-07-14 01:53 UTC by Suqin Huang
Modified:	2013-01-09 22:51 UTC (History)
CC List:	6 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed:	2010-07-27 00:15:46 UTC
Target Upstream Version:
Embargoed:

Attachments	(Terms of Use)
kernel panic (19.64 KB, image/png) 2010-07-19 06:00 UTC, Suqin Huang	no flags	Details
View All

Description Suqin Huang 2010-07-14 01:53:21 UTC

Description of problem:
boot rhel5 guest, update to newest kernel, them reboot guest.

Version-Release number of selected component (if applicable):
qemu-kvm-0.12.1.2-2.93.el6.x86_64

How reproducible:
1/50

Steps to Reproduce:
1. boot guest
/usr/qemu -name 'vm1' -monitor stdio\
 -drive file='/usr/images/RHEL-Server-5.5-64-virtio.qcow2',if=none,id=drive-virtio-disk1,media=disk,cache=none,boot=on,format=qcow2 \
-device virtio-blk-pci,drive=drive-virtio-disk1,id=virtio-disk1 -net nic,vlan=0,netdev=braQ,model=virtio,macaddr='02:30:0C:D2:41:d9' \
-netdev tap,id=braQ,ifname=virtio_0_8000,script=/scripts/qemu-ifup-switch,downscript=no,vhost=on -m 2048 -smp 2 -vnc :0 -rtc base=utc,clock=host -M rhel6.0.0 -usbdevice tablet -cpu qemu64,+sse2 -no-kvm-pit-reinjection 

2. update kernel to kernel-2.6.18-206.el5
3. reboot guest
  
Actual results:


Expected results:


Additional info:
1. host

2.6.32-44.el6.x86_64
processor	: 3
vendor_id	: AuthenticAMD

flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nonstop_tsc extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs npt lbrv svm_lock

2. guest
rhel5-64, (kernel-2.6.18-206.el5)

3. call trace

CPU 1 
Modules linked in: autofs4 hidp rfcomm l2cap bluetooth lockd sunrpc ip_conntrack_netbios_ns ipt_REJECT xt_state ip_conntrack nfnetlink iptable_filter ip_tables ip6t_REJECT xt_tcpudp ip6table_filter ip6_tables x_tables dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec dell_wmi wmi button battery asus_acpi acpi_memhotplug ac ipv6 xfrm_nalgo crypto_api lp floppy joydev parport_pc parport pcspkr i2c_piix4 i2c_core ide_cd cdrom serio_raw virtio_net dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod ata_piix libata sd_mod scsi_mod virtio_blk virtio_pci virtio_ring virtio ext3 jbd uhci_hcd ohci_hcd ehci_hcd
Pid: 2503, comm: udevd Not tainted 2.6.18-206.el5 #1
RIP: 0010:[<ffffffff80064b0b>]  [<ffffffff80064b0b>] _spin_lock_irqsave+0x3/0xd
RSP: 0000:ffff810070a8be40  EFLAGS: 00010096
RAX: 0000000000000296 RBX: ffff81007fa9b128 RCX: 0000000000000000
RDX: ffff810070a8bef8 RSI: 000000000000000d RDI: ffff81007fa9b12c
RBP: ffff81007fa9b12c R08: 00007fff3d5e0480 R09: 00002b7c324bc2b8
R10: 00007fff3d5e02f0 R11: 00000000ffffffff R12: 00002b7c32b30dba
R13: 0000000000000004 R14: ffff810070a8bf58 R15: ffff810073335100
FS:  00002b7c33077710(0000) GS:ffff81007ff91840(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00002b7c32b30dba CR3: 000000007d953000 CR4: 00000000000006e0
Process udevd (pid: 2503, threadinfo ffff810070a8a000, task ffff810073335100)
Stack:  ffffffff8000ba3f 0000000000000000 ffff81007fa9b128 ffff81007fa9b0c0
 ffffffff800671af ffff81000253eaa0 ffffffff8005dde9 ffff81000253eaa0
 00002b7c00030001 0000000000400100 ffff810037dbb820 ffff810070a8bf48
Call Trace:
 [<ffffffff8000ba3f>] __down_read_trylock+0x15/0x44

Comment 2 Amit Shah 2010-07-15 11:30:59 UTC

Can you please give the entire panic message?

Does the guest hang after this message or does it continue fine?

I've seen a similar oops message recently, so it might be a dupe. (And it also might be a guest kernel issue rather than a host one.)

Comment 3 RHEL Program Management 2010-07-15 14:17:27 UTC

This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release. It has
been denied for the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **

Comment 4 Amit Shah 2010-07-15 14:34:02 UTC

Re-setting needinfo flag.

Comment 5 Suqin Huang 2010-07-16 03:08:34 UTC

it hang after this message.



Linux version 2.6.18-206.el5 (mockbuild.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Thu Jul 8 18:00:54 EDT 2010
Command line: ro root=/dev/VolGroup00/LogVol00 rhgb console=ttyS0,115200 console=tty0
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000010000 - 000000000009cc00 (usable)
 BIOS-e820: 000000000009cc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000007fffb000 (usable)
 BIOS-e820: 000000007fffb000 - 0000000080000000 (reserved)
 BIOS-e820: 00000000fffbc000 - 0000000100000000 (reserved)
DMI 2.4 present.
kvm-clock: cpu 0, msr 7eff:80492401, boot clock
No NUMA configuration found
Faking a node at 0000000000000000-000000007fffb000
Bootmem setup node 0 0000000000000000-000000007fffb000
Memory for crash kernel (0x0 to 0x0) notwithin permissible range
disabling kdump
ACPI: PM-Timer IO Port: 0xb008
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
Processor #0 6:2 APIC version 20
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
Processor #1 6:2 APIC version 20
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, version 17, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
Setting APIC routing to flat
Using ACPI (MADT) for SMP configuration information
Nosave address range: 000000000009c000 - 000000000009d000
Nosave address range: 000000000009d000 - 00000000000a0000
Nosave address range: 00000000000a0000 - 00000000000f0000
Nosave address range: 00000000000f0000 - 0000000000100000
Allocating PCI resources starting at 88000000 (gap: 80000000:7ffbc000)
SMP: Allowing 2 CPUs, 0 hotplug CPUs
kvm-clock: cpu 0, msr 0:2535401, primary cpu clock
Built 1 zonelists.  Total pages: 515616
Kernel command line: ro root=/dev/VolGroup00/LogVol00 rhgb console=ttyS0,115200 console=tty0
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
time.c: Using tsc for timekeeping HZ 1000
Console: colour VGA+ 80x25
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Checking aperture...
ACPI: DMAR not present
Memory: 2055664k/2097132k available (2583k kernel code, 41004k reserved, 1636k data, 212k init)
Calibrating delay loop (skipped), value calculated using timer frequency.. 5812.79 BogoMIPS (lpj=2906398)
Security Framework v1.0.0 initialized
SELinux:  Initializing.
selinux_register_security:  Registering secondary module capability
Capability LSM initialized as secondary
Mount-cache hash table entries: 256
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
CPU 0/0 -> Node 0
SMP alternatives: switching to UP code
ACPI: Core revision 20060707
Using local APIC timer interrupts.
WARNING calibrate_APIC_clock: the APIC timer calibration may be wrong.
Detected 62.507 MHz APIC timer.
SMP alternatives: switching to SMP code
Booting processor 1/2 APIC 0x1
Initializing CPU#1
calibrate_delay_direct() failed to get a good estimate for loops_per_jiffy.
Probably due to long platform interrupts. Consider using "lpj=" boot option.
Calibrating delay loop... 16777.21 BogoMIPS (lpj=8388608)
CPU: L1 I Cache: 64K (64 bytes/line), D cache 64K (64 bytes/line)
CPU: L2 Cache: 512K (64 bytes/line)
CPU 1/1 -> Node 0
QEMU Virtual CPU version 0.12.1 stepping 03
kvm-clock: cpu 1, msr 0:253da81, secondary cpu clock
CPU 1: Syncing TSC to CPU 0.
CPU 1: synchronized TSC with CPU 0 (last diff -37 cycles, maxerr 714 cycles)
Brought up 2 CPUs
testing NMI watchdog ... <4>WARNING: CPU#0: NMI appears to be stuck (0->0)!
time.c: Using 1.193182 MHz WALL KVM GTOD KVM timer.
time.c: Detected 2906.398 MHz processor.
migration_cost=329
checking if image is initramfs... it is
Freeing initrd memory: 3337k freed
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: Using configuration type 1
mtrr: your CPUs had inconsistent variable MTRR settings
mtrr: your CPUs had inconsistent MTRRdefType settings
mtrr: probably your BIOS does not setup all CPUs.
mtrr: corrected configuration.
ACPI: Interpreter enabled
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI quirk: region b000-b03f claimed by PIIX4 ACPI
PCI quirk: region b100-b10f claimed by PIIX4 SMB
ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
pnp: PnP ACPI: found 8 devices
usbcore: registered new driver usbfs
usbcore: registered new driver hub
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
ACPI: DMAR not present
PCI-GART: No AMD northbridge found.
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
TCP established hash table entries: 262144 (order: 10, 4194304 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 262144 bind 65536)
TCP reno registered
audit: initializing netlink socket (disabled)
type=2000 audit(1279020391.237:1): initialized
Total HugeTLB memory allocated, 0
VFS: Disk quotas dquot_6.5.1
Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
Initializing Cryptographic API
alg: No test for crc32c (crc32c-generic)
ksign: Installing public key data
Loading keyring
- Added public key 4C7CC1AEDED9466B
- User ID: Red Hat, Inc. (Kernel Module GPG key)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
Limiting direct PCI/PCI transfers.
PCI: PIIX3: Enabling Passive Release on 0000:00:01.0
Activating ISA DMA hang workarounds.
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
Real Time Clock Driver v1.12ac
hpet_acpi_add: no address or irqs in _CRS
Non-volatile memory driver v1.2
Linux agpgart interface v0.101 (c) Dave Jones
Serial: 8250/16550 driver $Revision: 1.90 $ 4 ports, IRQ sharing enabled
�serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:06: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
brd: module loaded
Uniform Multi-Platform E-IDE driver Revision: 7.00alpha2
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PIIX3: IDE controller at PCI slot 0000:00:01.1
PIIX3: chipset revision 0
PIIX3: not 100% native mode: will probe irqs later
    ide0: BM-DMA at 0xc000-0xc007, BIOS settings: hda:pio, hdb:pio
    ide1: BM-DMA at 0xc008-0xc00f, BIOS settings: hdc:pio, hdd:pio
hdc: QEMU DVD-ROM, ATAPI CD/DVD-ROM drive
ide1 at 0x170-0x177,0x376 on irq 15
ide-floppy driver 0.99.newide
usbcore: registered new driver hiddev
usbcore: registered new driver usbhid
drivers/usb/input/hid-core.c: v2.6:USB HID core driver
PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
md: bitmap version 4.39
TCP bic registered
Initializing IPsec netlink socket
input: AT Translated Set 2 keyboard as /class/input/input0
NET: Registered protocol family 1
NET: Registered protocol family 17
ACPI: (supports S3 S4 S5)
Initalizing network drop monitor service
Freeing unused kernel memory: 212k freed
Write protecting the kernel read-only data: 515k
USB Universal Host Controller Interface driver v3.0
ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
ACPI: PCI Interrupt 0000:00:01.2[D] -> Link [LNKD] -> GSI 11 (level, high) -> IRQ 11
uhci_hcd 0000:00:01.2: UHCI Host Controller
uhci_hcd 0000:00:01.2: new USB bus registered, assigned bus number 1
uhci_hcd 0000:00:01.2: irq 11, io base 0x0000c020
usb usb1: configuration #1 chosen from 1 choice
hub 1-0:1.0: USB hub found
hub 1-0:1.0: 2 ports detected
ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10
ACPI: PCI Interrupt 0000:00:03.0[A] -> <6>usb 1-1: new full speed USB device using uhci_hcd and address 2
Link [LNKC] -> GSI 10 (level, high) -> IRQ 10
ACPI: PCI Interrupt 0000:00:04.0[A] -> Link [LNKD] -> GSI 11 (level, high) -> IRQ 11
 vda: vda1 vda2
SCSI subsystem initialized
input: ImExPS/2 Generic Explorer Mouse as /class/input/input1
usb 1-1: configuration #1 chosen from 1 choice
device-mapper: uevent: version 1.0.3
input: QEMU 0.12.1 QEMU USB Tablet as /class/input/input2
device-mapper: ioctl: 4.11.5-ioctl (2007-12-12) initialised: dm-devel
input: USB HID v0.01 Pointer [QEMU 0.12.1 QEMU USB Tablet] on usb-0000:00:01.2-1
device-mapper: dm-raid45: initialized v0.2594l
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
SELinux:  Disabled at runtime.
type=1404 audit(1279020416.886:2): selinux=0 auid=4294967295 ses=4294967295
hdc: drive_cmd: status=0x41 { DriveReady Error }
hdc: drive_cmd: error=0x04 { AbortedCommand }
ide: failed opcode was: 0xec
invalid opcode: 0000 [1] SMP 
last sysfs file: /class/misc/autofs/dev
CPU 1 
Modules linked in: autofs4 hidp rfcomm l2cap bluetooth lockd sunrpc ip_conntrack_netbios_ns ipt_REJECT xt_state ip_conntrack nfnetlink iptable_filter ip_tables ip6t_REJECT xt_tcpudp ip6table_filter ip6_tables x_tables dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec dell_wmi wmi button battery asus_acpi acpi_memhotplug ac ipv6 xfrm_nalgo crypto_api lp floppy joydev parport_pc parport pcspkr i2c_piix4 i2c_core ide_cd cdrom serio_raw virtio_net dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapshot dm_zero dm_mirror dm_log dm_mod ata_piix libata sd_mod scsi_mod virtio_blk virtio_pci virtio_ring virtio ext3 jbd uhci_hcd ohci_hcd ehci_hcd
Pid: 2503, comm: udevd Not tainted 2.6.18-206.el5 #1
RIP: 0010:[<ffffffff80064b0b>]  [<ffffffff80064b0b>] _spin_lock_irqsave+0x3/0xd
RSP: 0000:ffff810070a8be40  EFLAGS: 00010096
RAX: 0000000000000296 RBX: ffff81007fa9b128 RCX: 0000000000000000
RDX: ffff810070a8bef8 RSI: 000000000000000d RDI: ffff81007fa9b12c
RBP: ffff81007fa9b12c R08: 00007fff3d5e0480 R09: 00002b7c324bc2b8
R10: 00007fff3d5e02f0 R11: 00000000ffffffff R12: 00002b7c32b30dba
R13: 0000000000000004 R14: ffff810070a8bf58 R15: ffff810073335100
FS:  00002b7c33077710(0000) GS:ffff81007ff91840(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00002b7c32b30dba CR3: 000000007d953000 CR4: 00000000000006e0
Process udevd (pid: 2503, threadinfo ffff810070a8a000, task ffff810073335100)
Stack:  ffffffff8000ba3f 0000000000000000 ffff81007fa9b128 ffff81007fa9b0c0
 ffffffff800671af ffff81000253eaa0 ffffffff8005dde9 ffff81000253eaa0
 00002b7c00030001 0000000000400100 ffff810037dbb820 ffff810070a8bf48
Call Trace:
 [<ffffffff8000ba3f>] __down_read_trylock+0x15/0x44

Comment 6 Dor Laor 2010-07-18 12:35:50 UTC

We're still missing the trace after the line below:
 [<ffffffff8000ba3f>] __down_read_trylock+0x15/0x44

Comment 7 Suqin Huang 2010-07-19 06:00:37 UTC

Created attachment 432764 [details]
kernel panic

kernel panic when I try to reproduce this issue.

Comment 8 Marcelo Tosatti 2010-07-22 00:20:39 UTC

0xffffffff80064b0b <__raw_spin_lock+0>: lock decl (%rdi)

0xffffffff80064b16 <__raw_spin_lock+0>: lock decl (%rdi)

This is likely a duplicate of https://bugzilla.redhat.com/show_bug.cgi?id=615925.

Was NPT disabled during the tests?

Comment 9 Suqin Huang 2010-07-22 02:08:01 UTC

npt is enabled during the test

Comment 10 Marcelo Tosatti 2010-07-22 12:15:45 UTC

Suqin,

Please try the kernel from

https://bugzilla.redhat.com/show_bug.cgi?id=608613#c35

Should probably see a different oops message with it.

Comment 11 Suqin Huang 2010-07-23 05:31:15 UTC

Marcelo,
the package is removed

Comment 12 Suqin Huang 2010-07-23 10:46:02 UTC

repeat 50 times, can not reproduce

Comment 13 Amit Shah 2010-07-23 11:31:16 UTC

What were the changes you made in your config so that it's not reproducible anymore? Updated kernel in guest / host?

This can now be closed, right?

Comment 14 RHEL Program Management 2010-07-23 13:57:42 UTC

This issue has been proposed when we are only considering blocker
issues in the current Red Hat Enterprise Linux release.

** If you would still like this issue considered for the current
release, ask your support representative to file as a blocker on
your behalf. Otherwise ask that it be considered for the next
Red Hat Enterprise Linux release. **

Comment 15 Suqin Huang 2010-07-23 17:37:46 UTC

I test the kernel from https://bugzilla.redhat.com/show_bug.cgi?id=608613#c37

Comment 16 Marcelo Tosatti 2010-07-27 00:15:46 UTC


*** This bug has been marked as a duplicate of bug 608613 ***

Note You need to log in before you can comment on or make changes to this bug.