Bug 477005 - lockdep warnings on RHEL5.3 xen guest
Summary: lockdep warnings on RHEL5.3 xen guest
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 5
Classification: Red Hat
Component: kernel-xen
Version: 5.3
Hardware: x86_64
OS: Linux
low
medium
Target Milestone: rc
: ---
Assignee: Don Dutile (Red Hat)
QA Contact: Martin Jenner
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2008-12-18 15:56 UTC by Jeff Layton
Modified: 2014-06-18 07:38 UTC (History)
4 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2009-09-02 08:55:54 UTC
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
Proposed, to-be posted patch (6.93 KB, patch)
2009-02-23 15:13 UTC, Don Dutile (Red Hat)
no flags Details | Diff


Links
System ID Private Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2009:1243 0 normal SHIPPED_LIVE Important: Red Hat Enterprise Linux 5.4 kernel security and bug fix update 2009-09-01 08:53:34 UTC

Description Jeff Layton 2008-12-18 15:56:48 UTC
From a -128.el5 kernel (though I think the problem started in -127.el5). The message seems to reliably pop during udev init. The kernel here has some extra patches, but they're mostly VFS related stuff -- nothing that involves Xen.

------------------------[snip]-------------------------

type=1403 audit(1229615656.565:3): policy loaded auid=4294967295 ses=4294967295
input: PC Speaker as /class/input/input2
GSI 17 sharing vector 0xB1 and IRQ 17
ACPI: PCI Interrupt 0000:00:03.0[A] -> GSI 28 (level, low) -> IRQ 177
Xen version 3.1.
Hypercall area is 1 pages.
Grant table initialized
xen_mem: Initialising balloon driver.
netfront: Initialising virtual ethernet driver.
Registering block device major 3
register_blkdev: cannot get major 3 for ide
xen_blk: can't get major 3 with name ide
vbd vbd-768: 19 xlvbd_add at /local/domain/0/backend/vbd/8/768
Registering block device major 3
register_blkdev: cannot get major 3 for ide
xen_blk: can't get major 3 with name ide
vbd vbd-768: 19 xlvbd_add at /local/domain/0/backend/vbd/8/768
netfront: device eth0 has copying receive path.
XENBUS: Timeout connecting to device: device/vbd/768 (state 6)

=======================================================
[ INFO: possible circular locking dependency detected ]
2.6.18-128.el5.jtltest.57debug #1
-------------------------------------------------------
modprobe/915 is trying to acquire lock:
 (xenwatch_mutex){--..}, at: [<ffffffff88170b01>] unregister_xenbus_watch+0x10c/0x121 [xen_platform_pci]

but task is already holding lock:
 (old_style_spin_init#20){--..}, at: [<ffffffff881710ad>] xvd_dev_shutdown+0x1a/0x8a [xen_platform_pci]

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (old_style_spin_init#20){--..}:
       [<ffffffff800a7cf9>] __lock_acquire+0x9a9/0xadf
       [<ffffffff800a85ef>] lock_acquire+0x55/0x70
       [<ffffffff881711ce>] xendev_rem_add+0xb1/0x10b [xen_platform_pci]
       [<ffffffff80068576>] _spin_lock+0x26/0x52
       [<ffffffff881711ce>] xendev_rem_add+0xb1/0x10b [xen_platform_pci]
       [<ffffffff8819f3eb>] xlvbd_add+0x48a/0x4af [xen_vbd]
       [<ffffffff800a251d>] keventd_create_kthread+0x0/0xc9
       [<ffffffff8819edbc>] backend_changed+0xb0/0x192 [xen_vbd]
       [<ffffffff88172129>] xenbus_read_driver_state+0x26/0x3b [xen_platform_pci]
       [<ffffffff88170e4d>] xenwatch_thread+0x0/0x140 [xen_platform_pci]
       [<ffffffff8817027f>] xenwatch_handle_callback+0x15/0x48 [xen_platform_pci]
       [<ffffffff88170f74>] xenwatch_thread+0x127/0x140 [xen_platform_pci]
       [<ffffffff800a2734>] autoremove_wake_function+0x0/0x2e
       [<ffffffff800a251d>] keventd_create_kthread+0x0/0xc9
       [<ffffffff80034306>] kthread+0xfe/0x132
       [<ffffffff8006808c>] trace_hardirqs_on_thunk+0x35/0x37
       [<ffffffff80061079>] child_rip+0xa/0x11
       [<ffffffff8006898d>] _spin_unlock_irq+0x24/0x27
       [<ffffffff800606a8>] restore_args+0x0/0x30
       [<ffffffff80034208>] kthread+0x0/0x132
       [<ffffffff8006106f>] child_rip+0x0/0x11
       [<ffffffffffffffff>] 0xffffffffffffffff

-> #0 (xenwatch_mutex){--..}:
       [<ffffffff800a7c0f>] __lock_acquire+0x8bf/0xadf
       [<ffffffff88170b01>] unregister_xenbus_watch+0x10c/0x121 [xen_platform_pci]
       [<ffffffff800a85ef>] lock_acquire+0x55/0x70
       [<ffffffff88170b01>] unregister_xenbus_watch+0x10c/0x121 [xen_platform_pci]
       [<ffffffff800672c6>] mutex_lock_nested+0x104/0x29c
       [<ffffffff88170b01>] unregister_xenbus_watch+0x10c/0x121 [xen_platform_pci]
       [<ffffffff88171298>] free_otherend_watch+0x14/0x27 [xen_platform_pci]
       [<ffffffff88171f9f>] xenbus_dev_remove+0x19/0x46 [xen_platform_pci]
       [<ffffffff801c5247>] __device_release_driver+0xa2/0xc6
       [<ffffffff801c555b>] device_release_driver+0x3d/0x5f
       [<ffffffff801c4a0c>] bus_remove_device+0x9b/0xb0
       [<ffffffff801c36a2>] device_del+0x13a/0x1ba
       [<ffffffff801c373f>] device_unregister+0x9/0x12
       [<ffffffff881710e8>] xvd_dev_shutdown+0x55/0x8a [xen_platform_pci]
       [<ffffffff88190014>] xlblk_init+0x14/0x18 [xen_vbd]
       [<ffffffff800adf3f>] sys_init_module+0xb1/0x1ee
       [<ffffffff80060116>] system_call+0x7e/0x83
       [<ffffffffffffffff>] 0xffffffffffffffff

other info that might help us debug this:

1 lock held by modprobe/915:
 #0:  (old_style_spin_init#20){--..}, at: [<ffffffff881710ad>] xvd_dev_shutdown+0x1a/0x8a [xen_platform_pci]

stack backtrace:

Call Trace:
 [<ffffffff800a65f8>] print_circular_bug_tail+0x65/0x6e
 [<ffffffff800a7c0f>] __lock_acquire+0x8bf/0xadf
 [<ffffffff88170b01>] :xen_platform_pci:unregister_xenbus_watch+0x10c/0x121
 [<ffffffff800a85ef>] lock_acquire+0x55/0x70
 [<ffffffff88170b01>] :xen_platform_pci:unregister_xenbus_watch+0x10c/0x121
 [<ffffffff800672c6>] mutex_lock_nested+0x104/0x29c
 [<ffffffff88170b01>] :xen_platform_pci:unregister_xenbus_watch+0x10c/0x121
 [<ffffffff88171298>] :xen_platform_pci:free_otherend_watch+0x14/0x27
 [<ffffffff88171f9f>] :xen_platform_pci:xenbus_dev_remove+0x19/0x46
 [<ffffffff801c5247>] __device_release_driver+0xa2/0xc6
 [<ffffffff801c555b>] device_release_driver+0x3d/0x5f
 [<ffffffff801c4a0c>] bus_remove_device+0x9b/0xb0
 [<ffffffff801c36a2>] device_del+0x13a/0x1ba
 [<ffffffff801c373f>] device_unregister+0x9/0x12
 [<ffffffff881710e8>] :xen_platform_pci:xvd_dev_shutdown+0x55/0x8a
 [<ffffffff88190014>] :xen_vbd:xlblk_init+0x14/0x18
 [<ffffffff800adf3f>] sys_init_module+0xb1/0x1ee
 [<ffffffff80060116>] system_call+0x7e/0x83

Comment 1 Chris Lalancette 2009-01-09 11:11:01 UTC
Don is working on some patches for 5.4 that should remove this code path completely.  Assuming that all works out, this should go away.  We'll leave it open until then, though, just to make sure.

Chris Lalancette

Comment 3 Don Dutile (Red Hat) 2009-02-23 15:13:42 UTC
Created attachment 332936 [details]
Proposed, to-be posted patch

Attached is proposed patch.
brew built.
based on backport of 4.8.

This *removes* the hacks for anaconda installation failures when
xen-platform-pci.ko was built into the (bare-metal, FV) kernel.
With xen-platform-pci.ko now a loadable, and not part of the
5.3-anaconda initrd, the anaconda failure cannot occur.

The removal of the hacky code gets rid of the debug-kernel warning,
which pops up due to the extra code.

Will report back on testing results later today.

Comment 4 Don Dutile (Red Hat) 2009-02-23 23:20:34 UTC
Tested the following with the patch & all worked well:
(a) x86_64 FV; block attach & detach multiple times
(b) x86_64 pv kernel; block attach & detach multiple times
(c) i686 FV kernel
(d) x86_64 -debug kernel -- none of the debug messages listed in Description of problem / bug

Comment 5 Don Dutile (Red Hat) 2009-02-23 23:47:41 UTC
Patch in c#3 posted for 5.4 inclusion.

Comment 6 Don Zickus 2009-03-04 20:00:45 UTC
in kernel-2.6.18-133.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5

Please do NOT transition this bugzilla state to VERIFIED until our QE team
has sent specific instructions indicating when to do so.  However feel free
to provide a comment indicating that this fix has been verified.

Comment 9 errata-xmlrpc 2009-09-02 08:55:54 UTC
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2009-1243.html


Note You need to log in before you can comment on or make changes to this bug.