673058 – kernel panic in pg_init_done - pgpath already deleted

Bug 673058 - kernel panic in pg_init_done - pgpath already deleted

Summary: kernel panic in pg_init_done - pgpath already deleted

Keywords:
Status:	CLOSED ERRATA
Alias:	None
Product:	Red Hat Enterprise Linux 5
Classification:	Red Hat
Component:	kernel
Sub Component:
Version:	5.5
Hardware:	x86_64
OS:	Linux
Priority:	urgent
Severity:	urgent
Target Milestone:	rc
Target Release:	5.7
Assignee:	Mike Snitzer
QA Contact:	Gris Ge
Docs Contact:
URL:
Whiteboard:
Depends On:
Blocks:	683443
TreeView+	depends on / blocked

Reported:	2011-01-27 08:10 UTC by Menny Hamburger
Modified:	2011-11-08 14:46 UTC (History)
CC List:	13 users (show)
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:	A race could occur when an internal multipath structure (pgpath) was freed before it was used to signal the path group initialization was complete (via pg_init_done). This update includes a number of fixes that address this issue. multipath is now increasingly robust when multipathd restarts are combined with I/O operations to multipath devices and storage failures.
Clone Of:
Environment:
Last Closed:	2011-07-21 09:27:31 UTC
Target Upstream Version:
Embargoed:
Dependent Products:

Attachments	(Terms of Use)
Fix panic in pg_init_done called from within send_mode_select (1.68 KB, patch) 2011-01-30 08:52 UTC, Menny Hamburger	no flags	Details \| Diff
Add a wait for pg init completion in multipath destructor (1.38 KB, patch) 2011-02-06 11:18 UTC, Menny Hamburger	no flags	Details \| Diff
Flush workqueues on postsuspend as well as at the destructor (1.02 KB, patch) 2011-02-06 11:21 UTC, Menny Hamburger	no flags	Details \| Diff
Show Obsolete (1) View All

Links
System	ID	Private	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHSA-2011:1065	0	normal	SHIPPED_LIVE	Important: Red Hat Enterprise Linux 5.7 kernel security and bug fix update	2011-07-21 09:21:37 UTC

Description Menny Hamburger 2011-01-27 08:10:55 UTC

The machine is running with multipath over MD3xxx storage (RDAC).

Our scripts perform a series of multipathd restarts, udevtrigger and udevsettle calls, during which we logout of the storage.

The following panic appears from time to time:
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-11: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: lun5: event checker started
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-11: add map (uevent)
2011 Jan 26 22:27:01 node0 MAJOR: multipathd: dm-11: devmap already registered
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-11: add map (uevent)
2011 Jan 26 22:27:01 node0 MAJOR: multipathd: dm-11: devmap already registered
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-11: add map (uevent)
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 MAJOR: multipathd: dm-11: devmap already registered
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-0: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath0: event checker started
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-1: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath1: event checker started
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-2: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath2: event checker started
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-3: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath3: event checker started
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-4: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath4: event checker started
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: lun5: event checker exit
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-5: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath6: event checker started
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-6: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath7: event checker started
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-7: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath8: event checker started
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-8: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath9: event checker started
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-9: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath10: event checker started
2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-10: add map (uevent)
2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath11: event checker started
2011 Jan 26 22:27:02 node0 INFO: kernel: sd 194:0:0:4: rdac: array MD3200i-c7-d7, ctlr 0, queueing MODE_SELECT command
2011 Jan 26 22:27:02 node0 DEBUG: hba_loader:iscsi_logout:1130: Started
2011 Jan 26 22:27:02 node0 DEBUG: hba_loader:iscsi_logout:1130: iscsi_luns_num=1 logout_iqn=
2011 Jan 26 22:27:02 node0 DEBUG: hba_loader:iscsi_logout:1130: Performing logout of all iSCSI nodes
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:0: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:1: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:0: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:2: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:3: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:1: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:4: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:2: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:5: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:0: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: multipathd: sdb: remove path (uevent)
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:3: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:1: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:4: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:2: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:3: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:5: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:4: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:6: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:5: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:6: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:7: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:7: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:8: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:8: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:9: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:9: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:6: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:10: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:7: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:8: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:11: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:9: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:10: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:11: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:0: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:10: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:1: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:2: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:3: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:4: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:5: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:6: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:7: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:8: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:9: rdac Dettached
2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:10: rdac Dettached
2011 Jan 26 22:27:02 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:02 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type
2011 Jan 26 22:27:02 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:02 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:02 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type
2011 Jan 26 22:27:02 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:02 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:02 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type
2011 Jan 26 22:27:02 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:02 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:02 node0 MAJOR: multipathd: mpath8: failed in domap for removal of path sdb
2011 Jan 26 22:27:02 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:02 node0 NOTICE: multipathd: sdd: remove path (uevent)
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type
2011 Jan 26 22:27:02 node0 MAJOR: multipathd: mpath9: failed in domap for removal of path sdd
2011 Jan 26 22:27:02 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:02 node0 NOTICE: multipathd: sdh: remove path (uevent)
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath8: failed in domap for removal of path sdh
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdf: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath10: failed in domap for removal of path sdf
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdo: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath9: failed in domap for removal of path sdo
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdj: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: scsi 194:0:0:8: rejecting I/O to dead device
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: multipath: Failing path 65:112.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: scsi 195:0:0:8: rejecting I/O to dead device
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: multipath: Failing path 65:160.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: multipath: Cannot failover device because scsi_dh_rdac was not loaded.
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: multipath: Failing path 66:208.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: multipath: Cannot failover device because scsi_dh_rdac was not loaded.
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: multipath: Failing path 66:224.
2011 Jan 26 22:27:03 node0 WARNING: kernel: printk: 46 messages suppressed.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 0
2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 1
2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 2
2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 3
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 0
2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 70778879
2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 70778879
2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 0
2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 0
2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 3
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath11: failed in domap for removal of path sdj
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdl: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath3: failed in domap for removal of path sdl
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: 65:160: mark as failed
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: mpath0: remaining active paths: 3
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: 65:112: mark as failed
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: mpath0: remaining active paths: 2
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: 66:208: mark as failed
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: mpath0: remaining active paths: 1
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: 66:224: mark as failed
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: mpath0: remaining active paths: 0
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdt: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath10: failed in domap for removal of path sdt
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdn: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath4: failed in domap for removal of path sdn
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdy: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath11: failed in domap for removal of path sdy
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdc: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath8: failed in domap for removal of path sdc
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sde: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath9: failed in domap for removal of path sde
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdaf: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath3: failed in domap for removal of path sdaf
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdg: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath10: failed in domap for removal of path sdg
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdah: remove path (uevent)
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath4: failed in domap for removal of path sdah
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdk: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath11: failed in domap for removal of path sdk
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdm: remove path (uevent)
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath3: failed in domap for removal of path sdm
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdan: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:5: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:5: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath6: failed in domap for removal of path sdan
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdq: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath4: failed in domap for removal of path sdq
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:6: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:6: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdap: remove path (uevent)
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath7: failed in domap for removal of path sdap
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sds: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:5: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:5: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath6: failed in domap for removal of path sds
2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error
2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdw: remove path (uevent)
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:6: multipath: unknown path selector type
2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ----------- [cut here ] --------- [please bite here ] ---------
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Kernel BUG at drivers/scsi/device_handler/scsi_dh_rdac.c:252
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: invalid opcode: 0000 [1] 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: SMP 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: last sysfs file: /class/iscsi_session/session9/ifacename
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: CPU 2 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Modules linked in:
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: iscsi_tcp(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: libiscsi_tcp(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: libiscsi2(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_transport_iscsi2(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_transport_iscsi(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ipmi_si(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_dh_rdac(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_rdac(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_queue_depth(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: netconsole(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_round_robin(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: mptctl(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dell_rbu(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: exa_ioctls(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: nfs(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: lockd(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: nfs_acl(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: xt_tcpudp(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: iptable_filter(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ip_tables(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ipt_SYSRQ(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: x_tables(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: sunrpc(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ipmi_devintf(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ipmi_msghandler(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: bonding1(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: bonding(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ipv6(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: xfrm_nalgo(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: crypto_api(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_mirror(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_log(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_multipath(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_dh(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_mod(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: video(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: hwmon(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: backlight(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: sbs(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: i2c_ec(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: i2c_core(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: button(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: battery(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: asus_acpi(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ac(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: joydev(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: sr_mod(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: cdrom(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: bnx2(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: sg(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: pcspkr(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ata_piix(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: libata(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: mptsas(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: mptscsih(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: mptbase(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_transport_sas(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: sd_mod(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_mod(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ext3(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: jbd(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: uhci_hcd(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ohci_hcd(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ehci_hcd(U)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Pid: 12912, comm: kmpath_rdacd Tainted: G      2.6.18-164sys #1
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RIP: 0010:[<ffffffff8838dc4f>] 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff8838dc4f>] :scsi_dh_rdac:send_mode_select+0x2cf/0x4b0
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: device-mapper: table: 253:6: multipath: unknown path selector type
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: device-mapper: ioctl: error adding target to table
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RSP: 0018:ffff810086959d80  EFLAGS: 00010246
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RAX: 0000000000000000 RBX: 00000000fffffffb RCX: 0000000000000000
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RDX: 0000000000000800 RSI: 0000000000000286 RDI: ffff810055e2d800
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RBP: ffff81005a647216 R08: ffffffff80370b58 R09: 0000000000000001
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: R10: 0000000000000046 R11: ffffffff8004c200 R12: 0000000000000000
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: R13: 0000000000000000 R14: ffff8102352bc000 R15: ffff8102352bc062
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: FS:  0000000000000000(0000) GS:ffff810107eda340(0000) knlGS:0000000000000000
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: CR2: 00002ae8ff6760a0 CR3: 000000023487c000 CR4: 00000000000006e0
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Process kmpath_rdacd (pid: 12912, threadinfo ffff810086958000, task ffff810092a54140)
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Stack: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0000000010008040
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff81005b23a5c0
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff810086959e70
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff810086959dd0
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff810001004498
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff810055e2d800
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff81005a647208
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff81004d3226f0
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0000000380438420
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0000000000000001
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff81023a498d80
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff81023a498d80
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Call Trace:
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff8838d980>] :scsi_dh_rdac:send_mode_select+0x0/0x4b0
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff800561a3>] run_workqueue+0xb3/0x110
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff80052180>] worker_thread+0x0/0x150
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff800b33b0>] keventd_create_kthread+0x0/0xa0
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff80052291>] worker_thread+0x111/0x150
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff8009ca00>] default_wake_function+0x0/0x10
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff80052180>] worker_thread+0x0/0x150
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff800373c9>] kthread+0xd9/0x120
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff80068fb1>] child_rip+0xa/0x11
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff800b33b0>] keventd_create_kthread+0x0/0xa0
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff800372f0>] kthread+0x0/0x120
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff80068fa7>] child_rip+0x0/0x11
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Code: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0f 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0b 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 68 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 98 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: e1 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 38 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 88 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: c2 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: fc 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 00 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0f 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 1f 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 80 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 00 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 00 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 00 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 00 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 48 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 8d 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 54 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RIP 
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff8838dc4f>] :scsi_dh_rdac:send_mode_select+0x2cf/0x4b0
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RSP <ffff810086959d80>
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 EMERG: Kernel panic - not syncing: Fatal exception
2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 
2011 Jan 26 22:27:03 172.19.59.130 EMERG: Rebooting in 1 seconds..

At first glance this seems like a case where the pgpath is deleted during the send_mode_select call and when send_mode_select calls pg_init_done, the pgpath is no longer valid.

Comment 1 Menny Hamburger 2011-01-27 08:28:30 UTC

Babu.Moger suggested I try the following:

http://git.kernel.org/linus/2bded7bd7e8b12a913b0b58167a48220560e1514

I patched the kernel by only adding the call to multipath_wait_for_pg_init_completion after flush_workqueue(kmpath_handlerd) in the multipath_dtr code.

This did not solve the problem.

Comment 2 Menny Hamburger 2011-01-30 08:52:31 UTC

Created attachment 475994 [details]
Fix panic in pg_init_done called from within send_mode_select

This patch is over RHEL54 code but is suitable for RHEL55/RHEL56 also.
Waiting for pg_init completion code was moved to just before the multipath is deleted.
This fix makes me wonder why some multipath release code is also done in postsuspend (in Vanilla).

Comment 3 Menny Hamburger 2011-01-30 08:59:18 UTC

Sorry
The above panic is from another bug - here is the correct one:


2011 Jan 20 02:53:08 172.19.58.130 NOTICE: Unable to handle kernel paging request
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: at 0000000100000010 RIP: 
2011 Jan 20 02:53:08 node0 INFO: kernel: sd 350:0:0:0: rdac: array MD3200i-c7-d7, ctlr 1, MODE_SELECT completed
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff881b713f>] :dm_multipath:pg_init_done+0x2f/0x1c0
2011 Jan 20 02:53:08 node0 ALERT: kernel: Unable to handle kernel paging request at 0000000100000010 RIP: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: PGD 0 
2011 Jan 20 02:53:08 node0 ALERT: kernel: [<ffffffff881b713f>] :dm_multipath:pg_init_done+0x2f/0x1c0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: Oops: 0000 [1] 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: SMP 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: last sysfs file: /devices/platform/host353/session76/target353:0:0/353:0:0:0/rev
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: CPU 3 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: Modules linked in:
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: iscsi_tcp(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: libiscsi_tcp(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: libiscsi2(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: scsi_transport_iscsi2(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: scsi_transport_iscsi(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ipmi_si(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: dell_rbu(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: scsi_dh_rdac(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: dm_rdac(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: dm_queue_depth(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: dm_round_robin(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: xt_tcpudp(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ipt_SYSRQ(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: netconsole(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: iptable_filter(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ip_tables(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: vfat(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: fat(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: usb_storage(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: mptctl(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: exa_ioctls(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: nfs(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: lockd(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: nfs_acl(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: x_tables(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: sunrpc(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ipmi_devintf(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ipmi_msghandler(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: bonding1(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: bonding(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ipv6(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: xfrm_nalgo(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: crypto_api(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: dm_mirror(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: dm_log(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: dm_multipath(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: scsi_dh(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: dm_mod(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: video(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: hwmon(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: backlight(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: sbs(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: i2c_ec(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: i2c_core(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: button(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: battery(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: asus_acpi(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ac(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: sr_mod(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: cdrom(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: joydev(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: sg(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: bnx2(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: pcspkr(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ata_piix(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: libata(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: mptsas(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: mptscsih(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: mptbase(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: scsi_transport_sas(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: sd_mod(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: scsi_mod(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ext3(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: jbd(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: uhci_hcd(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ohci_hcd(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ehci_hcd(U)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: Pid: 23040, comm: kmpath_rdacd Tainted: G      2.6.18-164sys #1
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: RIP: 0010:[<ffffffff881b713f>] 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff881b713f>] :dm_multipath:pg_init_done+0x2f/0x1c0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: RSP: 0018:ffff810081e71d40  EFLAGS: 00010293
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: RAX: ffffffff881b7110 RBX: ffff81004c488780 RCX: ffff810080721c00
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: RDX: ffff81004c488780 RSI: 0000000000000000 RDI: ffff810071521ea0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: R10: 0000000000000000 R11: ffffffff80087ee0 R12: ffff810080721c00
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: R13: ffff810087a247d8 R14: 0000000100000000 R15: ffff810071521e80
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: FS:  0000000000000000(0000) GS:ffff810107f5da40(0000) knlGS:0000000000000000
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: CR2: 0000000100000010 CR3: 00000000592df000 CR4: 00000000000006e0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: Process kmpath_rdacd (pid: 23040, threadinfo ffff810081e70000, task ffff81008a0f5080)
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: Stack: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 0000000000000282
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ffff81004c488780
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ffff81006c14fc00
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ffff810080721c00
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ffff810087a247d8
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 0000000000000000
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ffff81006c14fc62
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ffffffff883bcdf7
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 0000000010008040
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ffff81008c7f65c0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ffff810081e71e70
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ffff810081e71dd0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: Call Trace:
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff883bcdf7>] :scsi_dh_rdac:send_mode_select+0x477/0x4b0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff80032183>] __wake_up+0x43/0x70
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff883bc980>] :scsi_dh_rdac:send_mode_select+0x0/0x4b0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff800561a3>] run_workqueue+0xb3/0x110
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff80052180>] worker_thread+0x0/0x150
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff800b33b0>] keventd_create_kthread+0x0/0xa0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff80052291>] worker_thread+0x111/0x150
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff8009ca00>] default_wake_function+0x0/0x10
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff80052180>] worker_thread+0x0/0x150
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff800373c9>] kthread+0xd9/0x120
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff80068fb1>] child_rip+0xa/0x11
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff800b33b0>] keventd_create_kthread+0x0/0xa0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff800372f0>] kthread+0x0/0x120
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff80068fa7>] child_rip+0x0/0x11
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: Code: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 49 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 8b 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 5e 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 10 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 77 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 7b 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 89 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: f0 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: ff 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 24 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: c5 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: a8 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 8c 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 1b 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 88 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: c7 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 44 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 24 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 04 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 00 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: RIP 
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: [<ffffffff881b713f>] :dm_multipath:pg_init_done+0x2f/0x1c0
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: RSP <ffff810081e71d40>
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: CR2: 0000000100000010
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 EMERG: Kernel panic - not syncing: Fatal exception
2011 Jan 20 02:53:08 172.19.58.130 NOTICE: 
2011 Jan 20 02:53:08 172.19.58.130 EMERG: Rebooting in 1 seconds..

Comment 4 Menny Hamburger 2011-02-06 11:18:25 UTC

Created attachment 477286 [details]
Add a wait for pg init completion in multipath destructor

Comment 5 Menny Hamburger 2011-02-06 11:21:45 UTC

Created attachment 477288 [details]
Flush workqueues on postsuspend as well as at the destructor

After running additional tests it seems that flushing at postsuspend is also required, otherwise we will still have some queues doing work when multipathn is freed.

Note:
Both patches are over RHEL5.6

Comment 6 Mike Snitzer 2011-02-07 14:38:26 UTC

The patch from comment#4 is missing 'init_waitqueue_head(&m->pg_init_wait);' in alloc_multipath().

Taking a step back, it seems we'd want a faithful backport of the following upstream commits:

6380f26 dm mpath: add mutex to synchronize adding and flushing work
67a46da dm mpath: prevent io from work queue while suspended
c2f3d24 dm mpath: reject messages when device is suspended
83c0d5d dm mpath: pass struct pgpath to pg init done
f7b934c dm mpath: skip activate_path for failed paths -- unrelated but worthwhile
d0259bf dm mpath: hold io until all pg_inits completed
2bded7b dm mpath: wait for pg_init completion when suspending
fb61264 dm mpath: refactor pg_init

Comment 7 RHEL Program Management 2011-02-07 14:50:35 UTC

This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.

Comment 8 Mike Snitzer 2011-02-08 18:55:51 UTC

Patches that backport the following upstream commits are available here:
http://people.redhat.com/msnitzer/patches//.rhel5.7/bz673058/

6df400a v2.6.33-rc1 dm mpath: flush workqueues before suspend completes
6380f26 v2.6.33-rc1 dm mpath: add mutex to synchronize adding and flushing work
67a46da v2.6.33-rc1 dm mpath: prevent io from work queue while suspended
83c0d5d v2.6.34-rc1 dm mpath: pass struct pgpath to pg init done
f7b934c v2.6.34-rc1 dm mpath: skip activate_path for failed paths
d0259bf v2.6.34-rc1 dm mpath: hold io until all pg_inits completed
2bded7b v2.6.34-rc1 dm mpath: wait for pg_init completion when suspending
6bbf79a v2.6.36-rc1 dm mpath: fix NULL pointer dereference when path parameters missing

Comment 22 Jarod Wilson 2011-03-23 21:44:51 UTC

Patch(es) available in kernel-2.6.18-250.el5
Detailed testing feedback is always welcomed.

Comment 26 Martin Prpič 2011-04-14 10:31:16 UTC

    Technical note added. If any revisions are required, please edit the "Technical Notes" field
    accordingly. All revisions will be proofread by the Engineering Content Services team.
    
    New Contents:
A race could occur when an internal multipath structure (pgpath) was freed before it was used to signal the path group initialization was complete (via pg_init_done). This update includes a number of fixes that address this issue. multipath is now increasingly robust when multipathd restarts are combined with I/O operations to multipath devices and storage failures.

Comment 27 Gris Ge 2011-05-30 07:05:45 UTC

I have tried to running these commands at same time, but not able to hit the problem:

=========
for X in `seq 1 100`;do
    multipath -F;
    service multipathd restart
    sleep 10;
done
=========
for Y in `seq 1 100`;do
    sudo ./include_4_python.sh FC_Switch_Link_Trigger_By_WWPN down "0x10000000c990be2b";
    sudo ./include_4_python.sh FC_Switch_Link_Trigger_By_WWPN up "0x10000000c990be2b";
done
=========

Enviroment:
NetApp 3170 ALUA via lpfc. 50 LUNs over 8 path (disabled 1 HBA port during testing, only 4 path from 1 hba port in this testing).

Does this bug need "drac" hardware handler?

Comment 31 Gris Ge 2011-06-24 02:17:11 UTC

Code reviewed.

Sanity Only.

Comment 32 errata-xmlrpc 2011-07-21 09:27:31 UTC

An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2011-1065.html

Note You need to log in before you can comment on or make changes to this bug.