Bug 591172

Summary: insmod: can't insert '/lib/dm-mod.ko': File exists
Product: Red Hat Enterprise Linux 6 Reporter: Qian Cai <qcai>
Component: kexec-toolsAssignee: Neil Horman <nhorman>
Status: CLOSED CURRENTRELEASE QA Contact: Han Pingtian <phan>
Severity: medium Docs Contact:
Priority: low    
Version: 6.0CC: phan, xyang
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: kexec-tools-2.0.0-71 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2010-07-02 20:52:16 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 524819    
Attachments:
Description Flags
patch to filter module list to list modules once
none
updated patch
none
updated patch none

Description Qian Cai 2010-05-11 15:31:44 UTC
Description of problem:
When the kdump in progress, the following error is displayed,
...
Loading virtio_blk.ko module
Loading dm-mod.ko module
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.16.0-ioctl (2009-11-05) initialised:
dm-devel
Loading dm-log.ko module
Loading dm-region-hash.ko module
Loading dm-mirror.ko module
Loading dm-zero.ko module
Loading dm-snapshot.ko module
Loading ipv6.ko module
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
Loading sg.ko module
Loading virtio_net.ko module
Loading virtio_balloon.ko module
Loading i2c-core.ko module
Loading cdrom.ko module
Loading pata_acpi.ko module
Loading ata_generic.ko module
Loading virtio_ring.ko module
Loading ata_piix.ko module
scsi0 : ata_piix
scsi1 : ata_piix
ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc000 irq 14
ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc008 irq 15
do_IRQ: 0.97 No irq handler for vector (irq -1)
ata2.00: ATAPI: QEMU DVD-ROM, 0.12.1, max UDMA/100
ata2.00: configured for MWDMA2
scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     0.12 PQ: 0 ANSI: 5
scsi 1:0:0:0: Attached scsi generic sg0 type 5
Loading dm-mod.ko module
insmod: can't insert '/lib/dm-mod.ko': File exists
...

You can see dm-mod.ko has been loaded twice.

Version-Release number of selected component (if applicable):
kexec-tools-2.0.0-68.el6

How reproducible:
Saw it several times - one from the kvm guest, and also bare-metal systems if I remembered correctly.

Comment 1 RHEL Program Management 2010-05-11 15:33:48 UTC
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux major release.  Product Management has requested further
review of this request by Red Hat Engineering, for potential inclusion in a Red
Hat Enterprise Linux Major release.  This request is not yet committed for
inclusion.

Comment 2 Neil Horman 2010-05-12 12:27:39 UTC
Created attachment 413412 [details]
patch to filter module list to list modules once

Cai, Please give this patch a try and confirm that it solves the problem for you.  Thanks!

Comment 4 Xudong Yang 2010-05-20 03:50:40 UTC
It seems that the patch haven't solved the problem.

First, code in the patch has a little bug:
    if [ $? -ne 0 ]
        MODULES="$MODULES /lib/modules/$kernel/$fmPath"
    fi
Where the line of if condition missed "then", which would cause error.

Then after it was added, it still have the problem:

1. Check the script is patched.
[root@dell-pe1800-01 ~]# cat /sbin/mkdumprd | nl | less
...
   277      # only need to add each module once
   278      echo $MODULES | grep -q "/lib/modules/$kernel/$fmpath"
   279      if [ $? -ne 0 ]; then
   280          MODULES="$MODULES /lib/modules/$kernel/$fmPath"
   281      fi
...

2. Then cause kernel panic:
[root@dell-pe1800-01 ~]# echo "c" > /proc/sysrq-trigger
...
Loading dm-mod.ko module
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.17.0-ioctl (2010-03-05) initialised: dm-devel
Loading dm-log.ko module
Loading dm-region-hash.ko module
Loading dm-mirror.ko module
Loading dm-zero.ko module
Loading dm-snapshot.ko module
Loading freq_table.ko module
Loading speedstep-lib.ko module
Loading ipv6.ko module
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
Loading uinput.ko module
Loading e1000.ko module
Intel(R) PRO/1000 Network Driver - version 7.3.21-k5-NAPI
Copyright (c) 1999-2006 Intel Corporation.
e1000 0000:03:07.0: PCI INT A -> GSI 69 (level, low) -> IRQ 69
e1000: 0000:03:07.0: e1000_probe: (PCI:66MHz:32-bit) 00:0f:1f:f9:cd:f8
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
Loading iTCO_vendor_support.ko miTCO_vendor_support: vendor-support=0
odule
Loading edac_core.ko module
EDAC MC: Ver: 2.1.0 May 10 2010
Loading shpchp.ko module
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
Loading parport.ko module
Loading sg.ko module
Loading dcdbas.ko module
dcdbas dcdbas: Dell Systems Management Base Driver (version 5.6.0-3.2)
Loading serio_raw.ko module
Loading crc-t10dif.ko module
insmod: can't insert '/lib/crc-t10pata_acpi 0000:00:1f.1: PCI INT A -> GSI 18 (level, low) -> IRQ 18
dif.ko': File expata_acpi 0000:00:1f.1: PCI INT A disabled
ists
Loading cdrom.ko module
L3ware Storage Controller device driver for Linux v1.26.02.003RH.
oading pata_acpi3w-xxxx 0000:03:09.0: PCI INT A -> GSI 64 (level, low) -> IRQ 64
.ko module
Loading 3w-xxxx.ko module
scsi0 : 3ware Storage Controller
3w-xxxx: scsi0: Found a 3ware Storage Controller at 0xecb0, IRQ: 64.
Loading output.kscsi 0:0:0:0: Direct-Access     3ware    Logical Disk 0   1.2  PQ: 0 ANSI: 0
o module
Loading ata_piix.ko module
ata_piix 0000:00:1f.1: PCI INT A -> GSI 18 (level, low) -> IRQ 18
scsi1 : ata_piix
scsi2 : ata_piix
ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xfc00 irq 14
ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xfc08 irq 15
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 0:0:0:0: [sda] 234489600 512-byte logical blocks: (120 GB/111 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Write cache: enabled, read cache: disabled, supports DPO and FUA
 sda: sda1 sda2
sd 0:0:0:0: [sda] Attached SCSI disk
ata1.00: ATAPI: GCR-8483B, 1.05, max UDMA/33
ata1.00: configured for UDMA/33
scsi 1:0:0:0: CD-ROM            HL-DT-ST CD-ROM GCR-8483B 1.05 PQ: 0 ANSI: 5
scsi 1:0:0:0: Attached scsi generic sg1 type 5
Loading i2c-core.ko module
Loading dm-mod.ko module
insmod: can't insert 'p4-clockmod: P4/Xeon(TM) CPU On-Demand Clock Modulation available
/lib/dm-mod.ko': File exists
...

The version-release number of kexec-tools is:
kexec-tools-2.0.0-69.el6
which runs on RHEL6.0-Snapshot-4_nfs-Workstation-x86_64

Hope this will help, thanks!

Comment 7 Neil Horman 2010-05-20 14:32:44 UTC
Created attachment 415432 [details]
updated patch

could you try this version please?  I might need to quote the MODULES variable to avoid inadvertent behavior in the echo

Comment 8 Xudong Yang 2010-05-21 05:00:42 UTC
The new patch still doesn't help.

1. Check mkdumprd is patched
[root@dell-pe1800-01 ~]# cat /sbin/mkdumprd | nl | less
...
   277      # only need to add each module once
   278      echo "$MODULES" | grep -q "/lib/modules/$kernel/$fmpath"
   279      if [ $? -ne 0 ]; then
   280          MODULES="$MODULES /lib/modules/$kernel/$fmPath"
   281      fi
...

2. Then cause kernel panic
[root@dell-pe1800-01 ~]# echo "c" > /proc/sysrq-trigger
...
Loading dm-mod.ko module
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.17.0-ioctl (2010-03-05) initialised: dm-devel
Loading dm-log.ko module
Loading dm-region-hash.ko module
Loading dm-mirror.ko module
Loading dm-zero.ko module
Loading dm-snapshot.ko module
Loading freq_table.ko module
Loading speedstep-lib.ko module
Loading ipv6.ko module
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
Loading iTCO_veniTCO_vendor_support: vendor-support=0
dor_support.ko module
Loading e1000.ko module
Intel(R) PRO/1000 Network Driver - version 7.3.21-k5-NAPI
Copyright (c) 1999-2006 Intel Corporation.
e1000 0000:03:07.0: PCI INT A -> GSI 69 (level, low) -> IRQ 69
e1000: 0000:03:07.0: e1000_probe: (PCI:66MHz:32-bit) 00:0f:1f:f9:cd:f8
e1000: eth0: e1000_probe: Intel(R) PRO/1000 Network Connection
Loading shpchp.ko module
shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
Loading sg.ko module
Loading parport.ko module
Loading edac_core.ko module
EDAC MC: Ver: 2.1.0 May 10 2010
Loading serio_raw.ko module
Loading dcdbas.ko module
dcdbas dcdbas: Dell Systems Management Base Driver (version 5.6.0-3.2)
Loading crc-t10dif.ko module
insmod: can't insert '/lib/crc-t10pata_acpi 0000:00:1f.1: PCI INT A -> GSI 18 (l8
dif.ko': File expata_acpi 0000:00:1f.1: PCI INT A disabled
ists
Loading cdrom.ko module
Loading pata_acpi.ko module
Load3ware Storage Controller device driver for Linux v1.26.02.003RH.
ing output.ko mo3w-xxxx 0000:03:09.0: PCI INT A -> GSI 64 (level, low) -> IRQ 64
dule
Loading 3w-xxxx.ko module
scsi0 : 3ware Storage Controller
3w-xxxx: scsi0: Found a 3ware Storage Controller at 0xecb0, IRQ: 64.
Loading ata_piixscsi 0:0:0:0: Direct-Access     3ware    Logical Disk 0   1.2  P0
.ko module
ata_piix 0000:00:1f.1: PCI INT A -> GSI 18 (level, low) -> IRQ 18
scsi1 : ata_piix
scsi2 : ata_piix
ata1: PATA max UDMA/100 cmd 0x1f0 ctl 0x3f6 bmdma 0xfc00 irq 14
ata2: PATA max UDMA/100 cmd 0x170 ctl 0x376 bmdma 0xfc08 irq 15
sd 0:0:0:0: Attached scsi generic sg0 type 0
sd 0:0:0:0: [sda] 234489600 512-byte logical blocks: (120 GB/111 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Write cache: enabled, read cache: disabled, supports DPO and FA
 sda: sda1 sda2
sd 0:0:0:0: [sda] Attached SCSI disk
ata1.00: ATAPI: GCR-8483B, 1.05, max UDMA/33
ata1.00: configured for UDMA/33
scsi 1:0:0:0: CD-ROM            HL-DT-ST CD-ROM GCR-8483B 1.05 PQ: 0 ANSI: 5
scsi 1:0:0:0: Attached scsi generic sg1 type 5
Loading i2c-core.ko module
Loading dm-mod.ko module
insmod: can't insert 'p4-clockmod: P4/Xeon(TM) CPU On-Demand Clock Modulation ave
/lib/dm-mod.ko': File exists
...

Based on the first line and the last line of the output, it still tried to load dm-mod.ko twice.

Hope it helps, Neil. Thanks.

Comment 9 Neil Horman 2010-05-21 12:48:28 UTC
Created attachment 415665 [details]
updated patch

Hope you don't mind, poked about on the system you have reserved.  This patch fixed the issue for me.  Please confirm and I'll commit this ASAP.  Thanks!

Comment 10 Qian Cai 2010-05-23 09:02:23 UTC
I tried the latest patch, and did not see the problem again.

Comment 13 releng-rhel@redhat.com 2010-07-02 20:52:16 UTC
Red Hat Enterprise Linux Beta 2 is now available and should resolve
the problem described in this bug report. This report is therefore being closed
with a resolution of CURRENTRELEASE. You may reopen this bug report if the
solution does not work for you.