Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Cause: Multipath wasn't blacklisting tapdev, which cannot be multipathed. It tried to multipath them, but their sysfs format is different from expected path devices.
Consequence: Multipathd could crash if multiple tadpdev devices were removed from the system.
Fix: Multipath now blacklists /dev/td[a-z].* device by default.
Result: Multipath will no longer try to setup multipath devices on tapdev devices, and will not crash when they are removed.
Created attachment 776672[details]
the core dump file
Description of problem:
the multipathd deamon process exit with segfault
reproduce:
1.create the block devices(/dev/td*)
2.delete the block devices(/dev/td*)
3.after minutes,the multipathd exit with segfault
cat /var/log/messages:
kernel: multipathd[32369]: segfault at 7f57a9dd6ff8 ip 0000003574a2d3c9 sp 00007f57a9dd7000 error 6 in libmultipath.so[3574a00000+3b000]
I have a dumb question. What kind of devices are td devices? Are they supposed to be multipathed? multipath certainly doesn't blacklist them by default, but perhaps it should.
The issue is that these td devices all have the sysfs parent of /devices/virtual/block/. All devices that multipath currently works with have unique parents. What's happening here is that for all paths devices, multipath stores their sysfs devices and their parents in a cache, since it needs to reference them to grab sysfs values. When the second td* path device is added by multipath, it uses the cached version of the parent from the first device instead of creating it's own device in the cache. When multipath removes the first device, it deletes that cached parent device. When it removes the second device, it again tries to delete the cached parent device, but it already has done so.
If these devices aren't supposed to be multipathed, then the workaround is to add
blacklist {
devnode "^td"
}
to /etc/multipath.conf, and I can add this to the default blacklist, so that starting in RHEL-6.5, multipath will automatically blacklist these devices.
If these devices really are supposed to be blacklisted, that's a trickier problem. Multipath would need to either cache separate parents for these devices, or it would need to keep reference counts and only delete the device when the last user drops it. Unfortunately, the way multipath uses these cached devices, neither solution is particularly straightforward.
I'm sorry. last few days, I'm on holidays .so didn't reply the bug.
/dev/td* are tapdev devices：
[root@localhost ~]# cat /proc/devices |grep tap
251 blktap2
253 tapdev
[root@localhost ~]# ll /dev/tda
brw-rw---- 1 root disk 253, 0 Aug 5 09:19 /dev/tda
brw-rw---- 1 root disk 253, 1 Aug 5 09:19 /dev/tdb
/dev/td* did not support the multipath.
So, I'll go ahead and make multipath automatically blacklist these devices. Can you verify that the workaround of adding
blacklist {
devnode "^td"
}
to /etc/multipath.conf solves the issue for you?
Reproduced on device-mapper-multipath-0.4.9-64.el6:
[root@storageqe-17 ~]# multipathd show conf|grep td
[root@storageqe-17 ~]# echo $?
1
Verified on device-mapper-multipath-0.4.9-71.el6, "td[a-z]" devices are now blacklisted by default.
[root@storageqe-17 ~]# multipathd show conf|grep td
devnode "^(td|hd)[a-z]"
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
http://rhn.redhat.com/errata/RHBA-2013-1574.html
Created attachment 776672 [details] the core dump file Description of problem: the multipathd deamon process exit with segfault reproduce: 1.create the block devices(/dev/td*) 2.delete the block devices(/dev/td*) 3.after minutes,the multipathd exit with segfault cat /var/log/messages: kernel: multipathd[32369]: segfault at 7f57a9dd6ff8 ip 0000003574a2d3c9 sp 00007f57a9dd7000 error 6 in libmultipath.so[3574a00000+3b000]