Cause: Multipath wasn't allocating enough space to store the string for the /sys/class/scsi_host/host<X> pathname if X was larger than 3 digits
Consequence: Multipath would crash with memory corruption if there were more than 1000 scsi hosts. This can happen if there are too many iscsi devices.
Fix: Multipath now allocates enough space to handle the pathname at any possible size.
Result: Multipath will no longer crash with memory corruption if there are two many iscsi devices that it is trying to use.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://rhn.redhat.com/errata/RHBA-2015-2132.html
Created attachment 1069598 [details] dmesg log Description of problem: During our test for checking send targets discovery many iSCSI LUNs, we hit a issue, when multipath gets terminated after logging in to many iSCSI devices. There is 2048 iSCSI devices After running multipath: *** buffer overflow detected ***: multipath terminated ======= Backtrace: ========= /lib64/libc.so.6(__fortify_fail+0x37)[0x7f1befc46a77] /lib64/libc.so.6(+0x10bc30)[0x7f1befc44c30] /lib64/libc.so.6(+0x10b139)[0x7f1befc44139] /lib64/libc.so.6(_IO_default_xsputn+0xbc)[0x7f1befbb1a1c] /lib64/libc.so.6(_IO_vfprintf+0xb0d)[0x7f1befb8105d] /lib64/libc.so.6(__vsprintf_chk+0x88)[0x7f1befc441c8] /lib64/libc.so.6(__sprintf_chk+0x7d)[0x7f1befc4411d] /lib64/libmultipath.so.0(sysfs_get_iscsi_ip_address+0x40)[0x7f1beff137f0] /lib64/libmultipath.so.0(+0x2dc69)[0x7f1beff27c69] /lib64/libmultipath.so.0(get_path_layout+0x8c)[0x7f1beff27ebc] multipath[0x402804] /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f1befb5ab15] multipath[0x402cd1] ======= Memory map: ======== 00400000-00405000 r-xp 00000000 fd:00 203854686 /usr/sbin/multipath 00604000-00605000 r--p 00004000 fd:00 203854686 /usr/sbin/multipath 00605000-00606000 rw-p 00005000 fd:00 203854686 /usr/sbin/multipath 00627000-025cf000 rw-p 00000000 00:00 0 [heap] 7f1bed48e000-7f1bed490000 r-xp 00000000 fd:00 817763 /usr/lib64/multipath/libprioontap.so 7f1bed490000-7f1bed68f000 ---p 00002000 fd:00 817763 /usr/lib64/multipath/libprioontap.so 7f1bed68f000-7f1bed690000 r--p 00001000 fd:00 817763 /usr/lib64/multipath/libprioontap.so 7f1bed690000-7f1bed691000 rw-p 00002000 fd:00 817763 /usr/lib64/multipath/libprioontap.so 7f1bed691000-7f1bed694000 r-xp 00000000 fd:00 817147 /usr/lib64/multipath/libchecktur.so 7f1bed694000-7f1bed893000 ---p 00003000 fd:00 817147 /usr/lib64/multipath/libchecktur.so 7f1bed893000-7f1bed894000 r--p 00002000 fd:00 817147 /usr/lib64/multipath/libchecktur.so 7f1bed894000-7f1bed895000 rw-p 00003000 fd:00 817147 /usr/lib64/multipath/libchecktur.so 7f1bed895000-7f1bed896000 r-xp 00000000 fd:00 817149 /usr/lib64/multipath/libprioconst.so 7f1bed896000-7f1beda95000 ---p 00001000 fd:00 817149 /usr/lib64/multipath/libprioconst.so 7f1beda95000-7f1beda96000 r--p 00000000 fd:00 817149 /usr/lib64/multipath/libprioconst.so 7f1beda96000-7f1beda97000 rw-p 00001000 fd:00 817149 /usr/lib64/multipath/libprioconst.so 7f1beda97000-7f1beda98000 r-xp 00000000 fd:00 202270762 /usr/lib64/libaio.so.1.0.1 7f1beda98000-7f1bedc97000 ---p 00001000 fd:00 202270762 /usr/lib64/libaio.so.1.0.1 7f1bedc97000-7f1bedc98000 r--p 00000000 fd:00 202270762 /usr/lib64/libaio.so.1.0.1 7f1bedc98000-7f1bedc99000 rw-p 00001000 fd:00 202270762 /usr/lib64/libaio.so.1.0.1 7f1bedc99000-7f1bedc9b000 r-xp 00000000 fd:00 817141 /usr/lib64/multipath/libcheckdirectio.so 7f1bedc9b000-7f1bede9a000 ---p 00002000 fd:00 817141 /usr/lib64/multipath/libcheckdirectio.so 7f1bede9a000-7f1bede9b000 r--p 00001000 fd:00 817141 /usr/lib64/multipath/libcheckdirectio.so 7f1bede9b000-7f1bede9c000 rw-p 00002000 fd:00 817141 /usr/lib64/multipath/libcheckdirectio.so 7f1bede9c000-7f1bedeb1000 r-xp 00000000 fd:00 201329780 /usr/lib64/libz.so.1.2.7 7f1bedeb1000-7f1bee0b0000 ---p 00015000 fd:00 201329780 /usr/lib64/libz.so.1.2.7 7f1bee0b0000-7f1bee0b1000 r--p 00014000 fd:00 201329780 /usr/lib64/libz.so.1.2.7 7f1bee0b1000-7f1bee0b2000 rw-p 00015000 fd:00 201329780 /usr/lib64/libz.so.1.2.7 7f1bee0b2000-7f1bee0c1000 r-xp 00000000 fd:00 201329821 /usr/lib64/libbz2.so.1.0.6 7f1bee0c1000-7f1bee2c0000 ---p 0000f000 fd:00 201329821 /usr/lib64/libbz2.so.1.0.6 7f1bee2c0000-7f1bee2c1000 r--p 0000e000 fd:00 201329821 /usr/lib64/libbz2.so.1.0.6 7f1bee2c1000-7f1bee2c2000 rw-p 0000f000 fd:00 201329821 /usr/lib64/libbz2.so.1.0.6 7f1bee2c2000-7f1bee2d7000 r-xp 00000000 fd:00 201329809 /usr/lib64/libelf-0.163.so 7f1bee2d7000-7f1bee4d6000 ---p 00015000 fd:00 201329809 /usr/lib64/libelf-0.163.so 7f1bee4d6000-7f1bee4d7000 r--p 00014000 fd:00 201329809 /usr/lib64/libelf-0.163.so 7f1bee4d7000-7f1bee4d8000 rw-p 00015000 fd:00 201329809 /usr/lib64/libelf-0.163.so 7f1bee4d8000-7f1bee4dc000 r-xp 00000000 fd:00 201329974 /usr/lib64/libattr.so.1.1.0 7f1bee4dc000-7f1bee6db000 ---p 00004000 fd:00 201329974 /usr/lib64/libattr.so.1.1.0 7f1bee6db000-7f1bee6dc000 r--p 00003000 fd:00 201329974 /usr/lib64/libattr.so.1.1.0 7f1bee6dc000-7f1bee6dd000 rw-p 00004000 fd:00 201329974 /usr/lib64/libattr.so.1.1.0 7f1bee6dd000-7f1bee701000 r-xp 00000000 fd:00 201329694 /usr/lib64/liblzma.so.5.0.99 7f1bee701000-7f1bee900000 ---p 00024000 fd:00 201329694 /usr/lib64/liblzma.so.5.0.99 7f1bee900000-7f1bee901000 r--p 00023000 fd:00 201329694 /usr/lib64/liblzma.so.5.0.99 7f1bee901000-7f1bee902000 rw-p 00024000 fd:00 201329694 /usr/lib64/liblzma.so.5.0.99 7f1bee902000-7f1bee962000 r-xp 00000000 fd:00 201329761 /usr/lib64/libpcre.so.1.2.0 7f1bee962000-7f1beeb61000 ---p 00060000 fd:00 201329761 /usr/lib64/libpcre.so.1.2.0 7f1beeb61000-7f1beeb62000 r--p 0005f000 fd:00 201329761 /usr/lib64/libpcre.so.1.2.0 7f1beeb62000-7f1beeb63000 rw-p 00060000 fd:00 201329761 /usr/lib64/libpcre.so.1.2.0 7f1beeb63000-7f1beeba8000 r-xp 00000000 fd:00 201330052 /usr/lib64/libdw-0.163.so 7f1beeba8000-7f1beeda7000 ---p 00045000 fd:00 201330052 /usr/lib64/libdw-0.163.so 7f1beeda7000-7f1beeda9000 r--p 00044000 fd:00 201330052 /usr/lib64/libdw-0.163.so 7f1beeda9000-7f1beedaa000 rw-p 00046000 fd:00 201330052 /usr/lib64/libdw-0.163.so 7f1beedaa000-7f1beeeab000 r-xp 00000000 fd:00 201329596 /usr/lib64/libm-2.17.so 7f1beeeab000-7f1bef0aa000 ---p 00101000 fd:00 201329596 /usr/lib64/libm-2.17.so 7f1bef0aa000-7f1bef0ab000 r--p 00100000 fd:00 201329596 /usr/lib64/libm-2.17.so 7f1bef0ab000-7f1bef0ac000 rw-p 00101000 fd:00 201329596 /usr/lib64/libm-2.17.so 7f1bef0ac000-7f1bef0b0000 r-xp 00000000 fd:00 201329976 /usr/lib64/libcap.so.2.22 7f1bef0b0000-7f1bef2af000 ---p 00004000 fd:00 201329976 /usr/lib64/libcap.so.2.22 7f1bef2af000-7f1bef2b0000 r--p 00003000 fd:00 201329976 /usr/lib64/libcap.so.2.22 7f1bef2b0000-7f1bef2b1000 rw-p 00004000 fd:00 201329976 /usr/lib64/libcap.so.2.22 7f1bef2b1000-7f1bef2c6000 r-xp 00000000 fd:00 201326732 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f1bef2c6000-7f1bef4c5000 ---p 00015000 fd:00 201326732 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f1bef4c5000-7f1bef4c6000 r--p 00014000 fd:00 201326732 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f1bef4c6000-7f1bef4c7000 rw-p 00015000 fd:00 201326732 /usr/lib64/libgcc_s-4.8.5-20150702.so.1 7f1bef4c7000-7f1bef50b000 r-xp 00000000 fd:00 201329695 /usr/lib64/libsepol.so.1 7f1bef50b000-7f1bef70a000 ---p 00044000 fd:00 201329695 /usr/lib64/libsepol.so.1 7f1bef70a000-7f1bef70b000 r--p 00043000 fd:00 201329695 /usr/lib64/libsepol.so.1 7f1bef70b000-7f1bef70c000 rw-p 00044000 fd:00 201329695 /usr/lib64/libsepol.so.1 7f1bef70c000-7f1bef72d000 r-xp 00000000 fd:00 201329777 /usr/lib64/libselinux.so.1 7f1bef72d000-7f1bef92d000 ---p 00021000 fd:00 201329777 /usr/lib64/libselinux.so.1 7f1bef92d000-7f1bef92e000 r--p 00021000 fd:00 201329777 /usr/lib64/libselinux.so.1 7f1bef92e000-7f1bef92f000 rw-p 00022000 fd:00 201329777 /usr/lib64/libselinux.so.1 7f1bef92f000-7f1bef931000 rw-p 00000000 00:00 0 7f1bef931000-7f1bef938000 r-xp 00000000 fd:00 201329618 /usr/lib64/librt-2.17.so 7f1bef938000-7f1befb37000 ---p 00007000 fd:00 201329618 /usr/lib64/librt-2.17.so 7f1befb37000-7f1befb38000 r--p 00006000 fd:00 201329618 /usr/lib64/librt-2.17.so 7f1befb38000-7f1befb39000 rw-p 00007000 fd:00 201329618 /usr/lib64/librt-2.17.so 7f1befb39000-7f1befcef000 r-xp 00000000 fd:00 201329588 /usr/lib64/libc-2.17.so 7f1befcef000-7f1befeef000 ---p 001b6000 fd:00 201329588 /usr/lib64/libc-2.17.so 7f1befeef000-7f1befef3000 r--p 001b6000 fd:00 201329588 /usr/lib64/libc-2.17.so 7f1befef3000-7f1befef5000 rw-p 001ba000 fd:00 201329588 /usr/lib64/libc-2.17.so 7f1befef5000-7f1befefa000 rw-p 00000000 00:00 0 7f1befefa000-7f1beff42000 r-xp 00000000 fd:00 203854683 /usr/lib64/libmultipath.so.0 7f1beff42000-7f1bf0142000 ---p 00048000 fd:00 203854683 /usr/lib64/libmultipath.so.0 7f1bf0142000-7f1bf0143000 r--p 00048000 fd:00 203854683 /usr/lib64/libmultipath.so.0 7f1bf0143000-7f1bf0148000 rw-p 00049000 fd:00 203854683 /usr/lib64/libmultipath.so.0 7f1bf0148000-7f1bf0149000 rw-p 00000000 00:00 0 7f1bf0149000-7f1bf014c000 r-xp 00000000 fd:00 201329594 /usr/lib64/libdl-2.17.so 7f1bf014c000-7f1bf034b000 ---p 00003000 fd:00 201329594 /usr/lib64/libdl-2.17.so 7f1bf034b000-7f1bf034c000 r--p 00002000 fd:00 201329594 /usr/lib64/libdl-2.17.so 7f1bf034c000-7f1bf034d000 rw-p 00003000 fd:00 201329594 /usr/lib64/libdl-2.17.so 7f1bf034d000-7f1bf039e000 r-xp 00000000 fd:00 202296243 /usr/lib64/libdevmapper.so.1.02 7f1bf039e000-7f1bf059e000 ---p 00051000 fd:00 202296243 /usr/lib64/libdevmapper.so.1.02 7f1bf059e000-7f1bf059f000 r--p 00051000 fd:00 202296243 /usr/lib64/libdevmapper.so.1.02 7f1bf059f000-7f1bf05a2000 rw-p 00052000 fd:00 202296243 /usr/lib64/libdevmapper.so.1.02 7f1bf05a2000-7f1bf05a4000 rw-p 00000000 00:00 0 7f1bf05a4000-7f1bf05ba000 r-xp 00000000 fd:00 201329614 /usr/lib64/libpthread-2.17.so 7f1bf05ba000-7f1bf07ba000 ---p 00016000 fd:00 201329614 /usr/lib64/libpthread-2.17.so 7f1bf07ba000-7f1bf07bb000 r--p 00016000 fd:00 201329614 /usr/lib64/libpthread-2.17.so 7f1bf07bb000-7f1bf07bc000 rw-p 00017000 fd:00 201329614 /usr/lib64/libpthread-2.17.so 7f1bf07bc000-7f1bf07c0000 rw-p 00000000 00:00 0 7f1bf07c0000-7f1bf07e1000 r-xp 00000000 fd:00 201329581 /usr/lib64/ld-2.17.so 7f1bf09b7000-7f1bf09c0000 rw-p 00000000 00:00 0 7f1bf09c0000-7f1bf09d2000 r-xp 00000000 fd:00 202181518 /usr/lib64/libudev.so.1.6.2 7f1bf09d2000-7f1bf09d3000 ---p 00012000 fd:00 202181518 /usr/lib64/libudev.so.1.6.2 7f1bf09d3000-7f1bf09d4000 r--p 00012000 fd:00 202181518 /usr/lib64/libudev.so.1.6.2 7f1bf09d4000-7f1bf09d5000 rw-p 00013000 fd:00 202181518 /usr/lib64/libudev.so.1.6.2 7f1bf09d5000-7f1bf09d6000 rw-p 00000000 00:00 0 7f1bf09df000-7f1bf09e1000 rw-p 00000000 00:00 0 7f1bf09e1000-7f1bf09e2000 r--p 00021000 fd:00 201329581 /usr/lib64/ld-2.17.so 7f1bf09e2000-7f1bf09e3000 rw-p 00022000 fd:00 201329581 /usr/lib64/ld-2.17.so 7f1bf09e3000-7f1bf09e4000 rw-p 00000000 00:00 0 7ffe22bf6000-7ffe22c17000 rw-p 00000000 00:00 0 [stack] 7ffe22d3f000-7ffe22d41000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Return:0 This issue was not seen on RHEL-7.1 or RHEL-7.2 alpha(device-mapper-multipath-0.4.9-77.el7.x86_64) Version-Release number of selected component (if applicable): device-mapper-multipath-0.4.9-82.el7.x86_64 kernel-3.10.0-306.0.1.el7 iscsi-initiator-utils-6.2.0.873-32.el7 How reproducible: needs further testing Steps to Reproduce: 1.Establish iSCSI session with many LUNs 2.Run multipath on them 3.see '*** buffer overflow detected ***: multipath terminated' message Actual results: multipath terminated Expected results: created multipath devices Additional info: see call trace in dmesg log