Bug 1250728

Summary: IPaddr2 send_arp causes a buffer overflow on infiniband devices
Product: Red Hat Enterprise Linux 7 Reporter: Dan Lavu <dlavu>
Component: resource-agentsAssignee: Oyvind Albrigtsen <oalbrigt>
Status: CLOSED ERRATA QA Contact: cluster-qe <cluster-qe>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.1CC: agk, cluster-maint, dwood, fdinitto, lars.ellenberg, mnovacek, rkerry
Target Milestone: rc   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: resource-agents-3.9.5-61.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1351717 1369810 (view as bug list) Environment:
Last Closed: 2016-11-03 23:58:17 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1351717, 1369810    

Description Dan Lavu 2015-08-05 20:31:22 UTC 
Description of problem:
Using HA resource IPaddr2 with infiniband by default uses a deprecated binary to clear the arp table, 'ipoibarping'. Per the documentation 'send_arp' is used if the binary is not available' and causes a buffer overflow

-------------------------------------------------
<parameter name="arp_sender">
<longdesc lang="en">
The program to send ARP packets with on start. For infiniband
interfaces, default is ipoibarping. If ipoibarping is not
available, set this to send_arp.
</longdesc>
<shortdesc lang="en">ARP sender</shortdesc>
<content type="string" default=""/>
</parameter>
-------------------------------------------------

Version-Release number of selected component (if applicable):

pacemaker-cli-1.1.12-22.el7_1.2.x86_64
pacemaker-1.1.12-22.el7_1.2.x86_64
pacemaker-libs-1.1.12-22.el7_1.2.x86_64
pacemaker-cluster-libs-1.1.12-22.el7_1.2.x86_64
resource-agents-3.9.5-40.el7_1.4.x86_64

How reproducible:

Always


Steps to Reproduce:
1. Create an IPaddr2 resource using infiniband 
2. Try to send_arp on an infiniband interface or move the resource, the IP is in-accessible for 2-3 minutes until it is manually pinged from the clients
3.

Actual results:

[root@deadpool:~]#  /usr/libexec/heartbeat/send_arp -i 500 -r 5 -p /var/run/resource-agents/send_arp-100.64.78.12 ib0 100.64.78.12 auto 100.64.78.12 ffffffffffff 
*** buffer overflow detected ***: /usr/libexec/heartbeat/send_arp terminated
======= Backtrace: =========
/lib64/libc.so.6(__fortify_fail+0x37)[0x7f2ac1842a57]
/lib64/libc.so.6(+0x10bc10)[0x7f2ac1840c10]
/usr/libexec/heartbeat/send_arp[0x40177b]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f2ac1756af5]
/usr/libexec/heartbeat/send_arp[0x401d91]
======= Memory map: ========
00400000-00403000 r-xp 00000000 fd:02 2900                               /usr/libexec/heartbeat/send_arp
00602000-00603000 r--p 00002000 fd:02 2900                               /usr/libexec/heartbeat/send_arp
00603000-00604000 rw-p 00003000 fd:02 2900                               /usr/libexec/heartbeat/send_arp
00eea000-00f0b000 rw-p 00000000 00:00 0                                  [heap]
7f2ac1303000-7f2ac1318000 r-xp 00000000 fd:02 8780                       /usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f2ac1318000-7f2ac1517000 ---p 00015000 fd:02 8780                       /usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f2ac1517000-7f2ac1518000 r--p 00014000 fd:02 8780                       /usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f2ac1518000-7f2ac1519000 rw-p 00015000 fd:02 8780                       /usr/lib64/libgcc_s-4.8.3-20140911.so.1
7f2ac1519000-7f2ac152f000 r-xp 00000000 fd:02 4131                       /usr/lib64/libpthread-2.17.so
7f2ac152f000-7f2ac172f000 ---p 00016000 fd:02 4131                       /usr/lib64/libpthread-2.17.so
7f2ac172f000-7f2ac1730000 r--p 00016000 fd:02 4131                       /usr/lib64/libpthread-2.17.so
7f2ac1730000-7f2ac1731000 rw-p 00017000 fd:02 4131                       /usr/lib64/libpthread-2.17.so
7f2ac1731000-7f2ac1735000 rw-p 00000000 00:00 0 
7f2ac1735000-7f2ac18eb000 r-xp 00000000 fd:02 4105                       /usr/lib64/libc-2.17.so
7f2ac18eb000-7f2ac1aeb000 ---p 001b6000 fd:02 4105                       /usr/lib64/libc-2.17.so
7f2ac1aeb000-7f2ac1aef000 r--p 001b6000 fd:02 4105                       /usr/lib64/libc-2.17.so
7f2ac1aef000-7f2ac1af1000 rw-p 001ba000 fd:02 4105                       /usr/lib64/libc-2.17.so
7f2ac1af1000-7f2ac1af6000 rw-p 00000000 00:00 0 
7f2ac1af6000-7f2ac1c27000 r-xp 00000000 fd:02 4541                       /usr/lib64/libglib-2.0.so.0.4000.0
7f2ac1c27000-7f2ac1e26000 ---p 00131000 fd:02 4541                       /usr/lib64/libglib-2.0.so.0.4000.0
7f2ac1e26000-7f2ac1e27000 r--p 00130000 fd:02 4541                       /usr/lib64/libglib-2.0.so.0.4000.0
7f2ac1e27000-7f2ac1e28000 rw-p 00131000 fd:02 4541                       /usr/lib64/libglib-2.0.so.0.4000.0
7f2ac1e28000-7f2ac1e29000 rw-p 00000000 00:00 0 
7f2ac1e29000-7f2ac1e4a000 r-xp 00000000 fd:02 4097                       /usr/lib64/ld-2.17.so
7f2ac203b000-7f2ac203f000 rw-p 00000000 00:00 0 
7f2ac2048000-7f2ac204a000 rw-p 00000000 00:00 0 
7f2ac204a000-7f2ac204b000 r--p 00021000 fd:02 4097                       /usr/lib64/ld-2.17.so
7f2ac204b000-7f2ac204c000 rw-p 00022000 fd:02 4097                       /usr/lib64/ld-2.17.so
7f2ac204c000-7f2ac204d000 rw-p 00000000 00:00 0 
7fffe248e000-7fffe24af000 rw-p 00000000 00:00 0                          [stack]
7fffe25a4000-7fffe25a6000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]
Aborted (core dumped)

Expected results:

Ran the same command on an enp3s0, and the command completed successfully

[root@deadpool:~]#  /usr/libexec/heartbeat/send_arp -i 500 -r 5 -p /var/run/resource-agents/send_arp-192.168.71.12 enp3s0 192.168.71.12 auto 192.168.71.12 ffffffffffff 
ARPING 192.168.71.12 from 192.168.71.12 enp3s0
Sent 5 probes (5 broadcast(s))


Additional info:
The equivalent arping command 'arping -q -c 200 -U -I ib0 100.64.78.12' works, substituted for ipoibarping in IPaddr2, since ipoibarping seems deprecated and does not seem available in any distro,
Comment 2 Andrew Beekhof 2015-08-05 23:26:50 UTC 
# rpm -qf /usr/libexec/heartbeat/send_arp
resource-agents-3.9.6-2.fc21.x86_64
Comment 4 Lars Ellenberg 2015-08-25 15:52:02 UTC 
BTW, I already prepared a pull request for this:
https://github.com/ClusterLabs/resource-agents/pull/654
Comment 5 Lars Ellenberg 2015-10-02 13:15:39 UTC 
Seeing that this is now re-assigned,
I'd like to point out that I merged this already, as you probably noticed
(see above link).

What I think still needs to be done (as in "nice to have")
is a slight cleanup of the IPaddr2 agent:
 - drop mention of ipoibping, and the associated log messages
 - maybe change calling conventions to that of arping
 - at some point drop the resource-agents send_arp,
   and require recent-enough iputils arping.

(And of course, package, test and ship it).

Cheers,
    Lars
Comment 6 Fabio Massimo Di Nitto 2015-10-06 04:02:50 UTC 
(In reply to Lars Ellenberg from comment #5)
> Seeing that this is now re-assigned,
> I'd like to point out that I merged this already, as you probably noticed
> (see above link).

Hi Lars, yes new resource agent maintainer on the way :)

> 
> What I think still needs to be done (as in "nice to have")
> is a slight cleanup of the IPaddr2 agent:
>  - drop mention of ipoibping, and the associated log messages
>  - maybe change calling conventions to that of arping
>  - at some point drop the resource-agents send_arp,
>    and require recent-enough iputils arping.
> 
> (And of course, package, test and ship it).
> 
> Cheers,
>     Lars
Comment 8 Oyvind Albrigtsen 2015-12-08 15:19:04 UTC 
Tested and working on ethernet and infiniband.
Comment 11 michal novacek 2016-09-02 07:06:37 UTC 
I have verified that the patch is present in
resource-agents-3.9.5-81.el7.src.rpm as
bz1250728-send_arp-fix-buffer-overflow-on-infiniband.patch and that the package
compiles with that patch.
Comment 13 errata-xmlrpc 2016-11-03 23:58:17 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-2174.html