Bug 1286035 - ipmitool: lanplus.c:1172: read_session_data_v2x: Assertion `0' failed.
ipmitool: lanplus.c:1172: read_session_data_v2x: Assertion `0' failed.
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: ipmitool (Show other bugs)
7.1
x86_64 Linux
unspecified Severity high
: rc
: ---
Assigned To: Josef Ridky
Rachel Sibley
: Reproducer
Depends On:
Blocks: 1203710 1313485 1289025 1295829
  Show dependency treegraph
 
Reported: 2015-11-27 05:04 EST by Marcel Kolaja
Modified: 2017-04-19 00:32 EDT (History)
7 users (show)

See Also:
Fixed In Version: ipmitool-1.8.15-3.el7
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-11-03 19:30:08 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)
proposed patch for RHEL 7.1 (529 bytes, patch)
2015-12-20 08:48 EST, Marcel Kolaja
no flags Details | Diff
proposed patch for RHEL 7.1 (1.55 KB, patch)
2016-01-06 05:37 EST, Marcel Kolaja
no flags Details | Diff

  None (edit)
Description Marcel Kolaja 2015-11-27 05:04:53 EST
Description of problem:
ipmitool crashes with the following error output:

packet session id 0x0 does not match active session 0xa0a2a3a4
ipmitool: lanplus.c:1172: read_session_data_v2x: Assertion `0' failed.

Version-Release number of selected component (if applicable):
ipmitool-1.8.13-8.el7_1.x86_64

How reproducible:
Not reproduced. It might be possible to reproduce it by sending duplicate packets (see below).

Steps to Reproduce:
N/A

Actual results:
ipmitool crashes

Expected results:
ipmitool does not crash

Additional info:
It was found that the issue occurs after ipmitool opens a session upon receiving RAKP Message 4, sends an IPMI command in the open session but then (instead of a response to the IPMI command) receives a duplicate of the previously received RAKP Message 4. That packet's session id is 0x0, which does not match the active session and ipmitool crashes.
Comment 6 Marcel Kolaja 2015-12-18 09:22:32 EST
(In reply to Marcel Kolaja from comment #0)
> How reproducible:
> Not reproduced. It might be possible to reproduce it by sending duplicate
> packets (see below).

Confirmed. The bug can be reproduced by duplicating the packets coming from the server to the client. See below:

> Steps to Reproduce:

1.
Route traffic from the server to the client through a router where packets can be duplicated.

2.
Duplicate all packets coming from the server to the client. This can be done for instance using netem and tc on the outgoing interface:

[root@router ~]# tc qdisc add dev virbr1 root netem duplicate 100%
[root@router ~]# 

3.
Query power status using ipmitool:

[mkolaja@rhel7 ~]$ ipmitool -I lanplus -H host -U user -P password chassis power status
Error: Received an Unexpected Open Session Response
Error: Received an Unexpected RAKP 2 message
packet session id 0x0 does not match active session 0xa0a2a3a4
ipmitool: lanplus.c:1172: read_session_data_v2x: Assertion `0' failed.
Aborted
[mkolaja@rhel7 ~]$
Comment 7 Marcel Kolaja 2015-12-20 07:59:51 EST
If I understand the code correctly, we go through ipmi_lan_poll_recv() with each packet. In that function, the data are read with read_session_data(). After that, it verifies whether the packet is expected. It contains the code for handling unexpected RAKP 4 messages too:

--snip

                /*
                 * RAKP 4
                 */
                else if (rsp->session.payloadtype == IPMI_PAYLOAD_TYPE_RAKP_4)
                {
                        if (session->v2_data.session_state != LANPLUS_STATE_RAKP_3_SENT)
                        {
                                lprintf(LOG_ERR, "Error: Received an Unexpected RAKP 4 message");
                                rsp = ipmi_lan_recv_packet(intf);
                                continue;
                        }

                        read_rakp4_message(rsp, offset, session->v2_data.auth_alg);
                        break;
                }

--snip

The problem, however, is that we crash in read_session_data() already. read_session_data_v2x() is called from read_session_data():

--snip

        if (rsp->session.authtype == IPMI_SESSION_AUTHTYPE_RMCP_PLUS)
                read_session_data_v2x(rsp, offset, s);
        else
                read_session_data_v15(rsp, offset, s);

--snip

In read_session_data_v2x(), we verify that the packet session ID matches the active session:

--snip

        /*
         * Verify that the session ID is what we think it should be
         */
        if ((s->v2_data.session_state == LANPLUS_STATE_ACTIVE) &&
                (rsp->session.id != s->v2_data.console_id))
        {
                lprintf(LOG_ERR, "packet session id 0x%x does not "
                        "match active session 0x%0x",
                        rsp->session.id, s->v2_data.console_id);
                assert(0);
        }

--snip

It does not match, of course, because the packet is a RAKP 4 message with ID 0x0, while there's an active session already, which means that the session ID is already set. Therefore, we call assert(0) and crash.
Comment 8 Marcel Kolaja 2015-12-20 08:48 EST
Created attachment 1107977 [details]
proposed patch for RHEL 7.1

This patch addresses the bug. It's not clear to me why exactly the different IDs are handled this way, so I'm not sure about possible regressions here.
Comment 14 Marcel Kolaja 2016-01-06 05:37 EST
Created attachment 1112086 [details]
proposed patch for RHEL 7.1

backport of the patch in the upstream bug:

http://sourceforge.net/p/ipmitool/bugs/407/
Comment 18 Boris Ranto 2016-01-25 02:45:49 EST
FYI: The patch was accepted upstream so we are good to go, here.
Comment 19 Boris Ranto 2016-02-18 11:01:11 EST
dist-git commits related to build ipmitool-1.8.15-3.el7:
http://pkgs.devel.redhat.com/cgit/rpms/ipmitool/commit/?h=rhel-7.3&id=3acda4dc7ebc7ed13b65f88fa34fb2417eac4331
Comment 21 anant 2016-04-27 00:43:46 EDT
Hi ,

Could you please confirm that does this issue exists in RHEL6.6 also.
Comment 22 Rachel Sibley 2016-05-05 12:04:00 EDT
Hi Anant, yes this is also reproducible in RHEL 6.8, see below:

- On the host system I used tc to duplicate packets on the virtual bridge
# tc qdisc add dev virbr0 root netem duplicate 100%
# tc qdisc show dev virbr0
qdisc netem 8002: root refcnt 2 limit 1000 duplicate 100%

- On the virtual machine, I performed a query of chassis power status in which it failed and core dumped:
 
[root@localhost ~]# uname -r
2.6.32-642.el6.x86_64

[root@localhost ~]# rpm -q ipmitool
ipmitool-1.8.15-2.el6.x86_64

[root@localhost ~]# ipmitool -I lanplus -H hp-ilo -U redacted -E -L redacted chassis power status
Error: Received an Unexpected Open Session Response
Error: Received an Unexpected RAKP 2 message
packet session id 0x0 does not match active session 0xa0a2a3a4
ipmitool: lanplus.c:1184: read_session_data_v2x: Assertion `0' failed.
Aborted (core dumped)
Comment 23 anant 2016-05-06 03:41:00 EDT
(In reply to Rachel Sibley from comment #22)
> Hi Anant, yes this is also reproducible in RHEL 6.8, see below:
> 
> - On the host system I used tc to duplicate packets on the virtual bridge
> # tc qdisc add dev virbr0 root netem duplicate 100%
> # tc qdisc show dev virbr0
> qdisc netem 8002: root refcnt 2 limit 1000 duplicate 100%
> 
> - On the virtual machine, I performed a query of chassis power status in
> which it failed and core dumped:
>  
> [root@localhost ~]# uname -r
> 2.6.32-642.el6.x86_64
> 
> [root@localhost ~]# rpm -q ipmitool
> ipmitool-1.8.15-2.el6.x86_64
> 
> [root@localhost ~]# ipmitool -I lanplus -H hp-ilo -U redacted -E -L redacted
> chassis power status
> Error: Received an Unexpected Open Session Response
> Error: Received an Unexpected RAKP 2 message
> packet session id 0x0 does not match active session 0xa0a2a3a4
> ipmitool: lanplus.c:1184: read_session_data_v2x: Assertion `0' failed.
> Aborted (core dumped)

Thank you for the response.
Comment 25 Rachel Sibley 2016-09-21 12:05:12 EDT
ALL TESTS PASSED

No longer seeing 'ipmitool: lanplus.c:1184: read_session_data_v2x: Assertion `0' failed.' error message or core dump using the latest ipmitool pkg (ipmitool-1.8.15-7.el7.x86_64).

[root@localhost ~]# uname -r
3.10.0-500.el7.x86_64
[root@localhost ~]# rpm -q ipmitool
ipmitool-1.8.15-7.el7.x86_64

- On the virtual machine, I performed a query of chassis power status in
localhost login: root
Password:
[root@localhost ~]#  export IPMITOOL_PASSWORD=password
[root@localhost ~]# ipmitool -I lanplus -H hp-ms-03-ilo.mgmt.lab.eng.rdu.redhat.com -U Administrator -E -L Administrator -B 0 -T 0x82 -b 7 -t 0x72 chassis power status
Chassis Power is on

- On the host system I used tc to duplicate packets on the virtual bridge
[root@dell-per510-01 ~]# tc qdisc add dev virbr0 root netem duplicate 100%
[root@dell-per510-01 ~]# tc qdisc show dev virbr0
qdisc netem 8001: root refcnt 2 limit 1000 duplicate 100%

- Back on virtual machine, I attempt to query the chassis power, but no longer see 'ipmitool: lanplus.c:1184: read_session_data_v2x: Assertion `0' failed.' error message and no longer seeing a core dump.
[root@localhost ~]# ipmitool -I lanplus -H hp-ms-03-ilo.mgmt.lab.eng.rdu.redhat.com -U Administrator -E -L Administrator -B 0 -T 0x82 -b 7 -t 0x72 chassis power status
Error: Received an Unexpected Open Session Response
Error: Received an Unexpected RAKP 2 message
ERROR: Received an Unexpected message ID
Comment 27 Rachel Sibley 2016-10-20 08:16:44 EDT
Moving back to Verified State as it was incorrectly moved back to Assigned when switching the package owner.
Comment 29 errata-xmlrpc 2016-11-03 19:30:08 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2159.html
Comment 30 Vinay 2017-04-19 00:32:21 EDT
Hi All,

I just wanted to confirm that, Does this issue occur with below command only
ipmitool -I lanplus -H hp-ilo -U redacted -E -L redacted chassis power status

ie using ipmitool with lanplus interface and there is no impact of this issue on other ipmitool commands.

Note You need to log in before you can comment on or make changes to this bug.