Bug 1286035

(In reply to Marcel Kolaja from comment #0)
> How reproducible:
> Not reproduced. It might be possible to reproduce it by sending duplicate
> packets (see below).

Confirmed. The bug can be reproduced by duplicating the packets coming from the server to the client. See below:

> Steps to Reproduce:

1.
Route traffic from the server to the client through a router where packets can be duplicated.

2.
Duplicate all packets coming from the server to the client. This can be done for instance using netem and tc on the outgoing interface:

[root@router ~]# tc qdisc add dev virbr1 root netem duplicate 100%
[root@router ~]# 

3.
Query power status using ipmitool:

[mkolaja@rhel7 ~]$ ipmitool -I lanplus -H host -U user -P password chassis power status
Error: Received an Unexpected Open Session Response
Error: Received an Unexpected RAKP 2 message
packet session id 0x0 does not match active session 0xa0a2a3a4
ipmitool: lanplus.c:1172: read_session_data_v2x: Assertion `0' failed.
Aborted
[mkolaja@rhel7 ~]$

If I understand the code correctly, we go through ipmi_lan_poll_recv() with each packet. In that function, the data are read with read_session_data(). After that, it verifies whether the packet is expected. It contains the code for handling unexpected RAKP 4 messages too:

--snip

                /*
                 * RAKP 4
                 */
                else if (rsp->session.payloadtype == IPMI_PAYLOAD_TYPE_RAKP_4)
                {
                        if (session->v2_data.session_state != LANPLUS_STATE_RAKP_3_SENT)
                        {
                                lprintf(LOG_ERR, "Error: Received an Unexpected RAKP 4 message");
                                rsp = ipmi_lan_recv_packet(intf);
                                continue;
                        }

                        read_rakp4_message(rsp, offset, session->v2_data.auth_alg);
                        break;
                }

--snip

The problem, however, is that we crash in read_session_data() already. read_session_data_v2x() is called from read_session_data():

--snip

        if (rsp->session.authtype == IPMI_SESSION_AUTHTYPE_RMCP_PLUS)
                read_session_data_v2x(rsp, offset, s);
        else
                read_session_data_v15(rsp, offset, s);

--snip

In read_session_data_v2x(), we verify that the packet session ID matches the active session:

--snip

        /*
         * Verify that the session ID is what we think it should be
         */
        if ((s->v2_data.session_state == LANPLUS_STATE_ACTIVE) &&
                (rsp->session.id != s->v2_data.console_id))
        {
                lprintf(LOG_ERR, "packet session id 0x%x does not "
                        "match active session 0x%0x",
                        rsp->session.id, s->v2_data.console_id);
                assert(0);
        }

--snip

It does not match, of course, because the packet is a RAKP 4 message with ID 0x0, while there's an active session already, which means that the session ID is already set. Therefore, we call assert(0) and crash.

Created attachment 1107977 [details]
proposed patch for RHEL 7.1

This patch addresses the bug. It's not clear to me why exactly the different IDs are handled this way, so I'm not sure about possible regressions here.

Created attachment 1112086 [details]
proposed patch for RHEL 7.1

backport of the patch in the upstream bug:

http://sourceforge.net/p/ipmitool/bugs/407/

FYI: The patch was accepted upstream so we are good to go, here.

dist-git commits related to build ipmitool-1.8.15-3.el7:
http://pkgs.devel.redhat.com/cgit/rpms/ipmitool/commit/?h=rhel-7.3&id=3acda4dc7ebc7ed13b65f88fa34fb2417eac4331

Hi ,

Could you please confirm that does this issue exists in RHEL6.6 also.

Hi Anant, yes this is also reproducible in RHEL 6.8, see below:

- On the host system I used tc to duplicate packets on the virtual bridge
# tc qdisc add dev virbr0 root netem duplicate 100%
# tc qdisc show dev virbr0
qdisc netem 8002: root refcnt 2 limit 1000 duplicate 100%

- On the virtual machine, I performed a query of chassis power status in which it failed and core dumped:
 
[root@localhost ~]# uname -r
2.6.32-642.el6.x86_64

[root@localhost ~]# rpm -q ipmitool
ipmitool-1.8.15-2.el6.x86_64

[root@localhost ~]# ipmitool -I lanplus -H hp-ilo -U redacted -E -L redacted chassis power status
Error: Received an Unexpected Open Session Response
Error: Received an Unexpected RAKP 2 message
packet session id 0x0 does not match active session 0xa0a2a3a4
ipmitool: lanplus.c:1184: read_session_data_v2x: Assertion `0' failed.
Aborted (core dumped)

(In reply to Rachel Sibley from comment #22)
> Hi Anant, yes this is also reproducible in RHEL 6.8, see below:
> 
> - On the host system I used tc to duplicate packets on the virtual bridge
> # tc qdisc add dev virbr0 root netem duplicate 100%
> # tc qdisc show dev virbr0
> qdisc netem 8002: root refcnt 2 limit 1000 duplicate 100%
> 
> - On the virtual machine, I performed a query of chassis power status in
> which it failed and core dumped:
>  
> [root@localhost ~]# uname -r
> 2.6.32-642.el6.x86_64
> 
> [root@localhost ~]# rpm -q ipmitool
> ipmitool-1.8.15-2.el6.x86_64
> 
> [root@localhost ~]# ipmitool -I lanplus -H hp-ilo -U redacted -E -L redacted
> chassis power status
> Error: Received an Unexpected Open Session Response
> Error: Received an Unexpected RAKP 2 message
> packet session id 0x0 does not match active session 0xa0a2a3a4
> ipmitool: lanplus.c:1184: read_session_data_v2x: Assertion `0' failed.
> Aborted (core dumped)

Thank you for the response.

ALL TESTS PASSED

No longer seeing 'ipmitool: lanplus.c:1184: read_session_data_v2x: Assertion `0' failed.' error message or core dump using the latest ipmitool pkg (ipmitool-1.8.15-7.el7.x86_64).

[root@localhost ~]# uname -r
3.10.0-500.el7.x86_64
[root@localhost ~]# rpm -q ipmitool
ipmitool-1.8.15-7.el7.x86_64

- On the virtual machine, I performed a query of chassis power status in
localhost login: root
Password:
[root@localhost ~]#  export IPMITOOL_PASSWORD=password
[root@localhost ~]# ipmitool -I lanplus -H hp-ms-03-ilo.mgmt.lab.eng.rdu.redhat.com -U Administrator -E -L Administrator -B 0 -T 0x82 -b 7 -t 0x72 chassis power status
Chassis Power is on

- On the host system I used tc to duplicate packets on the virtual bridge
[root@dell-per510-01 ~]# tc qdisc add dev virbr0 root netem duplicate 100%
[root@dell-per510-01 ~]# tc qdisc show dev virbr0
qdisc netem 8001: root refcnt 2 limit 1000 duplicate 100%

- Back on virtual machine, I attempt to query the chassis power, but no longer see 'ipmitool: lanplus.c:1184: read_session_data_v2x: Assertion `0' failed.' error message and no longer seeing a core dump.
[root@localhost ~]# ipmitool -I lanplus -H hp-ms-03-ilo.mgmt.lab.eng.rdu.redhat.com -U Administrator -E -L Administrator -B 0 -T 0x82 -b 7 -t 0x72 chassis power status
Error: Received an Unexpected Open Session Response
Error: Received an Unexpected RAKP 2 message
ERROR: Received an Unexpected message ID

Moving back to Verified State as it was incorrectly moved back to Assigned when switching the package owner.

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHEA-2016-2159.html

Hi All,

I just wanted to confirm that, Does this issue occur with below command only
ipmitool -I lanplus -H hp-ilo -U redacted -E -L redacted chassis power status

ie using ipmitool with lanplus interface and there is no impact of this issue on other ipmitool commands.