Bug 803205 - possible memory corruption with buffer overflow due incorrect padding calculation in nss crypto
Summary: possible memory corruption with buffer overflow due incorrect padding calcula...
Keywords:
Status: CLOSED CANTFIX
Alias: None
Product: Corosync Cluster Engine
Classification: Retired
Component: totem
Version: 1.4
Hardware: Unspecified
OS: Unspecified
high
high
Target Milestone: ---
Assignee: Jan Friesse
QA Contact:
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2012-03-14 08:19 UTC by Fabio Massimo Di Nitto
Modified: 2012-03-16 21:34 UTC (History)
3 users (show)

Fixed In Version:
Clone Of:
Environment:
Last Closed: 2012-03-14 15:30:22 UTC
Embargoed:


Attachments (Terms of Use)

Description Fabio Massimo Di Nitto 2012-03-14 08:19:55 UTC
Description of problem:

The nss documentation requires that the output buffer for

        rv1 = PK11_CipherOp(enc_context, outdata,
                            &tmp1_outlen, FRAME_SIZE_MAX - sizeof(struct security_header),
                            data, datalen);

(as seen in both totemudp.c and udpu.c) to be at least inbuf + blocksize.

In flatiron (that can only use aes256 with fixed blocksize of 16) we never
check for that boundary.

The net_mtu adjustment routines don´t take that blocksize and padding into account.

In case a user has configured both nss, a net_mtu of 10000 and it is sending packets bigger than 10000 - sizeof(security_header), the padding will overflow the buffer.

Some debugging code shows (one example)

Mar 14 08:14:52 alert   [TOTEM ] crypt: input: 70 tmp1: 64 tmp2: 16

and translates to:
input buffer len: 70 bytes
tmp1_outlen:      64 bytes
tmp2_outlen:      16 bytes

final output len is:

        *buf_len = tmp1_outlen + tmp2_outlen;

that equals 80.

(note those padding values are dependent on the input buffer len and might vary
 based on what the cluster is doing)


Version-Release number of selected component (if applicable):

flatiron 1.*

How reproducible:

while it might be very rare to see such scenario, in theory it is possible to hit the overflow on every packet.

Comment 1 Jan Friesse 2012-03-14 15:30:22 UTC
I was testing that and my results are following:
- overflow will never happen because thanks to all mtu_adjustements everywhere, maximum message size which will arrive to encrypt_and_sign_nss is 9936 bytes, but output buffer is 10000 bytes -> plenty of space
- this (even larger then expected) message is then directly sent. May result in fragmentation

So problem there is really not overflow but possibility of havim packet larger then expected and fragmentation.

Sadly solution in flatiron seems to be not trivial or almost impossible, because:
- adjust_net_mtu don't know at beginning if we will use nss/sober AND this can be switched in runtime
- sober doesn't have this problems because it's stream cypher
- it's possible to ALWAYS subtract 16 bytes from mtu, but this will result in smaller maximum message size for nss (correct) but also for sober

In other words, we are doomed there. From two evils (reduce message size) and not correct packet size in NSS, I'm choosing second one, because sober is wider used in cluster environment.

So closing this bug as CANTFIX


Note You need to log in before you can comment on or make changes to this bug.