It was discovered that the AMQP type decoder was exposed pre-authentication because it was possible to send arbitrary types in the client-properties map in a connection.start-ok message. This is used to send an array with elements which are all of width zero and thus consume no space on the wire, but need storage after decoding by the server. On some systems, a suitably chosen SIZE value triggers the OOM killer and terminates the server process permanently.
Acknowledgements:
This issue was discovered by Florian Weimer of the Red Hat Product Security Team.