Bug 1230992 - ssh "Corrupted MAC on input." from using wrong MAC method
Summary: ssh "Corrupted MAC on input." from using wrong MAC method
Keywords:
Status: CLOSED NOTABUG
Alias: None
Product: Fedora
Classification: Fedora
Component: openssh
Version: 22
Hardware: x86_64
OS: Linux
unspecified
medium
Target Milestone: ---
Assignee: Petr Lautrbach
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2015-06-12 00:58 UTC by Jim Wilson
Modified: 2015-06-15 17:34 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-06-15 17:34:14 UTC
Type: ---
Embargoed:


Attachments (Terms of Use)

Description Jim Wilson 2015-06-12 00:58:43 UTC
User-Agent:       Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:38.0) Gecko/20100101 Firefox/38.0
Build Identifier: 

ssh to older ARM system running arch always fails with 

   "Corrupted MAC on input.
    ssh_packet_read: message authentication code incorrect"

but always succeeds invoked with '-m  <list_from_ssh_config>'


Reproducible: Always

Steps to Reproduce:
1. ssh <to 3.1.10-15ARCH on ARM with libssh2 1.4.3-1, openssh 6.1p1-4>

Actual Results:  
Corrupted MAC on input.
ssh_packet_read: message authentication code incorrect

Expected Results:  
command-line prompt from remote (no UID/PW prompt; I'm using crypto key authentication).

/etc/ssh/ssh_config sez the client defaults to:

#   MACs hmac-md5,hmac-sha1,umac-64,hmac-ripemd160

and it works when I specify these on ssh command line:

  'ssh -m hmac-md5,hmac-sha1,umac-64,hmac-ripemd160 ...'

or when I specify any but the third individually.

'ssh -Q mac' reports the client supports hmac-sha1, hmac-sha1-96, hmac-sha2-256, hmac-sha2-512, hmac-md5, hmac-md5-96, hmac-ripemd160, hmac-ripemd160, umac-64, umac-128, hmac-sha1-etm, hmac-sha1-96-etm
hmac-sha2-256-etm, hmac-sha2-512-etm, hmac-md5-etm, hmac-md5-96-etm, hmac-ripemd160-etm
umac-64-etm, and umac-128-etm

Only:

  'ssh -m umac-64 ...' produces the same error.

Note that this scheme is third on the list; the earlier two (working) schemes should be preferred; the first should succeed.

Comment 1 Jakub Jelen 2015-06-12 06:37:48 UTC
Hi. Can you make sure what versions of openssl are you using on client and server? I just read about openssl ABI change in openssl, which can be attribute for non-working MAC. It was worked around in current openssh, but other software can suffer with such problem.

http://lists.mindrot.org/pipermail/openssh-unix-dev/2015-June/034061.html

I can't reproduce it with current fedora openssh versions and I don't have old ARM machine around.

Can you make sure what MAC is finally used when you specify -m switch? It should be visible with -vvv switch in output as a lines:
> debug1: kex: server->client aes128-ctr hmac-sha1-etm none
> debug1: kex: client->server aes128-ctr hmac-sha1-etm none
(should be same)

Ssh is choosing the first matching MAC from the list provided by both parties so if some of them is first for a client, it doesn't mean it is first for a server. Full log (-vvv) from client and/or from server would help to explain the behaviour behind it.

Comment 2 Jim Wilson 2015-06-12 14:10:06 UTC
> versions of openssl are you using on client and server?

client: openssl-1.0.1k-8.fc22.x86_64 (from 'rpm -q openssl')
server: openssl 1.0.1.c-1            (from 'pacman -Q openssl')

I have a newer ARM server, also running ARCH, that doesn't exhibit this problem.  It's using openssl 1.0.2.a-1; this seems to support your hypothesis.

> what MAC is finally used when you specify -m switch?

'Ssh -m hmac-md5,hmac-sha1,umac-64,hmac-ripemd160 -vvv ...' announces:

> debug1: kex: server->client aes128-ctr hmac-md5 none
> debug1: kex: client->server aes128-ctr hmac-md5 none

as I would have expected, given the ordering of the -m options.  I interpreted the ssh (client) man page to mean the client would try these in turn until it found one the server supported.  The negotiation could be considerably more complex, but I'm a simple-minded guy and tend to deny that possibility.

FWIW, 'ssh -v ...' (no -m switch and one 'v' is enough) reports:

> debug1: kex: server->client aes128-ctr umac-64 none
> debug1: kex: client->server aes128-ctr umac-64 none

And this is the one that fails with '-m umac-64'.  (I wish I had seen this; it would have saved me a lot of laboriously typing each of the various MACs after '-m').

Further, if I uncomment the MACs line in /etc/ssh_config, it gives me the same joy as '-m hmac-md5,hmac-sha1,umac-64,hmac-ripemd160' on the command line.  Perhaps the compiled-in options simply don't agree with the documented defaults in /etc/ssh_config?

That does suggest an easy workaround for me: a host-specific ~/.ssh/config that omits the offending MAC for my broken host.  I just tried it, and it works.

Comment 3 Tomas Mraz 2015-06-12 14:22:09 UTC
I do not think it is related to the HMAC abi breakage upstream - this happened only in the security update release yesterday. It is much more possible that the umac-64 implementation is simply broken on the old ARM system you're connecting to.

Comment 4 Jim Wilson 2015-06-12 17:06:10 UTC
I agree, the older ARM implementation is probably borked; I recall openssh/openssl were recently discovered to be spectacularly flawed, and my ARM box may predate that discovery.  However, I have worked around any collateral damage from that borkage.

The real question remaining is why the Fedora ssh client disobeys its documentation and attempts to use the broken method since it is way down on the *documented* list of choices.

Hmac-md5 is the preferred choice.  And, it is chosen and successfully used when the default list of choices is overridden on the command line by *excactly the same list*.

Comment 5 Jakub Jelen 2015-06-15 06:08:02 UTC
This was the reason why I wanted you to post your debug output from connection using -vvv to investigate what is your ARM box proposing.

By the way, what is written in your default ssh_config is not a default, but example what can be used. You can see defaults in ssh_config manual pages, which is little bit loner:
> The default is:
> umac-64-etm,umac-128-etm,
> hmac-sha2-256-etm,hmac-sha2-512-etm,
> umac-64,umac-128,
> hmac-sha2-256,hmac-sha2-512,
> hmac-md5-etm,hmac-sha1-etm,
> hmac-ripemd160-etm,
> hmac-sha1-96-etm,hmac-md5-96-etm,
> hmac-md5,hmac-sha1,hmac-ripemd160,
> hmac-sha1-96,hmac-md5-96

Of course we can fit all of them also into the example ssh_config, but it would be real mess.

Comment 6 Jim Wilson 2015-06-15 17:34:14 UTC
I see now, I misinterpreted the ssh man page:

"For full details of the options listed below, and their possible values, see ssh_config(5)."

This directs me not to the file, .../ssh_config, but to its man page.  I see that now, and given the order of defaults, the prosecution moves for dismissal of all charges.

Or perhaps ssh can just plead guilty to the misdemeanor violation of Wilson's law: "No man page should exceed 24 lines."


Note You need to log in before you can comment on or make changes to this bug.