Bug 1402029

Summary: openssh-7.3p1-6.fc25.x86_64 fails to receive rsa certificate added to a remotely forwarded agent.
Product: [Fedora] Fedora Reporter: Peter Moody <pmoody>
Component: opensshAssignee: Jakub Jelen <jjelen>
Status: CLOSED ERRATA QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 24CC: evan, jjelen, mattias.ellert, mgrepl, plautrba, tmraz
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: openssh-7.3p1-7.fc25 Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-12-09 22:30:09 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
proposed dist-git patch none

Description Peter Moody 2016-12-06 15:29:21 UTC
Description of problem:

fedora clients running openssh 7.3p1-5 and above (which is I guess just fedora 24 and above) which connect to our ssh certificate authority fail to receive a valid certificate on their forwarded ssh-agent. the same clients running 7.3p1-4 and below receive the certificate just fine.

Version-Release number of selected component (if applicable):

fedora 24.
openssh 7.3p1 -4 and above

How reproducible:

client running the above connects to our ssh certificate authority and successfully authenticates.

Steps to Reproduce:
1. n/a
2. 
3.

Actual results:


Expected results:


Additional info:

ssh agent debug log from openssh-7.3p1-6.fc25.x86_64. the CA adds two keys, the first is just the rsa private key, the second is the private key and the certificate. Both are constrained (type 25).

debug2: fd 3 setting O_NONBLOCK
debug2: fd 4 setting O_NONBLOCK
debug1: type 11
debug1: type 11
debug2: fd 5 setting O_NONBLOCK
debug1: XXX shrink: 4 < 5
debug2: fd 5 setting O_NONBLOCK
debug1: type 11
debug1: type 25
debug1: type 25
process_add_identity: decode private key: error in libcrypto
debug1: XXX shrink: 4 < 5
debug1: XXX shrink: 3 < 4


emailed jjelen and he responded with

`
Hello,
if the issue is introduced by this change, there must be a bug in the compat changes for openssl-1.1.0. We tested the changes quite extensively to avoid such problems, but it is possible we missed something.

Having a look into the code, this is most probably the path [1] that was very questionable (agent does not provide N and E parameters and they are required in the new API when constructing RSA keys). So RSA_set0_key() or RSA_set0_factors() is probably failing for some reason. The only way to find out which is probably under GDB. I will try to reproduce the behavior and to see how I can fix it.

Thank you for letting me know about the problem. To properly track the problem, bug would be fine.

[1] http://pkgs.fedoraproject.org/cgit/rpms/openssh.git/tree/openssh-7.3p1-openssl-1.1.0.patch#n3634
`

this is an in-house ssh ca that accepts an ssh connection and then directly adds keys/certificates to the ssh-agent that's forwarded by the client. I'll see if I can get my user to repro the issue by just doing something like running 'ssh-add ./path/to/cert' with a forwarded ssh-agent. If you're having any trouble reproducing, I can write a pretty minimal ssh-ca that should replicate the issue for you.

Comment 1 Peter Moody 2016-12-06 19:58:34 UTC
the user who originally reported the bug to me replied that he can repro the issue with the following commands.


# in terminal one
$ ssh-agent -d -a /tmp/ssh.sock

# in terminal two
$ SSH_AUTH_SOCK=/tmp/ssh.sock ssh -A localhost
# auth
$ mkdir keys ; chmod 0700 keys
$ ssh-keygen -f keys/user_ca
$ ssh-keygen -f keys/test_user_key
$ ssh-keygen -s keys/user_ca -I user_foo -n foo -V +1w keys/test_user_key.pub
$ ssh-add keys/test_user_key

Comment 2 Jakub Jelen 2016-12-08 12:42:30 UTC
Created attachment 1229459 [details]
proposed dist-git patch

Thank you for letting me know about the problem and filling the reproducer. It fails for both Fedora 25 and 26.

The problem is, that the ssh-agent API does not contain N and E RSA parameters and the net OpenSSL API required them. We set them to 0 (or undefined), which led the later function  RSA_blinding_on() to the failure, because it is using the N value as a random seed (or something like that).

The attached patch solves the problems for me. I will try to put together also some unit test to make sure we will not break such use case again.

Comment 3 Fedora Update System 2016-12-08 14:27:44 UTC
openssh-7.3p1-7.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2016-af7b5cd77d

Comment 4 Fedora Update System 2016-12-08 19:27:02 UTC
openssh-7.3p1-7.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2016-af7b5cd77d

Comment 5 Evan Klitzke 2016-12-09 19:32:00 UTC
I'm the user who originally had problems, and I can confirm that the new build fixes the issue for me.

Comment 6 Fedora Update System 2016-12-09 22:30:09 UTC
openssh-7.3p1-7.fc25 has been pushed to the Fedora 25 stable repository. If problems still persist, please make note of it in this bug report.