Bug 1594847

Summary: smartcard access appears to be broken for curl/git
Product: [Fedora] Fedora Reporter: James Hubbard <jameshubbard>
Component: curlAssignee: Kamil Dudka <kdudka>
Status: CLOSED DUPLICATE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: high Docs Contact:
Priority: unspecified    
Version: 27CC: dwmw2, jameshubbard, john.j5live, kdudka, nmavrogi, paul
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2018-09-05 14:53:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description James Hubbard 2018-06-25 14:24:35 UTC
Description of problem:
Updated from Fedora 26 to Fedora 27. 
Prior to upgrade using git to access a repository over https via smartcard worked correctly.  

After update it no longer works. 

Version-Release number of selected component (if applicable):
curl 7.55.1-12
git 2.14.4-1

Fully up to date Fedora 27 install 

How reproducible:
Difficult unless you have access to smartcard enabled infrastructure

Steps to Reproduce:
1.
2.
3.

Actual results:
The following is used to access the git info
curl --verbose -k -v -E "token\:CAC Email Signature Certificate" --pass <pin> https://<host>/git/info   
 Trying <ip address>...
* TCP_NODELAY set
* Connected to <host> (<ipaddr>) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* could not load PEM client certificate, OpenSSL error error:02001002:system library:fopen:No such file or directory, (no key found, wrong pass phrase, or wrong file format?)
* Closing connection 0
curl: (58) could not load PEM client certificate, OpenSSL error error:02001002:system library:fopen:No such file or directory, (no key found, wrong pass phrase, or wrong file format?)

Output of git against the CAC enabled https repo is
GIT_CURL_VERBOSE=1 GIT_TRACE=1 git fetch
....
* ALPN, offering http/1.1
* could not load PEM client certificate, OpenSSL error error:02001002:system library:fopen:No such file or directory, (no key found, wrong pass phrase, or wrong file format?)
* Closing connection 0
fatal: unable to access '<url>': could not load PEM client certificate, OpenSSL error error:02001002:system library:fopen:No such file or directory, (no key found, wrong pass phrase, or wrong file format?)


Expected results:

The curl command would list the repositories on the git server. 

git would fetch the repo

Additional info:
The information provided by modutil shows the proper smartcard info. The output is different from what F26 was showing.  It does show coolkey and the proper token. 
modutil -dbdir /etc/pki/nssdb/ -list
or 
modutil -dbdir sql:/etc/pki/nssdb/ -list


Using the following commands, certutil prompts for the pin and displays the proper information.
certutil -L  -d sql:/etc/pki/nssdb/ -h all
or 
certutil -L  -d /etc/pki/nssdb/ -h all


The following is being used to set various pieces needed to enable git to work.
export CACID="<token>"
export GIT_SSL_CERT_PASSWORD_PROTECTED=1
export GIT_SSL_KEY="$CACID:CAC Email Signature Certificate"
export GIT_SSL_CERT="$CACID:CAC Email Signature Certificate"
export GIT_SSL_KEYTYPE=ENG
export GIT_SSL_CERTTYPE=ENG

The above works on RHEL 6/7.

Firefox and Chrome work properly when accessing the websites and urls.

Comment 1 James Hubbard 2018-06-26 13:23:05 UTC
Prior to Fedora 27 this worked due to the use of NSS by the curl/libcurl.  The switch to openssl has broken this.  

https://fedoraproject.org/wiki/Changes/libcurlBackToOpenSSL

Does anyone know if there's a magic incantation that will make this work again?  Otherwise, it looks like I'll need to recompile curl to make it work.

Comment 2 Kamil Dudka 2018-06-28 11:33:01 UTC
Did you try the --engine option of curl?

It is ignored when using NSS but, as far as I know, needed when using OpenSSL.

(In reply to James Hubbard from comment #1)
> Otherwise, it looks like I'll need to recompile curl to make it work.

Feel free to use my private (unsupported) copr for experimenting with this:

https://copr.fedorainfracloud.org/coprs/kdudka/curl-nss/

Comment 3 James Hubbard 2018-06-28 12:21:56 UTC
I attempted to use the -engine option, but nothing that I attempted worked.  If you have some suggestions for the proper options to pass, I'll try them.  

Using your repo for curl-nss works.  I was able to use git after installation. 
 I thought that I was going to have to patch the spec file and compile myself. 

If you know how to make this work with the openssl, I'm willing to experiment.  I noticed that having OpenSC installed was causing problems with Chrome prompting for pin frequently.  When using certutil I was being prompted for two pins.  One was PIV_II and the other was for the CAC.  

My biggest concern is that this git continues to function with https and smartcards when RHEL 8 comes out.  If it's broken, it will be a big problem. 

Thank you for your assistance.  Let me know if I can help.

Comment 4 David Woodhouse 2018-06-28 12:37:31 UTC
It should accept a standard PKCS#11 URI on the command line.

Try something like 'pkcs11:object=CAC Email Signature Certificate".

Use 'p11tool --list-all' to work out what the URI should be. There is some documentation at http://www.infradead.org/openconnect/pkcs11.html which is mostly relevant and should help identify the URI to use.

Comment 5 David Woodhouse 2018-06-28 12:40:35 UTC
kdudka, nobody should ever need to specify -engine in this case. That violates Fedora packaging guidelines. A simple PKCS#11 URI in place of the filename should always work.

See https://fedoraproject.org/wiki/Packaging:SSLCertificateHandling#Client_and_server_applications

  • Packages which use SSL certificates/keys from a file or elsewhere SHOULD also support using certs/keys from PKCS#11 tokens.

  • Where PKCS#11 objects are specified in a textual form which is visible to the user (e.g. on the command line or in a config file), objects SHOULD be specified in the form of a PKCS#11 URI as as described in RFC7512.

  • Packages which can use PKCS#11 tokens SHOULD automatically use the tokens which are present in the system's p11-kit configuration, rather than needing to have a PKCS#11 provider explicitly specified.

Comment 6 James Hubbard 2018-07-08 19:12:24 UTC
I setup another machine.  I did not install libcoolkey.  I used the suggested methods on the openconnect page.  Nothing appears to work with git. I get the same error message described above.  

I'm able to get the output that I expect from pkcs11_inspect and pkcs11_listcerts. I used the following to generate the urls for GIT_SSL_KEY and GIT_SSL_CERT.  None of the values that I used worked. 

p11tool --list-all-certs pkcs11:manufacturer=piv_II
p11tool --list-privkeys --login pkcs11:manufacturer=piv_II

Comment 7 David Woodhouse 2018-07-09 13:40:31 UTC
Thanks. So, to be clear, these work:

 openconnect -c 'pkcs11:manufacturer=piv_II;id=%01' $URL
 curl -E 'pkcs11:manufacturer=piv_II;id=%01' $URL

But setting the appropriate environment variables and invoking git does not?

Comment 8 Nikos Mavrogiannopoulos 2018-07-20 14:25:38 UTC
This issue is addressed by:
https://bugzilla.redhat.com/show_bug.cgi?id=1219544

The bottom line is that there is an upstream patch for curl/openssl which has not been accepted yet:
https://github.com/curl/curl/pull/2333

Comment 9 Nikos Mavrogiannopoulos 2018-07-20 14:26:36 UTC
Kamil isn't it possible to bring the pkcs11 patch in fedora for testing?

Comment 10 Kamil Dudka 2018-07-23 08:05:28 UTC
Not really.  This changes a publicly exposed interface of the library, so it needs to go upstream first.  I am fine with pushing it to Fedora before curl upstream releases it but the patch has not yet been merged upstream!

Would it help if I created a copr for testing it?

Comment 11 James Hubbard 2018-07-24 13:10:01 UTC
If you create a copr with the patches I'll try to test it this week. I won't have access to my boxes next week.

Comment 13 Kamil Dudka 2018-09-05 14:53:45 UTC
Please feel free to reopen if this is not a duplicate.

*** This bug has been marked as a duplicate of bug 1219544 ***

Comment 14 James Hubbard 2018-09-30 16:53:23 UTC
I know that this is closed, but I'll make a comment here to follow up because it's taken so long. 

David Woodhouse asked about passing using openonnect and curl.  I don't us openconnect.  I only have need to use curl. 

Using the form or the longer version of the url provided by pckcs11 tool gives the following answer.

curl -E 'pkcs11:manufacturer=piv_II;id=%01' $URL
curl: (58) SSL Engine 'pkcs11' not found

I also get the following when using the environment variables: 
fatal: unable to access 'https://repository.ncte.hpc.mil/git/DRMS_repo/': could not load PEM client certificate, OpenSSL error error:02001002:system library:fopen:No such file or directory, (no key found, wrong pass phrase, or wrong file format?)

I'll copy this over to the other bug report.