Could you please attach ipaserver-install.log from /var/log? You may want to replace actual subject/dc/hostname/IP address values to avoid mentioning your own private names.

Sorry to say, but the log file is gone. I have wiped out the container to continue. Fortunately the bad code is obvious:

	if subject == ca_subject:
		ca_nickname = nickname

Please check RFC5280 about how to compare distinguished names.

The code, correctly uses DN class to perform the comparison: https://git.fedorahosted.org/cgit/freeipa.git/tree/ipaserver/install/installutils.py?h=ipa-4-2#n1005

Issue here is different, RFC 5280 says: "and the matching RDNs appear in the same order in both DNs" which is not the case here.

I worked around this using 

	ipa-server-install --external-ca --subject="O=example AG,C=DE"

but I would assume this worked just by chance. Every fix or stable workaround is highly appreciated.

The code in certutil (actually, in nss library) claims that it does reverse RDNs to comply with RFC1485 which is the only one supported by certutil for subjects. We are checking with NSS maintainers to see whether this is indeed a correct behavior.

(In reply to Petr Vobornik from comment #3)
> The code, correctly uses DN class to perform the comparison:
> https://git.fedorahosted.org/cgit/freeipa.git/tree/ipaserver/install/
> installutils.py?h=ipa-4-2#n1005
> 
> Issue here is different, RFC 5280 says: "and the matching RDNs appear in the
> same order in both DNs" which is not the case here.

The problem here is that comparison should actually be done on DER encoding of the distinguished name, not the string. Also, ipapython.DN class handles LDAP distinguished names, not x.509 ones. The difference is that for LDAP the order of RDNs is important while for x.509 it is not: see RFC4514 2. and 5.2.

If I understand correctly, what happens is:

- IPA code takes a partial-DN input
- IPA adds CN to partial-DN and creates DN
- IPA requests NSS to produce a certificate with DN
- the certificate produced contains DN-reversed
- later, IPA looks at the code and produces DN-of-certificate

We have learned that different tools use different strategies for creating the text representation of the DN.

OpenSSL tools seem to always create the string in the order as encoded, while NSS tools always reverses the order prior to displaying it.

I suspect the problem is caused by the step that reads the certificate for comparison purposes, and attempts to build the string representation of the DN. At this step, you might currently use non-NSS code, which makes different assumptions.

I guess, if IPA used NSS for both creating the CSR, and for later creating the string representation of the certificates's DN, then things would match.

(In reply to Kai Engert (:kaie) from comment #7)
> 
> I suspect the problem is caused by the step that reads the certificate for
> comparison purposes, and attempts to build the string representation of the
> DN. At this step, you might currently use non-NSS code, which makes
> different assumptions.
> 
> I guess, if IPA used NSS for both creating the CSR, and for later creating
> the string representation of the certificates's DN, then things would match.

I've learned that you already use NSS code, but apparently one that directly operates on the storage format.

I think you'd have to use NSS code that converts the name to the string presentation. That NSS code should know that it needs to reverse the order when creating the string.

I think you'd have to use the CERT_NameToAscii() function.

python-nss Certificate.subject does call CERT_NameToAscii()

To the best of my understanding both the ipapython.DN implementation and the result of Certificate.subject produce a RFC 4514 representation with the RDN's in the same order.

NSS certutil calls CERT_AsciiToName on the string passed as the subject. So this would suggest everything is consistent.

Re comment #6

I disagree with the assertion that RDN ordering is insignificant in x509.

So, let me do a complete description of what you're doing, plus experiments.

User input "partial DN":
  "C=DE,O=example AG"

IPA prepends to create the "complete DN":
  "CN=Certificate Authority,C=DE,O=example AG"

IPA runs certutil to create a CSR:
    goto temp dir
    certutil -d . -N --empty-password
    certutil -d . -R -s "CN=Certificate Authority,C=DE,O=example AG" -a -o example.csr -z /proc/uptime

External tools are used to transform the CSR into a certificate.
We assume the representation inside both CSR and certificate is identical
(whichever ordering it is, it must be the same).

In order to have a complete example, I'm using certutil again, to transform
the CSR into a certificate.

    certutil -d . -C -x -m 0 -Z SHA256 -a -i example.csr > example.crt

Then IPA loads the certificate and uses python-nss to transform the
cert's DN to a "cert DN string".
IPA expects the "cert DN string" to be identical to the original "complete DN".

John sent me a helper script that uses python-nss.

I tweaked it to obtain "cert.subject" and print it, which is what IPA uses:

$ python cert-print.py -f pem example.crt 
certificate filename=example.crt
Subject: CN=Certificate Authority,C=DE,O=example AG

This matches our expectation.

I don't understand why IPA gets a different ordering.

For further analysis, I'd like to ask that you please provide a full set of:
- exact certutil command that you use with all parameters
- copies of the csr and the certificate files you're using

Thanks

The relevant portions in John script are:

cert_der = nss.read_der_from_file(filename, options.cert_format == 'pem')

cert = nss.Certificate(cert_der)

print("Subject: %s" % cert.subject)

FreeIPA actually calls out for certutil in its wrapper around nssdb:
   def get_cert(self, nickname, pem=False):
        args = ['-L', '-n', nickname]
        if pem:
            args.append('-a')
        else:
            args.append('-r')
        try:
            cert, err, returncode = self.run_certutil(args)
        except ipautil.CalledProcessError:
            raise RuntimeError("Failed to get %s" % nickname)
        return cert

for the code in question we get 'certutil -d path -L -n nickname -a' and use output as a PEM cert input into nss.Certificate(). The reason it is done this way is because, I believe, we have already other NSS database open, so opening a second one would crash nss code. I may be wrong but we had those issues before.

(In reply to Alexander Bokovoy from comment #13)
> ... we get 'certutil -d path -L -n nickname -a' and use
> output as a PEM cert input into nss.Certificate().

This should be fine. As long as you feed the raw certificate into python-nss, and use the same python-nss code to extract the DN, you should get the same results as I.

So, we still need more details (and files) to try to understand what's going on on your side.

FYI: my understanding is that OpenSSL and NSS handle the ordering of RDN's in opposite order, hopefully this is not causing some of the confusion.

It's a well documented fact that OpenSSL uses the reverse order from RFC 4514 (the display format for a directory name). There are several bugs reports open on this behavior, if need be I can probably find them.

Why does OpenSSL do this? Because apparently the DER encoding of a DN requires the RDN to be in opposite order from RFC 4514. One assumes the OpenSSL implementation is just a thin layer over the binary encoding (e.g. the protocol).

On the other hand NSS and every other library I've experimented with using the RFC 4514 ordering, which IMHO is correct.

FWIW OpenSSL also uses a completely non-standard string representation for subjects using backslashes. At one point I had to write a utility for Openstack to translated between RFC 4514 and OpenSSL formats.

The reason I raise this is because if at any point OpenSSL is involved you could be subject to the reverse ordering problem.

I'll double check the behavior of libpython.DN and python-nss subject handling.

After some debugging I was able to find the problem. It's not related to order of RDNs in DNs at all.

How to reproduce:

1) ipa-server-install --external-ca --subject="C=DE,O=example AG" --realm=CA.TEST

This creates a CSR:
/usr/lib64/nss/unsupported-tools/pp -t cr -a -i /root/ipa.csr | grep Subject:
        Subject: "CN=Certificate Authority,C=DE,O=example AG"

2) I signed /root/ipa.csr with a custom CA.

/usr/lib64/nss/unsupported-tools/pp -t c -a -i /root/ipa.crt | grep Subject:
        Subject: "CN=Certificate Authority,C=DE,O=example AG"

3) /sbin/ipa-server-install --external-cert-file=ipa.crt --external-cert-file=ca.crt

For step 3 I added pdb.set_trace() to load_external_cert():

> /usr/lib/python2.7/site-packages/ipaserver/install/installutils.py(1000)load_external_cert()
-> ca_subject = DN(('CN', 'Certificate Authority'), subject_base)
(Pdb) n
> /usr/lib/python2.7/site-packages/ipaserver/install/installutils.py(1001)load_external_cert()
-> ca_nickname = None
(Pdb) p ca_subject
ipapython.dn.DN('CN=Certificate Authority,O=CA.TEST')

Now that's interesting. The ca_subject looks completely different. Where does O=CA.TEST come from? It's the Kerberos realm!

https://git.fedorahosted.org/cgit/freeipa.git/tree/ipaserver/install/ca.py#n70
https://git.fedorahosted.org/cgit/freeipa.git/tree/ipaserver/install/server/install.py#n580

    if not options.subject:
        options.subject = DN(('O', realm_name))

The installer doesn't remember the custom subject and neglects to tell you that you must use the --subject argument in the second call, too. It's really a documentation and UI bug.

I was able to complete the second installation step with:

/sbin/ipa-server-install --external-cert-file=ipa.crt --external-cert-file=ca.crt --subject="C=DE,O=example AG"

Christian, when talking about the original problem report, can you please explain what must be done differently to avoid the problem? Thanks.

Kai, sure!

In order to avoid the problem it is necessary to call ipa-server-install with the --subject="C=DE,O=example AG" *both* times. The problem occurs when --subject is missing in the second step.

I'm sorry for the inconvenience. The installer doesn't explain the fact. I'll make sure that future versions of FreeIPA are more helpful.

seems to me that the root cause is the same as in bug 1292099 a regression in installer - installer doesn't use the options from first step of installation.

There is a patch on the devel-list: https://www.redhat.com/archives/freeipa-devel/2015-December/msg00523.html

see comment 19, was fixed upstream. Will be part of 4.2.4 update.

*** This bug has been marked as a duplicate of bug 1292099 ***