You need to wait until https://koji.fedoraproject.org/koji/buildinfo?buildID=1845811 is in Rawhide compose and freeipa-container is rebuilt.
I'll mark this as a duplicate of bug #2014658

*** This bug has been marked as a duplicate of bug 2014658 ***

Hello Alexander,

as I wrote in the description, I already did that:

> at first sight this *could* have been bug 2014658. But I updated to the latest container (version -2) with the fix, and same issue.

. -2 is in Rawhide: https://bodhi.fedoraproject.org/updates/FEDORA-2021-7dac22d63a , and quay.io has an up to date container. This is *not* a duplicate.

Do you have httpd's error_log and PKI logs?

I cannot find them in your PR's logs. Without server's logs it is hard to see what else is broken in recent PKI's removal of XML-RPC support.

Created attachment 1834215 [details]
/var/log

Indeed error_log has something that looks very relevant:

ipa: ERROR: non-public: ValueError: error parsing asn1 value: ParseError { kind: EncodedDefault, location: ["0", "Extension::critical"] }
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/ipaserver/rpcserver.py", line 405, in wsgi_execute
    result = command(*args, **options)
  File "/usr/lib/python3.10/site-packages/ipalib/frontend.py", line 471, in __call__
    return self.__do_call(*args, **options)
  File "/usr/lib/python3.10/site-packages/ipalib/frontend.py", line 499, in __do_call
    ret = self.run(*args, **options)
  File "/usr/lib/python3.10/site-packages/ipalib/frontend.py", line 821, in run
    return self.execute(*args, **options)
  File "/usr/lib/python3.10/site-packages/ipaserver/plugins/cert.py", line 719, in execute
    ext_san = csr.extensions.get_extension_for_oid(
ValueError: error parsing asn1 value: ParseError { kind: EncodedDefault, location: ["0", "Extension::critical"] }
ipa: INFO: [jsonserver_kerb] host/x0.cockpit.lan: cert_request('MIIDTzCCAjcCAQAwGTEXMBUGA1UEAxMOeDAuY29ja3BpdC5sYW4wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC9L2/NsS/B6qMywcUeqpgd9DtGRT7WjCRLGZSOt50+pfDYjP9NLLZAlHGFEqivlYrXhfP/rrOYmznZBwmAin7+f2SLIzcpKs1HEc8UoPtr9Tuf6gjYdFofGo94fjGjeZsax9pMFKAPc6Icm1YOqgzX1EZQ0OR2okvVV9p4lW14hy1xMZcDBmMvLovH+nABSeDwe4ucpdG/cE+yfwKEvirW7zx8Glc2hrzPaqBNBgIUcfZkfD8Nn+QAzx3JelI1u6D2qOaYDk9KQm0Z8MN5nXDYa4AbpjTnGRB2IOykFFSMA4OSwKxl3duH0P0vJ/ab8VjIAk3lRGvp3Baq7lgEgTnjAgMBAAGggfAwKwYJKoZIhvcNAQkUMR4eHAAyADAAMgAxADEAMAAxADgAMQAyADMANwAzADcwgcAGCSqGSIb3DQEJDjGBsjCBrzB9BgNVHREBAQAEczBxoC8GCisGAQQBgjcUAgOgIQwfSFRUUC94MC5jb2NrcGl0LmxhbkBDT0NLUElULkxBTqA+BgYrBgEFAgKgNDAyoA0bC0NPQ0tQSVQuTEFOoSEwH6ADAgEBoRgwFhsESFRUUBsOeDAuY29ja3BpdC5sYW4wDAYDVR0TAQH/BAIwADAgBgNVHQ4BAQAEFgQUglxkOudA1OQlVy0HBnAk23DoN9IwDQYJKoZIhvcNAQELBQADggEBAAyvaMeDs0MZ4b7TUIDr9HPVeulIQs6vs7lUVvZnnoCui2jEt8ydZiCIh/98LdlB5D6iLkQOlaM7AHnuSXwyKjNQMRz5nRMVCwg48atXeT/eDBYuONYn0+oBd3q7vDrsKEl+M3UabGaiPE1hqf1kkIj5k7s7t9t1qQl5hxGYQKU3z1h1s3mP+4xBt4M8hneXibP4kuB/zDVbXiZB5ViSTmHENfVPLNZ5w18BZv6BoQzpkOM5r+E8YiVDKfNajR7q02P24urI8eFbWgAvE+gmQvUS3nUHOqO6OCG09JZ+DEvdXh8tnKSow/CF61RpgyDaazOFDoV16D8huDeyndnV+R4=', principal='HTTP/x0.cockpit.lan', add=True): InternalError

I'll attach the complete /var/log from the container, in case it has something else that is interesting.

Ah, this is python-cryptography 35.0 issue: https://github.com/pyca/cryptography/issues/6340

We saw that in Rawhide just this weekend in IPA CI runs: http://freeipa-org-pr-ci.s3-website.eu-central-1.amazonaws.com/jobs/1ae3e78e-2eab-11ec-8838-fa163ef390f3/report.html

----
A few tests failed because of python cryptography issue #6340: CSR with 'BEGIN NEW CERTIFICATE REQUEST' no longer accepted (external_ca_TestExternalCAInvalidCert, external_ca_TestMultipleExternalCA, external_ca_TestExternalCAdirsrvStop, external_ca_templates, test_acme, test_caless_TestServerCALessToExternalCA, test_external_ca_TestExternalCA, test_ipahealthcheck_nodns_extca_file). There is already an upstream fix but no rawhide build yet.
----

python-cryptography update is not yet in Rawhide.

I will move this to python-cryptography for awareness.

What version of certmonger is this?

This is actually a certmonger bug that was revealed by a new ASN.1 parser in python-cryptography. The new parser does not accept invalid ASN.1 DER. The problem should be fixed by certmonger-0.79.14-6.fc36. The update https://bodhi.fedoraproject.org/updates/FEDORA-2021-1881de28ed is in Rawhide.

Upstream python-cryptography has agreed to accept a temporary workaround for backwards compatibility. The workaround is still work in progress.

Rob: certmonger-0.79.14-6.fc36.x86_64 on the server side (in the container), certmonger-0.79.14-2.fc35.x86_64 on the client side; I'm not sure which one is relevant.

The relevant side is side that generates the CSR. In your case it's the client. 

https://bodhi.fedoraproject.org/updates/FEDORA-2021-08cd0d66af is not available in F35 yet. Could you please verify the update?

On the client side I ran

    dnf update --enablerepo=updates-testing certmonger

which pulled in certmonger-0.79.14-3.fc35 from that bodhi update. I confirm that this works.

There is no corresponding fix for Fedora 34, so I can't test it there. Is this still planned? As that seems to break "over the wire" compatibility, all supported Fedora/RHEL versions ought to get that fix?

Thanks for tracking this down!

I'm working on builds for 33 and 34.

FEDORA-2021-bf70829200 has been submitted as an update to Fedora 33. https://bodhi.fedoraproject.org/updates/FEDORA-2021-bf70829200

FEDORA-2021-df9eee3325 has been pushed to the Fedora 34 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-df9eee3325`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-df9eee3325

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

FEDORA-2021-bf70829200 has been pushed to the Fedora 33 testing repository.
Soon you'll be able to install the update with the following command:
`sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2021-bf70829200`
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2021-bf70829200

See also https://fedoraproject.org/wiki/QA:Updates_Testing for more information on how to test updates.

I tested the F34 update [1] and this works again. (Still a bit sad that e.g. stable Debian or CentOS 7/8 clients will be broken with 4.9.7)

Thanks!

[1] https://bodhi.fedoraproject.org/updates/FEDORA-2021-df9eee3325

I believe that python-cryptography is going to add an option to relax this validation which will help with unpatched certmonger releases.

FEDORA-2021-df9eee3325 has been pushed to the Fedora 34 stable repository.
If problem still persists, please make note of it in this bug report.

FEDORA-2021-bf70829200 has been pushed to the Fedora 33 stable repository.
If problem still persists, please make note of it in this bug report.