Bug 2350322

Summary:	Missing rewrite.config in upgraded server
Product:	[Fedora] Fedora	Reporter:	Tomasz Torcz <tomek>
Component:	dogtag-pki	Assignee:	Endi Sukma Dewata <edewata>
Status:	POST ---	QA Contact:	Fedora Extras Quality Assurance <extras-qa>
Severity:	high	Docs Contact:
Priority:	unspecified
Version:	41	CC:	abokovoy, alee, cfu, edewata, ftrivino, ipa-maint, jmagne, mfargett, mhjacks, r3pek, rcritten, ssorce, taherrin, twoerner
Target Milestone:	---
Target Release:	---
Hardware:	Unspecified
OS:	Linux
Whiteboard:
Fixed In Version:		Doc Type:	---
Doc Text:		Story Points:	---
Clone Of:		Environment:
Last Closed:		Type:	---
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Tomasz Torcz 2025-03-06 13:15:59 UTC

Some time ago ACME on my FreeIPA installation stopped working. Certificate issuance is not working (ACME agent reports 404) and `ipa-acme-manage` fails to authenticate:

$ ipa-acme-manage status
Failed to authenticate to CA REST API
The ipa-acme-manage command failed.

With more debug:

$ ipa-acme-manage -d status
ipapython.ipaldap: DEBUG: retrieving schema for SchemaCache url=ldapi://%2Frun%2Fslapd-PIPEBREAKER-PL.socket conn=<ldap.ldapobject.SimpleLDAPObject object at 0x7f4bd2cc2660>
ipaserver.masters: DEBUG: Discovery: available servers for service 'CA' are kaitain.pipebreaker.pl
ipaserver.masters: DEBUG: Discovery: using kaitain.pipebreaker.pl for 'CA' service
ipapython.dogtag: DEBUG: request POST https://kaitain.pipebreaker.pl:8443/acme/login
ipapython.dogtag: DEBUG: request body ''
ipapython.dogtag: DEBUG: response status 404

At this time, /var/log/pki/pki-tomcat/localhost_access_log registers:

[06/Mar/2025:14:06:40 +0100] "POST /acme/login HTTP/1.1" 404 765

Further verification with curl:

$ curl https://kaitain.pipebreaker.pl:8443/acme/login
<!doctype html><html lang="en"><head><title>HTTP Status 404 – Not Found</title>

I've noticed there's a rewrite.config file at /usr/share/pki/server/conf/Catalina/localhost/rewrite.config, defining a number of mapping of /acme/* to /acme/rest/*.
Indeed, when I try to access the second kind of URL with curl, it somewhat works:

$ curl https://kaitain.pipebreaker.pl:8443/acme/rest/login
$ # no output -> no error

Access log:
 - [06/Mar/2025:14:10:21 +0100] "GET /acme/rest/login HTTP/1.1" 204 -

My suspicion is that rules defined in rewrite.config do not work.
How to proceed with debugging now?

Reproducible: Always




$ rpm -qf /usr/share/pki/server/conf/Catalina/localhost/rewrite.config
dogtag-pki-server-11.5.0-3.fc41.2.noarch
freeipa-server-4.12.2-8.fc41.x86_64

Comment 1 Tomasz Torcz 2025-03-06 13:25:56 UTC

The file seem to come from upstream https://github.com/dogtagpki/pki/commit/994d932100c7d335752fe817a7d8757f62439b08

Comment 2 Alexander Bokovoy 2025-03-06 13:28:08 UTC

Move to dogtag-pki, as it is purely within the dogtag.

Comment 3 Tomasz Torcz 2025-04-05 10:50:25 UTC

After some code reading, I got it working (I think? `ipa-acme-status` works, I'll see if any of my certificates will be issued).

What I did:
1. Symlinked
   /usr/share/pki/server/conf/Catalina/localhost/rewrite.config
   to
   /etc/pki/pki-tomcat/Catalina/localhost/rewrite.config

2. Edited /etc/pki/pki-tomcat/server.xml and added in line 133 (almost at the end):

        <Valve className="org.apache.catalina.valves.rewrite.RewriteValve"/>


Were the steps correct? Should it be done during FreeIPA upgrade?

Comment 4 Tomasz Torcz 2025-04-05 19:39:50 UTC

Certificates got reissured to acme clients.

Comment 5 Rob Crittenden 2025-04-07 15:02:40 UTC

This configuration file was added to dogtag in v11.3 to provide support for the EST protocol in f95df455c5f062ef024b91f5bfc95d919c91cfb7 . The commit message explicitly says that this configuration is not added on upgrade.

The acme rewrite rules were add in v11.5 in 994d932100c

So any version of PKI installed prior to v11.3 will not have rewrite.config enabled which is the root cause of the missing rules.

IMHO this upgrade to add the missing file and config needs to be managed by PKI as it will affect all older PKI deployments whether they use IPA or not.

Comment 6 Endi Sukma Dewata 2025-04-24 18:50:59 UTC

Thanks for the investigation. I suppose the issue can be reproduced in plain PKI instance with these steps:

1. Install PKI 11.2 or older
2. Create CA instance
3. Upgrade to the latest PKI version
4. Deploy ACME or EST in the same instance
5. Access ACME or EST services

Expected result: ACME and EST services should work.

Comment 7 Endi Sukma Dewata 2025-05-27 23:46:35 UTC

https://github.com/dogtagpki/pki/pull/5101

Comment 8 Endi Sukma Dewata 2025-05-28 22:43:40 UTC

Fixed upstream:

* v11.6 branch: https://github.com/dogtagpki/pki/commit/a0671f2c9e7385447c80f58d49ca06646811b975
* master branch (PKI 11.7): https://github.com/dogtagpki/pki/commit/17a44731c7d6aa2888ae8e700149712351fe081c