Bug 1475541

Summary: Certificate renewal scripts can crash and leave FreeIPA in a broken state
Product: [Fedora] Fedora Reporter: Adam Williamson <awilliam>
Component: freeipaAssignee: IPA Maintainers <ipa-maint>
Status: CLOSED EOL QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: urgent Docs Contact:
Priority: unspecified    
Version: 31CC: abokovoy, ipa-maint, jcholast, jhrozek, pvoborni, rcritten, ssorce, tkrizek
Target Milestone: ---Keywords: Reopened
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2020-11-24 20:06:45 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
journal extract none

Description Adam Williamson 2017-07-26 21:06:05 UTC 
Rob Crittenden kindly helped me diagnose and fix a serious problem on my FreeIPA server today. tomcat had suddenly started failing to authenticate when trying to access the LDAP server (389). Rob figured out (if I understand correctly) that it was down to a TLS certificate mismatch: the server certificate Tomcat was expecting 389 to be using was not the same as the server certificate it was *actually* using. Diagnosing this was not at all straightforward, and fixing it involved some fairly advanced (for a non-LDAP-specialist) ldapmodify stuff that again I probably couldn't have worked out for myself.

We think this was ultimately caused by a problem with the certmonger-based automatic certificate renewal stuff. It seems that an SELinux denial caused the renew_ca_cert and stop_pkicad scripts (both part of FreeIPA) not to be able to stop the pki-tomcatd service, and this failure caused renew_ca_cert to crash (several times, in fact). I think this resulted in the inconsistent state (the renewal process got as far as issuing the new cert and configuring 389 to use it, but didn't manage to configure tomcat to expect it).

Of course, we should fix the SELinux policy so the scripts *aren't* preventing from stopping the service, and I've filed a bug for that: https://bugzilla.redhat.com/show_bug.cgi?id=1475528 . But I also thought I should file this bug to see if the renewal process can be made safer - done more atomically, so if it fails things are correctly rolled back and FreeIPA left in a consistent state (and perhaps some kind of understandable alert generated and logged / sent to the admin / whatever). And certainly it shouldn't cause renew_ca_cert to just straight up *crash*, as it does.

I'll attaching the full extract of journal messages from the renewal.
Comment 1 Adam Williamson 2017-07-26 21:23:42 UTC 
Created attachment 1305048 [details]
journal extract
Comment 2 Petr Vobornik 2017-08-04 22:18:08 UTC 
Upstream ticket:
https://pagure.io/freeipa/issue/7091
Comment 3 Fedora End Of Life 2017-11-16 18:55:48 UTC 
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.
Comment 4 Fedora End Of Life 2017-12-12 10:19:44 UTC 
Fedora 25 changed to end-of-life (EOL) status on 2017-12-12. Fedora 25 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.
Comment 5 Ben Cotton 2019-08-13 16:48:05 UTC 
This bug appears to have been reported against 'rawhide' during the Fedora 31 development cycle.
Changing version to '31'.
Comment 6 Ben Cotton 2020-11-03 16:46:22 UTC 
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 7 Ben Cotton 2020-11-24 20:06:45 UTC 
Fedora 31 changed to end-of-life (EOL) status on 2020-11-24. Fedora 31 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.