Bug 699809 - Convert certificate system to use systemd
Convert certificate system to use systemd
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: pki-core (Show other bugs)
16
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Ade Lee
Fedora Extras Quality Assurance
:
Depends On:
Blocks: 699785
  Show dependency treegraph
 
Reported: 2011-04-26 12:28 EDT by Dmitri Pal
Modified: 2011-10-18 18:18 EDT (History)
4 users (show)

See Also:
Fixed In Version: pki-tks-9.0.7-1.fc15
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2011-09-30 14:39:35 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:


Attachments (Terms of Use)
patch to fix (82.18 KB, patch)
2011-09-09 13:32 EDT, Ade Lee
mharmsen: review+
Details | Diff
The "tomcat6-sysd" script used for testing on "goofy-vm10.dsdev.sjc.redhat.com" (2.63 KB, patch)
2011-09-13 01:17 EDT, Matthew Harmsen
no flags Details | Diff
The "tomcat6" script used for testing on "goofy-vm10.dsdev.sjc.redhat.com" (2.51 KB, patch)
2011-09-13 01:19 EDT, Matthew Harmsen
no flags Details | Diff
Code needed to migrate existing instances to systemd . . . (6.63 KB, patch)
2011-09-13 01:28 EDT, Matthew Harmsen
awnuk: review+
Details | Diff

  None (edit)
Description Dmitri Pal 2011-04-26 12:28:55 EDT
In F15 a new way to start services was introduced.
This is the bug to convert CS to use native systemd configuration files and scripts instead of init.d.
Comment 3 Ade Lee 2011-09-09 13:32:19 EDT
Created attachment 522372 [details]
patch to fix

This has most of the fix needed

There is some extra stuff in the spec file for pki-core for symkey -- I needed this just to get a build going.  I will remove this on commit.  The fix for this issue will be provided by mharmsen in a separate bug.

Whats missing:

Some logic in spec files to upgrade existing instance.  Will add that in a separate patch.
Comment 4 Matthew Harmsen 2011-09-09 15:23:26 EDT
Comment on attachment 522372 [details]
patch to fix

Reviewed in telephone conference with Ade Lee, Andrew Wnuk, and Adam Young.
Comment 5 Ade Lee 2011-09-09 16:43:32 EDT
checked into tip:

[vakwetu@goofy-vm10 pki]$ svn ci -m "Bugzilla BZ# 699809 - Convert certificate system to use systemd"
Sending        CMakeLists.txt
Sending        base/ca/CMakeLists.txt
Sending        base/ca/shared/conf/CS.cfg.in
Adding         base/ca/shared/lib
Adding         base/ca/shared/lib/systemd
Adding         base/ca/shared/lib/systemd/system
Adding         base/ca/shared/lib/systemd/system/pki-cad.target
Adding         base/ca/shared/lib/systemd/system/pki-cad@.service
Sending        base/common/CMakeLists.txt
Deleting       base/common/scripts
Sending        base/common/src/com/netscape/cms/servlet/csadmin/DonePanel.java
Sending        base/common/src/com/netscape/cms/servlet/csadmin/SecurityDomainPanel.java
Sending        base/kra/CMakeLists.txt
Sending        base/kra/shared/conf/CS.cfg.in
Adding         base/kra/shared/lib
Adding         base/kra/shared/lib/systemd
Adding         base/kra/shared/lib/systemd/system
Adding         base/kra/shared/lib/systemd/system/pki-krad.target
Adding         base/kra/shared/lib/systemd/system/pki-krad@.service
Sending        base/ocsp/CMakeLists.txt
Sending        base/ocsp/shared/conf/CS.cfg.in
Adding         base/ocsp/shared/lib
Adding         base/ocsp/shared/lib/systemd
Adding         base/ocsp/shared/lib/systemd/system
Adding         base/ocsp/shared/lib/systemd/system/pki-ocspd.target
Adding         base/ocsp/shared/lib/systemd/system/pki-ocspd@.service
Sending        base/setup/CMakeLists.txt
Sending        base/setup/pkicommon.pm
Sending        base/setup/pkicreate
Sending        base/setup/pkiremove
Adding         base/setup/scripts
Sending        base/setup/scripts/functions
Adding         base/setup/scripts/pkicontrol
Sending        base/tks/CMakeLists.txt
Sending        base/tks/shared/conf/CS.cfg.in
Adding         base/tks/shared/lib
Adding         base/tks/shared/lib/systemd
Adding         base/tks/shared/lib/systemd/system
Adding         base/tks/shared/lib/systemd/system/pki-tksd.target
Adding         base/tks/shared/lib/systemd/system/pki-tksd@.service
Sending        cmake/Modules/DefineInstallationPaths.cmake
Sending        scripts/compose_pki_core_packages
Sending        scripts/compose_pki_kra_packages
Sending        scripts/compose_pki_ocsp_packages
Sending        scripts/compose_pki_tks_packages
Sending        specs/pki-core.spec
Sending        specs/pki-kra.spec
Sending        specs/pki-ocsp.spec
Sending        specs/pki-tks.spec
Transmitting file data ...................................
Committed revision 2196.
Comment 6 Ade Lee 2011-09-09 16:44:32 EDT
Additional patch needed to migrate existing instances to systemd.
Comment 7 Matthew Harmsen 2011-09-13 01:13:17 EDT
On 09/09/2011, Ade Lee composed the following email:

1. knoxy is supposed to provide a F16 tomcat6 version to be tested.  We
need to test against this version.  We will need to change the spec
files (pki-core, pki-kra, pki-tks, pki-ocsp) to specifically require
this version or greater for f16+.

Up to now, he has just provided versions for f17.  The latest is at
http://koji.fedoraproject.org/koji/taskinfo?taskID=3340759

2. ipa will need to change the calls to "service pki-cad restart" etc.
to the new format as outlined in my email earlier today.

3. post install script code needs to be added to pki-core, pki-kra,
pki-tks, pki-ocsp to migrate existing instances to systemd. 

This should not be too hard - I just ran out of time.  The basic steps are:
loop through the instances in the directory  - /etc/sysconfig/pki/ca/*  (or /etc/sysconfig/pki/tks/* etc).
  --- for each instance, check if the instance has been updated
      if it has, then there will be a link under /etc/systemd/system/pki-cad.target.wants of the form
          pki-cad@<instance_name>.service -> /lib/systemd/system/pki-cad@.service
      if it has not been updated:
         -- create the above link
         -- also the link /var/lib/<instance_name>/<instance_name> points to the tomcat6 systemV file, 
            change it to point to /usr/sbin/tomcat6-sysd
         -- there is also a new entry in CS.cfg, but we do not need to update this as it is only used
            (for now) in the installation panels.  If you want to update it, then add the following to CS.cfg
            pkicreate.systemd.servicename=pki-cad@<instance_name>.service
Comment 8 Matthew Harmsen 2011-09-13 01:16:49 EDT
(In reply to comment #7)
> On 09/09/2011, Ade Lee composed the following email:
> 
> 1. knoxy is supposed to provide a F16 tomcat6 version to be tested.  We
> need to test against this version.  We will need to change the spec
> files (pki-core, pki-kra, pki-tks, pki-ocsp) to specifically require
> this version or greater for f16+.
> 
> Up to now, he has just provided versions for f17.  The latest is at
> http://koji.fedoraproject.org/koji/taskinfo?taskID=3340759
> 

Since no f16 versions of the tomcat6 files have appeared in Koji, this change cannot be instituted in time for Fedora 16 (beta).

However, as per 'Bugzilla Bug #734590 - Refactor JNI libraries for Fedora 16+ . . .', I am including the "tomcat6-sysd" and "tomcat6" scripts utilized for testing these systemd changes to Dogtag.
Comment 9 Matthew Harmsen 2011-09-13 01:17:34 EDT
Created attachment 522854 [details]
The "tomcat6-sysd" script used for testing on "goofy-vm10.dsdev.sjc.redhat.com"
Comment 10 Matthew Harmsen 2011-09-13 01:19:36 EDT
Created attachment 522855 [details]
The "tomcat6" script used for testing on "goofy-vm10.dsdev.sjc.redhat.com"
Comment 11 Matthew Harmsen 2011-09-13 01:22:09 EDT
(In reply to comment #7)
> On 09/09/2011, Ade Lee composed the following email:
> 
> 3. post install script code needs to be added to pki-core, pki-kra,
> pki-tks, pki-ocsp to migrate existing instances to systemd. 
> 
> This should not be too hard - I just ran out of time.  The basic steps are:
> loop through the instances in the directory  - /etc/sysconfig/pki/ca/*  (or
> /etc/sysconfig/pki/tks/* etc).
>   --- for each instance, check if the instance has been updated
>       if it has, then there will be a link under
> /etc/systemd/system/pki-cad.target.wants of the form
>           pki-cad@<instance_name>.service ->
> /lib/systemd/system/pki-cad@.service
>       if it has not been updated:
>          -- create the above link
>          -- also the link /var/lib/<instance_name>/<instance_name> points to
> the tomcat6 systemV file, 
>             change it to point to /usr/sbin/tomcat6-sysd
>          -- there is also a new entry in CS.cfg, but we do not need to update
> this as it is only used
>             (for now) in the installation panels.  If you want to update it,
> then add the following to CS.cfg
>             pkicreate.systemd.servicename=pki-cad@<instance_name>.service

Rudimentary testing (with some slight modifications) revealed some "hanging" issues in these post-installation scripts, but they have been included (and commented out) for pki-core, pki-kra, pki-ocsp, and pki-tks.
Comment 12 Matthew Harmsen 2011-09-13 01:24:49 EDT
(In reply to comment #7)
> On 09/09/2011, Ade Lee composed the following email:
>  
> 3. post install script code needs to be added to pki-core, pki-kra,
> pki-tks, pki-ocsp to migrate existing instances to systemd. 
> 
> This should not be too hard - I just ran out of time.  The basic steps are:
> loop through the instances in the directory  - /etc/sysconfig/pki/ca/*  (or
> /etc/sysconfig/pki/tks/* etc).
>   --- for each instance, check if the instance has been updated
>       if it has, then there will be a link under
> /etc/systemd/system/pki-cad.target.wants of the form
>           pki-cad@<instance_name>.service ->
> /lib/systemd/system/pki-cad@.service
>       if it has not been updated:
>          -- create the above link
>          -- also the link /var/lib/<instance_name>/<instance_name> points to
> the tomcat6 systemV file, 
>             change it to point to /usr/sbin/tomcat6-sysd
>          -- there is also a new entry in CS.cfg, but we do not need to update
> this as it is only used
>             (for now) in the installation panels.  If you want to update it,
> then add the following to CS.cfg
>             pkicreate.systemd.servicename=pki-cad@<instance_name>.service

On 09/12/2011, Ade Lee composed the following email:

I had a couple of minutes and could not resist

The scriptlet needed looks something like the following.  This is totally untested, needs to be rpm macro-ized,  and is just to give you a general idea.

This is for a ca.  Change as needed for kra, tks, ocsp.

post -n pki-ca
for name in `ls /etc/sysconfig/pki/ca`; do
    if [ ! -e "/etc/systemd/system/pki-cad.target.wants/pki-cad@$name.service" ]; then
        ln -s "/lib/systemd/system/pki-cad@.service"   "/etc/systemd/system/pki-cad.target.wants/pki-cad@$name.service"
        [ -e /var/lib/$name/$name ] && unlink /var/lib/$name/$name
        ln -s /usr/sbin/tomcat6-sysd /var/lib/$name/$name
        echo "pkicreate.systemd.service=pkicad@$name.service" >> /var/lib/$name/conf/CS.cfg
    fi
done

Again - totally untested - probably does not even run -- use at your own risk.  Check the variable name in the CS.cfg addition above. You may - or may not - also need to restart the instance.


As stated in the previous comment, rudimentary testing (with some slight modifications) revealed some "hanging" issues in these post-installation scripts, but they have been included (and commented out) for pki-core, pki-kra, pki-ocsp, and pki-tks.
Comment 13 Matthew Harmsen 2011-09-13 01:28:44 EDT
Created attachment 522856 [details]
Code needed to migrate existing instances to systemd . . .

See previous comments regarding this patch . . .
Comment 14 Matthew Harmsen 2011-09-13 01:49:03 EDT
TIP:

# cd pki

# svn status | grep -v ^$ | grep -v ^P | grep -v ^X | grep -v ^?
M       specs/pki-core.spec
M       specs/pki-kra.spec
M       specs/pki-tks.spec
M       specs/pki-ocsp.spec

# svn commit
Sending        specs/pki-core.spec
Sending        specs/pki-kra.spec
Sending        specs/pki-ocsp.spec
Sending        specs/pki-tks.spec
Transmitting file data ....
Committed revision 2198.
Comment 15 Fedora Update System 2011-09-13 05:29:31 EDT
pki-kra-9.0.7-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-kra-9.0.7-1.fc16
Comment 16 Fedora Update System 2011-09-13 05:31:24 EDT
pki-ocsp-9.0.6-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-ocsp-9.0.6-1.fc16
Comment 17 Fedora Update System 2011-09-13 05:58:59 EDT
pki-tks-9.0.6-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-tks-9.0.6-1.fc16
Comment 18 Fedora Update System 2011-09-13 14:24:16 EDT
pki-core-9.0.14-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-core-9.0.14-1.fc16
Comment 19 Fedora Update System 2011-09-13 18:20:21 EDT
Package pki-kra-9.0.7-1.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing pki-kra-9.0.7-1.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/pki-kra-9.0.7-1.fc16
then log in and leave karma (feedback).
Comment 20 Ade Lee 2011-09-14 08:56:30 EDT
Matt,

There is a known systemd bug about hanging restarts when one uses systemctl restart foo.target.  Check with richm for the bug number.

You can avoid this by using systemctl restart pki-cad@foo.service instead.  You can do this because you iterate through each of the instances in any case.

But the other thing to keep in mind is that we are doing something weird here - we are stopping instances started by systemV start scripts with systemd scripts - who knows if all the relevant pids etc. are cleaned up correctly.

The right way to do this is probably something like this:

%pre
service pki-cad stop

%post 
... all the conversion stuff for each instance foo ..
systemctl start pki-cad@foo.service

Actually, to really be right, one should probably keep track of which instances were actually running in %pre and only restart those instances.

Ade
Comment 21 Ade Lee 2011-09-26 12:47:26 EDT
Here is the corrected post-install script for the ca.  Matt is planning on including these with his own spec changes.  Will review the whole thing together.

for inst in `ls /etc/sysconfig/pki/ca`; do
    if [ ! -e "/etc/systemd/system/pki-cad.target.wants/pki-cad@${inst}.service" ]; then
        ln -s "/lib/systemd/system/pki-cad@.service" \
            "/etc/systemd/system/pki-cad.target.wants/pki-cad@${inst}.service"
        [ -L /var/lib/${inst}/${inst} ] && unlink /var/lib/${inst}/${inst}
        ln -s /usr/sbin/tomcat6-sysd /var/lib/${inst}/${inst}
  
        if [ -e /var/run/${inst}.pid ]; then
            kill -9 `cat /var/run/${inst}.pid` || :
            rm -f /var/run/${inst}.pid
            echo "pkicreate.systemd.servicename=pki-cad@${inst}.service" >> \
                /var/lib/${inst}/conf/CS.cfg || :
            /bin/systemctl daemon-reload >/dev/null 2>&1 || :
            /bin/systemctl restart pki-cad@${inst}.service || :
        else 
            echo "pkicreate.systemd.servicename=pki-cad@${inst}.service" >> \
                /var/lib/${inst}/conf/CS.cfg || :
        fi
    fi
done
/bin/systemctl daemon-reload >/dev/null 2>&1 || :
Comment 22 Fedora Update System 2011-09-30 14:39:23 EDT
pki-core-9.0.14-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 23 Fedora Update System 2011-09-30 14:43:51 EDT
pki-kra-9.0.7-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 24 Fedora Update System 2011-09-30 15:13:42 EDT
pki-tks-9.0.6-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 25 Fedora Update System 2011-09-30 15:18:26 EDT
pki-ocsp-9.0.6-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 26 Fedora Update System 2011-10-06 20:02:58 EDT
tomcatjss-6.0.2-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/tomcatjss-6.0.2-1.fc15
Comment 27 Fedora Update System 2011-10-06 22:41:45 EDT
pki-core-9.0.15-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pki-core-9.0.15-1.fc15
Comment 28 Fedora Update System 2011-10-06 22:42:49 EDT
pki-console-9.0.5-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pki-console-9.0.5-1.fc15
Comment 29 Fedora Update System 2011-10-06 22:45:03 EDT
pki-kra-9.0.8-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pki-kra-9.0.8-1.fc15
Comment 30 Fedora Update System 2011-10-06 22:46:55 EDT
pki-ocsp-9.0.7-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pki-ocsp-9.0.7-1.fc15
Comment 31 Fedora Update System 2011-10-06 22:53:39 EDT
pki-tks-9.0.7-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pki-tks-9.0.7-1.fc15
Comment 32 Fedora Update System 2011-10-06 23:49:07 EDT
dogtag-pki-9.0.0-7.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/dogtag-pki-9.0.0-7.fc15
Comment 33 Fedora Update System 2011-10-08 01:37:03 EDT
tomcatjss-6.0.2-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/tomcatjss-6.0.2-1.fc16
Comment 34 Fedora Update System 2011-10-08 01:40:44 EDT
pki-core-9.0.15-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-core-9.0.15-1.fc16
Comment 35 Fedora Update System 2011-10-08 01:41:24 EDT
pki-console-9.0.5-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-console-9.0.5-1.fc16
Comment 36 Fedora Update System 2011-10-08 01:47:21 EDT
pki-kra-9.0.8-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-kra-9.0.8-1.fc16
Comment 37 Fedora Update System 2011-10-08 01:48:26 EDT
pki-ocsp-9.0.7-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-ocsp-9.0.7-1.fc16
Comment 38 Fedora Update System 2011-10-08 02:11:21 EDT
pki-tks-9.0.7-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-tks-9.0.7-1.fc16
Comment 39 Fedora Update System 2011-10-08 02:56:50 EDT
dogtag-pki-9.0.0-7.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/dogtag-pki-9.0.0-7.fc16
Comment 40 Fedora Update System 2011-10-16 20:43:44 EDT
dogtag-pki-9.0.0-7.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 41 Fedora Update System 2011-10-16 20:44:24 EDT
pki-console-9.0.5-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 42 Fedora Update System 2011-10-16 20:45:03 EDT
pki-tks-9.0.7-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 43 Fedora Update System 2011-10-16 20:46:34 EDT
pki-core-9.0.15-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 44 Fedora Update System 2011-10-16 20:46:48 EDT
pki-ocsp-9.0.7-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 45 Fedora Update System 2011-10-16 20:47:39 EDT
tomcatjss-6.0.2-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 46 Fedora Update System 2011-10-16 20:48:00 EDT
pki-kra-9.0.8-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 47 Fedora Update System 2011-10-18 18:07:31 EDT
dogtag-pki-9.0.0-7.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 48 Fedora Update System 2011-10-18 18:08:32 EDT
pki-ocsp-9.0.7-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 49 Fedora Update System 2011-10-18 18:10:24 EDT
pki-core-9.0.15-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 50 Fedora Update System 2011-10-18 18:16:17 EDT
tomcatjss-6.0.2-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 51 Fedora Update System 2011-10-18 18:16:57 EDT
pki-console-9.0.5-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.
Comment 52 Fedora Update System 2011-10-18 18:18:19 EDT
pki-tks-9.0.7-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

Note You need to log in before you can comment on or make changes to this bug.