Bug 699809

Useful links:

    * http://fedoraproject.org/wiki/Features/systemd
    * https://fedoraproject.org/wiki/Packaging:Guidelines:Systemd

Created attachment 522372 [details]
patch to fix

This has most of the fix needed

There is some extra stuff in the spec file for pki-core for symkey -- I needed this just to get a build going.  I will remove this on commit.  The fix for this issue will be provided by mharmsen in a separate bug.

Whats missing:

Some logic in spec files to upgrade existing instance.  Will add that in a separate patch.

Comment on attachment 522372 [details]
patch to fix

Reviewed in telephone conference with Ade Lee, Andrew Wnuk, and Adam Young.

checked into tip:

[vakwetu@goofy-vm10 pki]$ svn ci -m "Bugzilla BZ# 699809 - Convert certificate system to use systemd"
Sending        CMakeLists.txt
Sending        base/ca/CMakeLists.txt
Sending        base/ca/shared/conf/CS.cfg.in
Adding         base/ca/shared/lib
Adding         base/ca/shared/lib/systemd
Adding         base/ca/shared/lib/systemd/system
Adding         base/ca/shared/lib/systemd/system/pki-cad.target
Adding         base/ca/shared/lib/systemd/system/pki-cad@.service
Sending        base/common/CMakeLists.txt
Deleting       base/common/scripts
Sending        base/common/src/com/netscape/cms/servlet/csadmin/DonePanel.java
Sending        base/common/src/com/netscape/cms/servlet/csadmin/SecurityDomainPanel.java
Sending        base/kra/CMakeLists.txt
Sending        base/kra/shared/conf/CS.cfg.in
Adding         base/kra/shared/lib
Adding         base/kra/shared/lib/systemd
Adding         base/kra/shared/lib/systemd/system
Adding         base/kra/shared/lib/systemd/system/pki-krad.target
Adding         base/kra/shared/lib/systemd/system/pki-krad@.service
Sending        base/ocsp/CMakeLists.txt
Sending        base/ocsp/shared/conf/CS.cfg.in
Adding         base/ocsp/shared/lib
Adding         base/ocsp/shared/lib/systemd
Adding         base/ocsp/shared/lib/systemd/system
Adding         base/ocsp/shared/lib/systemd/system/pki-ocspd.target
Adding         base/ocsp/shared/lib/systemd/system/pki-ocspd@.service
Sending        base/setup/CMakeLists.txt
Sending        base/setup/pkicommon.pm
Sending        base/setup/pkicreate
Sending        base/setup/pkiremove
Adding         base/setup/scripts
Sending        base/setup/scripts/functions
Adding         base/setup/scripts/pkicontrol
Sending        base/tks/CMakeLists.txt
Sending        base/tks/shared/conf/CS.cfg.in
Adding         base/tks/shared/lib
Adding         base/tks/shared/lib/systemd
Adding         base/tks/shared/lib/systemd/system
Adding         base/tks/shared/lib/systemd/system/pki-tksd.target
Adding         base/tks/shared/lib/systemd/system/pki-tksd@.service
Sending        cmake/Modules/DefineInstallationPaths.cmake
Sending        scripts/compose_pki_core_packages
Sending        scripts/compose_pki_kra_packages
Sending        scripts/compose_pki_ocsp_packages
Sending        scripts/compose_pki_tks_packages
Sending        specs/pki-core.spec
Sending        specs/pki-kra.spec
Sending        specs/pki-ocsp.spec
Sending        specs/pki-tks.spec
Transmitting file data ...................................
Committed revision 2196.

Additional patch needed to migrate existing instances to systemd.

On 09/09/2011, Ade Lee composed the following email:

1. knoxy is supposed to provide a F16 tomcat6 version to be tested.  We
need to test against this version.  We will need to change the spec
files (pki-core, pki-kra, pki-tks, pki-ocsp) to specifically require
this version or greater for f16+.

Up to now, he has just provided versions for f17.  The latest is at
http://koji.fedoraproject.org/koji/taskinfo?taskID=3340759

2. ipa will need to change the calls to "service pki-cad restart" etc.
to the new format as outlined in my email earlier today.

3. post install script code needs to be added to pki-core, pki-kra,
pki-tks, pki-ocsp to migrate existing instances to systemd. 

This should not be too hard - I just ran out of time.  The basic steps are:
loop through the instances in the directory  - /etc/sysconfig/pki/ca/*  (or /etc/sysconfig/pki/tks/* etc).
  --- for each instance, check if the instance has been updated
      if it has, then there will be a link under /etc/systemd/system/pki-cad.target.wants of the form
          pki-cad@<instance_name>.service -> /lib/systemd/system/pki-cad@.service
      if it has not been updated:
         -- create the above link
         -- also the link /var/lib/<instance_name>/<instance_name> points to the tomcat6 systemV file, 
            change it to point to /usr/sbin/tomcat6-sysd
         -- there is also a new entry in CS.cfg, but we do not need to update this as it is only used
            (for now) in the installation panels.  If you want to update it, then add the following to CS.cfg
            pkicreate.systemd.servicename=pki-cad@<instance_name>.service

(In reply to comment #7)
> On 09/09/2011, Ade Lee composed the following email:
> 
> 1. knoxy is supposed to provide a F16 tomcat6 version to be tested.  We
> need to test against this version.  We will need to change the spec
> files (pki-core, pki-kra, pki-tks, pki-ocsp) to specifically require
> this version or greater for f16+.
> 
> Up to now, he has just provided versions for f17.  The latest is at
> http://koji.fedoraproject.org/koji/taskinfo?taskID=3340759
> 

Since no f16 versions of the tomcat6 files have appeared in Koji, this change cannot be instituted in time for Fedora 16 (beta).

However, as per 'Bugzilla Bug #734590 - Refactor JNI libraries for Fedora 16+ . . .', I am including the "tomcat6-sysd" and "tomcat6" scripts utilized for testing these systemd changes to Dogtag.

Created attachment 522854 [details]
The "tomcat6-sysd" script used for testing on "goofy-vm10.dsdev.sjc.redhat.com"

Created attachment 522855 [details]
The "tomcat6" script used for testing on "goofy-vm10.dsdev.sjc.redhat.com"

(In reply to comment #7)
> On 09/09/2011, Ade Lee composed the following email:
> 
> 3. post install script code needs to be added to pki-core, pki-kra,
> pki-tks, pki-ocsp to migrate existing instances to systemd. 
> 
> This should not be too hard - I just ran out of time.  The basic steps are:
> loop through the instances in the directory  - /etc/sysconfig/pki/ca/*  (or
> /etc/sysconfig/pki/tks/* etc).
>   --- for each instance, check if the instance has been updated
>       if it has, then there will be a link under
> /etc/systemd/system/pki-cad.target.wants of the form
>           pki-cad@<instance_name>.service ->
> /lib/systemd/system/pki-cad@.service
>       if it has not been updated:
>          -- create the above link
>          -- also the link /var/lib/<instance_name>/<instance_name> points to
> the tomcat6 systemV file, 
>             change it to point to /usr/sbin/tomcat6-sysd
>          -- there is also a new entry in CS.cfg, but we do not need to update
> this as it is only used
>             (for now) in the installation panels.  If you want to update it,
> then add the following to CS.cfg
>             pkicreate.systemd.servicename=pki-cad@<instance_name>.service

Rudimentary testing (with some slight modifications) revealed some "hanging" issues in these post-installation scripts, but they have been included (and commented out) for pki-core, pki-kra, pki-ocsp, and pki-tks.

(In reply to comment #7)
> On 09/09/2011, Ade Lee composed the following email:
>  
> 3. post install script code needs to be added to pki-core, pki-kra,
> pki-tks, pki-ocsp to migrate existing instances to systemd. 
> 
> This should not be too hard - I just ran out of time.  The basic steps are:
> loop through the instances in the directory  - /etc/sysconfig/pki/ca/*  (or
> /etc/sysconfig/pki/tks/* etc).
>   --- for each instance, check if the instance has been updated
>       if it has, then there will be a link under
> /etc/systemd/system/pki-cad.target.wants of the form
>           pki-cad@<instance_name>.service ->
> /lib/systemd/system/pki-cad@.service
>       if it has not been updated:
>          -- create the above link
>          -- also the link /var/lib/<instance_name>/<instance_name> points to
> the tomcat6 systemV file, 
>             change it to point to /usr/sbin/tomcat6-sysd
>          -- there is also a new entry in CS.cfg, but we do not need to update
> this as it is only used
>             (for now) in the installation panels.  If you want to update it,
> then add the following to CS.cfg
>             pkicreate.systemd.servicename=pki-cad@<instance_name>.service

On 09/12/2011, Ade Lee composed the following email:

I had a couple of minutes and could not resist

The scriptlet needed looks something like the following.  This is totally untested, needs to be rpm macro-ized,  and is just to give you a general idea.

This is for a ca.  Change as needed for kra, tks, ocsp.

post -n pki-ca
for name in `ls /etc/sysconfig/pki/ca`; do
    if [ ! -e "/etc/systemd/system/pki-cad.target.wants/pki-cad@$name.service" ]; then
        ln -s "/lib/systemd/system/pki-cad@.service"   "/etc/systemd/system/pki-cad.target.wants/pki-cad@$name.service"
        [ -e /var/lib/$name/$name ] && unlink /var/lib/$name/$name
        ln -s /usr/sbin/tomcat6-sysd /var/lib/$name/$name
        echo "pkicreate.systemd.service=pkicad@$name.service" >> /var/lib/$name/conf/CS.cfg
    fi
done

Again - totally untested - probably does not even run -- use at your own risk.  Check the variable name in the CS.cfg addition above. You may - or may not - also need to restart the instance.


As stated in the previous comment, rudimentary testing (with some slight modifications) revealed some "hanging" issues in these post-installation scripts, but they have been included (and commented out) for pki-core, pki-kra, pki-ocsp, and pki-tks.

Created attachment 522856 [details]
Code needed to migrate existing instances to systemd . . .

See previous comments regarding this patch . . .

TIP:

# cd pki

# svn status | grep -v ^$ | grep -v ^P | grep -v ^X | grep -v ^?
M       specs/pki-core.spec
M       specs/pki-kra.spec
M       specs/pki-tks.spec
M       specs/pki-ocsp.spec

# svn commit
Sending        specs/pki-core.spec
Sending        specs/pki-kra.spec
Sending        specs/pki-ocsp.spec
Sending        specs/pki-tks.spec
Transmitting file data ....
Committed revision 2198.

pki-kra-9.0.7-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-kra-9.0.7-1.fc16

pki-ocsp-9.0.6-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-ocsp-9.0.6-1.fc16

pki-tks-9.0.6-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-tks-9.0.6-1.fc16

pki-core-9.0.14-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-core-9.0.14-1.fc16

Package pki-kra-9.0.7-1.fc16:
* should fix your issue,
* was pushed to the Fedora 16 testing repository,
* should be available at your local mirror within two days.
Update it with:
# su -c 'yum update --enablerepo=updates-testing pki-kra-9.0.7-1.fc16'
as soon as you are able to.
Please go to the following url:
https://admin.fedoraproject.org/updates/pki-kra-9.0.7-1.fc16
then log in and leave karma (feedback).

Matt,

There is a known systemd bug about hanging restarts when one uses systemctl restart foo.target.  Check with richm for the bug number.

You can avoid this by using systemctl restart pki-cad instead.  You can do this because you iterate through each of the instances in any case.

But the other thing to keep in mind is that we are doing something weird here - we are stopping instances started by systemV start scripts with systemd scripts - who knows if all the relevant pids etc. are cleaned up correctly.

The right way to do this is probably something like this:

%pre
service pki-cad stop

%post 
... all the conversion stuff for each instance foo ..
systemctl start pki-cad

Actually, to really be right, one should probably keep track of which instances were actually running in %pre and only restart those instances.

Ade

Here is the corrected post-install script for the ca.  Matt is planning on including these with his own spec changes.  Will review the whole thing together.

for inst in `ls /etc/sysconfig/pki/ca`; do
    if [ ! -e "/etc/systemd/system/pki-cad.target.wants/pki-cad@${inst}.service" ]; then
        ln -s "/lib/systemd/system/pki-cad@.service" \
            "/etc/systemd/system/pki-cad.target.wants/pki-cad@${inst}.service"
        [ -L /var/lib/${inst}/${inst} ] && unlink /var/lib/${inst}/${inst}
        ln -s /usr/sbin/tomcat6-sysd /var/lib/${inst}/${inst}
  
        if [ -e /var/run/${inst}.pid ]; then
            kill -9 `cat /var/run/${inst}.pid` || :
            rm -f /var/run/${inst}.pid
            echo "pkicreate.systemd.servicename=pki-cad@${inst}.service" >> \
                /var/lib/${inst}/conf/CS.cfg || :
            /bin/systemctl daemon-reload >/dev/null 2>&1 || :
            /bin/systemctl restart pki-cad@${inst}.service || :
        else 
            echo "pkicreate.systemd.servicename=pki-cad@${inst}.service" >> \
                /var/lib/${inst}/conf/CS.cfg || :
        fi
    fi
done
/bin/systemctl daemon-reload >/dev/null 2>&1 || :

pki-core-9.0.14-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

pki-kra-9.0.7-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

pki-tks-9.0.6-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

pki-ocsp-9.0.6-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

tomcatjss-6.0.2-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/tomcatjss-6.0.2-1.fc15

pki-core-9.0.15-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pki-core-9.0.15-1.fc15

pki-console-9.0.5-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pki-console-9.0.5-1.fc15

pki-kra-9.0.8-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pki-kra-9.0.8-1.fc15

pki-ocsp-9.0.7-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pki-ocsp-9.0.7-1.fc15

pki-tks-9.0.7-1.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/pki-tks-9.0.7-1.fc15

dogtag-pki-9.0.0-7.fc15 has been submitted as an update for Fedora 15.
https://admin.fedoraproject.org/updates/dogtag-pki-9.0.0-7.fc15

tomcatjss-6.0.2-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/tomcatjss-6.0.2-1.fc16

pki-core-9.0.15-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-core-9.0.15-1.fc16

pki-console-9.0.5-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-console-9.0.5-1.fc16

pki-kra-9.0.8-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-kra-9.0.8-1.fc16

pki-ocsp-9.0.7-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-ocsp-9.0.7-1.fc16

pki-tks-9.0.7-1.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/pki-tks-9.0.7-1.fc16

dogtag-pki-9.0.0-7.fc16 has been submitted as an update for Fedora 16.
https://admin.fedoraproject.org/updates/dogtag-pki-9.0.0-7.fc16

dogtag-pki-9.0.0-7.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

pki-console-9.0.5-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

pki-tks-9.0.7-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

pki-core-9.0.15-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

pki-ocsp-9.0.7-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

tomcatjss-6.0.2-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

pki-kra-9.0.8-1.fc16 has been pushed to the Fedora 16 stable repository.  If problems still persist, please make note of it in this bug report.

dogtag-pki-9.0.0-7.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

pki-ocsp-9.0.7-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

pki-core-9.0.15-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

tomcatjss-6.0.2-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

pki-console-9.0.5-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.

pki-tks-9.0.7-1.fc15 has been pushed to the Fedora 15 stable repository.  If problems still persist, please make note of it in this bug report.