Bug 1727378 - ipa-server-install fails after recent 389-ds-base update
Summary: ipa-server-install fails after recent 389-ds-base update
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Fedora
Classification: Fedora
Component: 389-ds-base
Version: 29
Hardware: Unspecified
OS: Unspecified
unspecified
unspecified
Target Milestone: ---
Assignee: mreynolds
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2019-07-05 18:47 UTC by Dinesh Prasanth
Modified: 2019-07-12 13:49 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2019-07-12 13:49:50 UTC


Attachments (Terms of Use)
dirsrv-access.log (1008.35 KB, text/plain)
2019-07-05 18:54 UTC, Dinesh Prasanth
no flags Details
dirsrv-error.log (55.01 KB, text/plain)
2019-07-05 18:54 UTC, Dinesh Prasanth
no flags Details
dirsrv-journalctl.log (9.56 KB, text/plain)
2019-07-05 18:55 UTC, Dinesh Prasanth
no flags Details
stacktrace (26.73 KB, text/plain)
2019-07-08 16:06 UTC, Viktor Ashirov
no flags Details
core (4.40 MB, application/x-lz4)
2019-07-08 16:06 UTC, Viktor Ashirov
no flags Details

Description Dinesh Prasanth 2019-07-05 18:47:47 UTC
Description of problem:

When running ipa-server-install, the command fails at the LDAP update step with the following error:

"Upgrade failed with cannot connect to 'ldapi://%2Fvar%2Frun%2Fslapd-PKI-TEST.socket':".

The issue was originally caught by PKI's nightly job running inside a docker container: https://travis-ci.org/dogtagpki/pki-nightly-test/builds/554537452


Version-Release number of selected component (if applicable):

389-ds-base-1.4.0.24-1.fc29.x86_64
389-ds-base-legacy-tools-1.4.0.24-1.fc29.x86_64
389-ds-base-libs-1.4.0.24-1.fc29.x86_64

How reproducible:
Always

Steps to Reproduce:
1. docker pull fedora:29

2. docker run \
    --detach \
    --name=ipatest \
    --hostname='master.pki.test' \
    --privileged \
    --tmpfs /tmp \
    --tmpfs /run \
    -v /sys/fs/cgroup:/sys/fs/cgroup:ro \
    -e COPR_REPO="@pki/master" \
    -e container=docker \
    --expose=389 \
    --expose=8080 \
    --expose=8443 \
    -i \
    fedora:29 "/usr/sbin/init"

3. docker exec -it ipatest /bin/bash

<inside the container, executue the following steps:>

1. dnf install -y dnf-plugins-core
2. dnf copr enable -y @pki/master
3. dnf install -y freeipa-server freeipa-server-dns freeipa-server-trust-ad python3-ipatests --best --allowerasing
4. sysctl net.ipv6.conf.lo.disable_ipv6=0
5. server_password="Secret.123"
6. ipa-server-install -U --domain pki.test --realm PKI.TEST -p ${server_password} -a ${server_password} --setup-dns --setup-kra --auto-forwarders



Actual results:

.
.
<clipped>
.
.
Applying LDAP updates
Upgrading IPA:. Estimated time: 1 minute 30 seconds
  [1/10]: stopping directory server
  [2/10]: saving configuration
  [3/10]: disabling listeners
  [4/10]: enabling DS global lock
  [5/10]: disabling Schema Compat
  [6/10]: starting directory server
  [7/10]: upgrading server
Upgrade failed with cannot connect to 'ldapi://%2Fvar%2Frun%2Fslapd-PKI-TEST.socket': 
  [error] RuntimeError: cannot connect to 'ldapi://%2Fvar%2Frun%2Fslapd-PKI-TEST.socket': 
  [cleanup]: stopping directory server
  [cleanup]: restoring configuration
Update failed: cannot connect to 'ldapi://%2Fvar%2Frun%2Fslapd-PKI-TEST.socket': 
The ipa-server-install command failed. See /var/log/ipaserver-install.log for more information

Expected results:
The ipa-client-install command was successful

Additional info:
This was reproducible in a local docker env

Comment 1 Dinesh Prasanth 2019-07-05 18:54:32 UTC
Created attachment 1587790 [details]
dirsrv-access.log

Comment 2 Dinesh Prasanth 2019-07-05 18:54:58 UTC
Created attachment 1587791 [details]
dirsrv-error.log

Comment 3 Dinesh Prasanth 2019-07-05 18:55:38 UTC
Created attachment 1587792 [details]
dirsrv-journalctl.log

Comment 4 Viktor Ashirov 2019-07-08 12:09:58 UTC
Is it the same issue https://pagure.io/389-ds-base/issue/50343 ?

Comment 5 mreynolds 2019-07-08 13:37:02 UTC
(In reply to Viktor Ashirov from comment #4)
> Is it the same issue https://pagure.io/389-ds-base/issue/50343 ?

It could be if dscreate was used.

Dinesh, is the server installed with dscreate (which enables LDAPI by default), or with setup-ds.pl (which does NOT enable LDAPI).

Comment 6 Viktor Ashirov 2019-07-08 14:13:08 UTC
That version of freeipa uses setup-ds.pl.
I recreated an environment and got a crash in libmemberof-plugin.so:

Jul 08 14:00:01 master.pki.test kernel: ns-slapd[31710]: segfault at 0 ip 00007f2183c14e2d sp 00007f215cdee8b0 error 4 in libmemberof-plugin.so[7f2183c10000+9000]
Jul 08 14:00:01 master.pki.test kernel: Code: ff ff 48 8b 74 24 38 89 c3 48 85 f6 0f 84 f3 00 00 00 45 84 f6 0f 84 ea 00 00 00 48 8b 45 40 48 8b 54 24 40 41 be 01 00 00 00 <48> 8b 38 e8 db c1 ff ff 41 89 c7 85 c0 74 bc 48 8b 7c 24 38 e8 9a
Jul 08 14:00:02 master.pki.test systemd[1]: dirsrv@PKI-TEST.service: Main process exited, code=killed, status=11/SEGV
Jul 08 14:00:02 master.pki.test systemd[1]: dirsrv@PKI-TEST.service: Failed with result 'signal'.


It didn't produce a core, I will try to get one.

Comment 7 mreynolds 2019-07-08 14:20:20 UTC
(In reply to Viktor Ashirov from comment #6)
> That version of freeipa uses setup-ds.pl.
> I recreated an environment and got a crash in libmemberof-plugin.so:
> 
> Jul 08 14:00:01 master.pki.test kernel: ns-slapd[31710]: segfault at 0 ip
> 00007f2183c14e2d sp 00007f215cdee8b0 error 4 in
> libmemberof-plugin.so[7f2183c10000+9000]
> Jul 08 14:00:01 master.pki.test kernel: Code: ff ff 48 8b 74 24 38 89 c3 48
> 85 f6 0f 84 f3 00 00 00 45 84 f6 0f 84 ea 00 00 00 48 8b 45 40 48 8b 54 24
> 40 41 be 01 00 00 00 <48> 8b 38 e8 db c1 ff ff 41 89 c7 85 c0 74 bc 48 8b 7c
> 24 38 e8 9a
> Jul 08 14:00:02 master.pki.test systemd[1]: dirsrv@PKI-TEST.service: Main
> process exited, code=killed, status=11/SEGV
> Jul 08 14:00:02 master.pki.test systemd[1]: dirsrv@PKI-TEST.service: Failed
> with result 'signal'.
> 
> 
> It didn't produce a core, I will try to get one.
  
Ah, so LDAPI is a red herring.  The server simply crashed and then the LDAPI connection failed (of course).

Comment 8 Viktor Ashirov 2019-07-08 16:06:01 UTC
Created attachment 1588429 [details]
stacktrace

Comment 9 Viktor Ashirov 2019-07-08 16:06:51 UTC
Created attachment 1588430 [details]
core

Comment 10 Viktor Ashirov 2019-07-10 12:48:39 UTC
Dinesh, could you please check if the tests pass for you with the latest build from https://bodhi.fedoraproject.org/updates/FEDORA-2019-1bfca53248 ?
Thanks!

Comment 11 Dinesh Prasanth 2019-07-10 17:59:00 UTC
Yes, both PKI 10.6 and 10.7 pass in the CI:

Nightly run triggered a while back: https://travis-ci.org/dogtagpki/pki-nightly-test/builds/556644463
Version of DS pulled from bodhi: https://travis-ci.org/dogtagpki/pki-nightly-test/jobs/556644466#L2698

Thanks for the fix @viktor and @mreynolds! :)

Comment 12 Viktor Ashirov 2019-07-12 13:49:50 UTC
Thank you for confirming.
Looks like update https://bodhi.fedoraproject.org/updates/FEDORA-2019-1bfca53248 was pushed to stable. Closing.


Note You need to log in before you can comment on or make changes to this bug.