Bug 2179278

Summary:	dirsrv failed to start after reboot because "dirsrv" did not have access on /run/dirsrv
Product:	Red Hat Enterprise Linux 9	Reporter:	Ding-Yi Chen <dchen>
Component:	389-ds-base	Assignee:	LDAP Maintainers <idm-ds-dev-bugs>
Status:	CLOSED ERRATA	QA Contact:	LDAP QA Team <idm-ds-qe-bugs>
Severity:	high	Docs Contact:	Evgenia Martynyuk <emartyny>
Priority:	high
Version:	9.1	CC:	bsmejkal, emartyny, idm-ds-dev-bugs, mreynolds, vashirov
Target Milestone:	rc	Keywords:	Triaged
Target Release:	9.3	Flags:	pm-rhel: mirror+
Hardware:	Unspecified
OS:	Unspecified
Whiteboard:	sync-to-jira
Fixed In Version:	389-ds-base-2.3.6-2.el9	Doc Type:	Bug Fix
Doc Text:	.The `dirsrv` service now starts correctly after reboot Previously, `dirsrv` service could fail to start after reboot because `dirsrv` service did not explicitly wait for `systemd-tmpfiles-setup.service` to finish. This led to a race condition. With this update, `dirsrv` service waits for the `systemd-tmpfiles-setup.service` to finish and no longer fail to start after reboot.	Story Points:	---
Clone Of:		Environment:
Last Closed:	2023-11-07 08:25:18 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Ding-Yi Chen 2023-03-17 07:53:43 UTC

Description of problem:

After reboot, dirsrv failed to start because /run/dirsrv is not create yet.

~~~
Mar 17 14:58:39 dchen ns-slapd[1994]: [17/Mar/2023:14:58:39.117785245 +1000] - EMERG - main - Unable to access nsslapd-rundir: No such file or directory
Mar 17 14:58:39 dchen ns-slapd[1994]: [17/Mar/2023:14:58:39.127244140 +1000] - EMERG - main - Ensure that user "dirsrv" has read and write permissions on /run/dirsrv
Mar 17 14:58:39 dchen ns-slapd[1994]: [17/Mar/2023:14:58:39.136663531 +1000] - EMERG - main - Shutting down.
Mar 17 14:58:39 dchen journal[2066]: Skipping parental controls support as it’s disabled
Mar 17 14:58:39 dchen systemd[1]: dirsrv: Main process exited, code=exited, status=1/FAILURE
Mar 17 14:58:39 dchen systemd[1]: dirsrv: Failed with result 'exit-code'.
Mar 17 14:58:39 dchen systemd[1]: Failed to start 389 Directory Server EXAMPLE-COM..
Mar 17 14:58:39 dchen ipactl[1686]: Failed to start Directory Service: CalledProcessError(Command ['/bin/systemctl', 'start', 'dirsrv'] returned non-zero exit status 1)
~~~

This is because /run mounted as tmpfs, thus it is possible no dirsrv/ under it
~~~
# mount  | grep /run
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,size=13107724k,nr_inodes=819200,mode=755,inode64)
~~~


Version-Release number of selected component (if applicable):

389-ds-base-2.1.3-4.el9_1.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Install RH IdM
2. ensure 'nsslapd-rundir: /run/dirsrv' in dse.ldif
3. ensure /run is mounted as tmpfs
4. reboot

Actual results:

Following error message shown:

EMERG - main - Ensure that user "dirsrv" has read and write permissions on /run/dirsrv

Expected results:

389ds create the /run/dirsrv with correct owner and file mod

Additional info:

IPA service that also suffer similar symptom: ipa, named, samba

Workaround:

1. create the directory before 389ds-start, like:
  mkdir -p dirsrv
  chmod -v 770 dirsrv
  chown -v dirsrv:dirsrv dirsrv
  mkdir -p lock/dirsrv/slapd-CHEN-HOME
  chmod -vR 770 lock/dirsrv
  chown -vR dirsrv:dirsrv lock/dirsrv

2. Service systemd-tmpfiles-setup also create the required directories.  

- service ipa, name

Comment 1 Viktor Ashirov 2023-07-14 10:52:21 UTC

I believe there is a race condition when dirsrv tries to start before systemd-tmpfiles-setup. I couldn't reproduce it myself, but this is not the first time this issue gets reported.

Perhaps we should explicitly require it in our unit file, like so:
After=systemd-tmpfiles-setup.service

Comment 2 Viktor Ashirov 2023-07-26 17:13:40 UTC

https://github.com/389ds/389-ds-base/pull/5865

Comment 6 Viktor Ashirov 2023-08-14 15:59:32 UTC

Build tested: 389-ds-base-2.3.6-2.el9.x86_64

# systemctl cat dirsrv@localhost | grep tmpfiles
After=chronyd.service ntpd.service network-online.target systemd-tmpfiles-setup.service
Wants=systemd-tmpfiles-setup.service

# systemctl list-dependencies dirsrv@localhost | grep tmpfiles
● ├─systemd-tmpfiles-setup.service
●   ├─systemd-tmpfiles-setup-dev.service
●   ├─systemd-tmpfiles-setup.service

Marking as Verified:Tested, SanityOnly.

Comment 11 errata-xmlrpc 2023-11-07 08:25:18 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (389-ds-base bug fix and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2023:6350