Red Hat Bugzilla – Bug 1267031
dbmail services daemons fail to start (systemd) on CentOS 7 (v7.1)
Last modified: 2016-01-08 17:28:36 EST
Description of problem:
dbmail-imapd, dbmail-lmtpd, dbmail-pop3d, dbmail-timsieved cannot be started automatically (systemd) at boot time or manually using systemctl start command
Version-Release number of selected component (if applicable):
dbmail.x86_64 3.2.3-1.el7 @epel
dbmail-auth-ldap.x86_64 3.2.3-1.el7 @epel
Configure dbmail with a database (in my case mariadb coming with CentOS 7) and start the daemon manually (from the command line) to test. Once everything is properly configured kill the test daemons and enable the services using
systemctl enable dbmail-imapd
systemctl enable dbmail-lmtpd
systemctl enable dbmail-pop3d
systemctl enable dbmail-timsieved
and reboot. The services will not start and will not leave any trace in the logs not even when adding -vv to the command line (by editing the respective /usr/lib/systemd/system/dbmail-XXXXX.service).
Steps to Reproduce:
1. configure dbmail
2. enable the respective services
3. boot or start the service using systemctl start
The daemons will not start and will not leave any message in /var/log/dbmail.log or dbmail.err
daemons to start as in CentOS 6
The default /usr/lib/systemd/system/dbmail-XXXXX.service files do not include mariadb.service in "After", however this isn't really the issue because after having added mariadb.service the daemons still do not start.
I discovered that if I issue the following commands manually in sequence:
# systemctl daemon-reload
# systemctl start dbmail-XXXXX #replace XXXXX withimapd|lmtpd|pop3d|timsieved
at this point one service will start. If I want to start the next service I have to issue "systemctl daemon-reload" again, and again before each subsequent service.
As you may understand this is unacceptable and it is most definately a bug, probably lying with "systemd", although dbmail rpm packages and startup files should be amended to cater for mariadb.service too since it is the default DB on CentOS 7.
Please find below a working systemd file for dbmail-imapd. The same can be edited and applied to pop3d, lmtpd and timsieved.
Description=DBMail IMAP Server
After=syslog.target network.target mariadb.service postgresql.service
ExecStartPre=/usr/bin/install -d -m 0777 -o nobody -g nobody /var/run/dbmail
ExecStart=/usr/sbin/dbmail-imapd -p /var/run/dbmail/dbmail-imapd.pid
ExecStop=/usr/bin/kill -TERM $MAINPID
ExecReload=/usr/bin/kill -HUP $MAINPID
I have tracked this down further because I have the same problem on F23. I have also looked at the solution in comment #1 and it too has the "daemon-reload" problem. I don't think systemd is at fault here because the dbmail code does some strange things on its way to spawning the working daemon proc. Other legacy daemons do work, probably because they let the spawned child do most of the initialization.
My F20 based service had the same problem of failing to start although it would work sometimes rather than the f23 behavior of not at all...
What works is to move from "forked" to "simple" daemon. Fortunately, dbmail supports this. Here is my override file for imapd:
# cat /etc/systemd/system/dbmail-imapd.service.d/override.conf
As with comment #1, apply this to all the other daemons.
By going to "simple" we let systemd manage the cgroup itself and eliminate the pid file (always was a) hack. The '-D' flag bypasses the daemonizing bit, saving a fork+exit in startup. However, while fixing the "daemon-reload" problem, the daemons still did not start at boot time even though I could reliably start them later with systemctl.
The second item is the "After". My dbmail instance is isolated in a VM and the database (postgresql) is on a bare-metal host. Without waiting for "nss-lookup.target", each of the daemons fail because they too eagerly and silently bail during initialization because they are racing not only the basic network but the nss services etc. because (in my case) the route table has not settled down. This target waits on DNS connectivity.
BTW, I found this by cranking up the file debugging to "511" but even that was vague. I ended up using strace which showed that the DB connect was failing with an ENOROUTE error when trying to connect to my DNS indicating that network.target is too weak a dependency. Note that stuffing the server address into /etc/host just postpones until the DB socket is connected...
I recommend that the package service files change to use "simple" and "-D". I haven't straced the "forked" daemons but I suspect that dbmail is quietly exiting due to unhappy initialization code in the parent that really should be done by the spawned child.
I also suggest changing the "After" so that partitioned services like mine would work. network.target is too weak and nss-lookup.target would work just fine for local (UNIX socket) database connections.