Bug 710343 - zabbix_server_mysql fails to start under systemd
Summary: zabbix_server_mysql fails to start under systemd
Keywords:
Status: CLOSED WONTFIX
Alias: None
Product: Fedora
Classification: Fedora
Component: zabbix
Version: 15
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
Assignee: Dan Horák
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2011-06-03 06:20 UTC by Kyle Brantley
Modified: 2012-08-07 20:19 UTC (History)
8 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2012-08-07 20:19:41 UTC
Type: ---


Attachments (Terms of Use)
relevant audit.log entries (1.74 KB, text/plain)
2011-06-04 02:38 UTC, Kyle Brantley
no flags Details
full audit log (6.66 KB, text/plain)
2011-06-07 03:19 UTC, Kyle Brantley
no flags Details
SELinux and zabbix_server_pgsql (2.34 KB, text/x-log)
2011-07-16 18:41 UTC, Volker Fröhlich
no flags Details
ausearch results for recent f15 install + zabbix_server_mysql (9.96 KB, text/plain)
2011-10-11 14:28 UTC, Kyle Brantley
no flags Details

Description Kyle Brantley 2011-06-03 06:20:53 UTC
Description of problem:
zabbix_server_mysql fails to start when executed under systemd on Fedora 15.

Version-Release number of selected component (if applicable):
Name        : zabbix-server-mysql
Arch        : x86_64
Version     : 1.8.5
Release     : 1.fc15


How reproducible:
100%

Steps to Reproduce:
1. `/etc/init.d/zabbix-server start` or `systemctl start zabbix-server.service`
2. 
3.
  
Actual results:
systemd thinks that it started (status confirms that it is running), when it isn't (there are no processes running)

Expected results:
running zabbix_server_mysql processes


Additional info:

This is a fresh x86_64 install of F15 with selinux enabled. The database was restored from a backup, and initially I was using a config file from F14, however to ensure that the config file was not the source of the issue, I removed the packages, renamed the config file, reinstalled the packages and then repopulated the relevant DB values in the config file. I did not wipe the database yet, as the startup does not get close to touching the database before it fails.

I did quite a bit of searching before narrowing it down to being executed under systemd. When I execute zabbix_server_mysql straight from the bash (outside of the init scripts/systemd containers), it starts up fine and begins monitoring everything that it is configured to.


When I start it under systemd, however, on initial process creation:

* The setpriority() call fails with permission denied,
[pid 11880] setpriority(PRIO_PROCESS, 0, 5 <unfinished ...>
[pid 11877] exit_group(0)               = ?
Process 11874 resumed
Process 11877 detached
[pid 11880] <... setpriority resumed> ) = -1 EACCES (Permission denied)
[pid 11880] write(2, "zabbix_server_mysql [11880]: ", 29) = 29
[pid 11880] write(2, "Unable to set process priority t"..., 53) = 53

* The semget() call fails with permission denied,
[pid 11880] semget(0x7a0307a8, 9, IPC_CREAT|IPC_EXCL|0600 <unfinished ...>
[pid 11881] <... close resumed> )       = 0
[pid 11874] <... rt_sigprocmask resumed> [], 8) = 0
[pid 11881] open("/lib64/libc.so.6", O_RDONLY) = 3
[pid 11880] <... semget resumed> )      = -1 EACCES (Permission denied)

* and then (I believe) as a result of the failed semaphore creation, the mutex fails to be created as a whole, and all of this is logged to zabbix_log:
zabbix_server_mysql [11880]: Unable to set process priority to 5. Leaving default.
zabbix_server_mysql [11880]: Can not create Semaphore [Permission denied]
zabbix_server_mysql [11880]: Unable to create mutex for log file

and the server fails to start. Again, if I simply execute `zabbix_server_mysql` then everything starts up without issue.

Commenting out the LogFile config statement does not resolve this problem.

Comment 1 Dan Horák 2011-06-03 16:50:25 UTC
Oh, both the failures (setpriority and semget) are in fact SELinux issues if I see the logs correctly. You can switch the system to the permissive mode with "setenforce Permissive" and zabbix should start.

Comment 2 Kyle Brantley 2011-06-03 17:20:49 UTC
Yes, that seems to have fixed it. I could have sworn that I checked audit.log when I was first checking through this and didn't see anything though -- oh well.

I will upload the relevant audit.log entries when I get home.

If I switch selinux back to enforcing after it has started, will that cause any issues?

Comment 3 Kyle Brantley 2011-06-04 02:38:58 UTC
Created attachment 502939 [details]
relevant audit.log entries

Comment 4 Volker Fröhlich 2011-06-05 00:13:49 UTC
I don't get zabbix_server_pgsql running with a fresh F15 and fresh Zabbix. Selinux is permissive. Neither running "service" nor "systemctl" achieve what they should.

Systemd's status says "Active: active (exited) ...". No processes exist.

I had to create the /var/run/zabbix directory though, to get it working with /usr/sbin/zabbix_server.

Comment 5 Volker Fröhlich 2011-06-05 00:14:48 UTC
Should have been /usr/sbin/zabbix_server_pgsql in the last line.

Comment 6 Kyle Brantley 2011-06-05 22:04:36 UTC
Volker, systemd will clean up the /var/run/zabbix directory when you stop the service, so having to create this directory is expected.

It would seem that zabbix_server_pgsql is just as broken as zabbix_server_mysql, though.

Comment 7 Dan Horák 2011-06-06 06:56:09 UTC
zabbix-1.8.5-3.fc15 (now waiting to be pushed into updates-testing) contains the fix for the /var/run/zabbix issue

Comment 8 Miroslav Grepl 2011-06-06 07:44:18 UTC
Kyle,

please turn on full auditing

# auditctl -w /etc/shadow -p w

Try to recreate AVC. Then execute

# ausearch -m avc -ts recent

dac_override means there exists a file with bad ownership/permissions.

Comment 9 Daniel Walsh 2011-06-06 19:10:26 UTC

*** This bug has been marked as a duplicate of bug 674627 ***

Comment 10 Daniel Walsh 2011-06-06 19:11:17 UTC
Oops not really a dup.

Comment 11 Kyle Brantley 2011-06-07 03:19:37 UTC
Created attachment 503367 [details]
full audit log

Attached is the full audit log.

Comment 12 Daniel Walsh 2011-06-07 19:44:49 UTC
Ok I just updated the pool with fixes for your problems.

Will be in selinux-policy-3.9.16-29.fc15

Comment 13 Kyle Brantley 2011-07-10 23:48:45 UTC
This still fails to start via systemctl with selinux-policy-3.9.16-30.fc15. Executing zabbix_server_mysql still works great.

Is any other information needed?

Comment 14 Daniel Walsh 2011-07-11 22:01:26 UTC
Please attach the latest AVC's you are seeing?

Comment 15 Volker Fröhlich 2011-07-16 18:41:22 UTC
Created attachment 513501 [details]
SELinux and zabbix_server_pgsql

Comment 16 Hiran Arias 2011-07-24 00:04:51 UTC
There is no /lib/systemd/*/zabbix-server.service file so systemctl shouldn't be use to start zabbix.

To solve I just put SYSTEMCTL_SKIP_REDIRECT=true before the call for /etc/rc.d/init.d/funtions in the init script file for zabbix-server in /etc/init.d/zabbix-server

Comment 17 Kyle Brantley 2011-10-11 14:28:14 UTC
Created attachment 527453 [details]
ausearch results for recent f15 install + zabbix_server_mysql

Sorry for the bug negligence..

I just moved my zabbix install to another fedora machine and hit this again, and then remember that it was never full fixed, because I never gave everything that was needed. :)

Attached is the full results of the `ausearch -m avc -ts recent` for a brand new (but full updated) F15 install.

Comment 18 Kyle Brantley 2011-12-18 05:56:32 UTC
This is now working under F16.

Comment 19 Fedora End Of Life 2012-08-07 20:19:46 UTC
This message is a notice that Fedora 15 is now at end of life. Fedora
has stopped maintaining and issuing updates for Fedora 15. It is
Fedora's policy to close all bug reports from releases that are no
longer maintained. At this time, all open bugs with a Fedora 'version'
of '15' have been closed as WONTFIX.

(Please note: Our normal process is to give advanced warning of this
occurring, but we forgot to do that. A thousand apologies.)

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, feel free to reopen
this bug and simply change the 'version' to a later Fedora version.

Bug Reporter: Thank you for reporting this issue and we are sorry that
we were unable to fix it before Fedora 15 reached end of life. If you
would still like to see this bug fixed and are able to reproduce it
against a later version of Fedora, you are encouraged to click on
"Clone This Bug" (top right of this page) and open it against that
version of Fedora.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

The process we are following is described here:
http://fedoraproject.org/wiki/BugZappers/HouseKeeping


Note You need to log in before you can comment on or make changes to this bug.