Description of problem:
On a clean install from ovirt-3.6-snapshot repo, vdsm fails to start during hosted-engine deployment or adding a node to the existing engine.
systemctl status vdsmd.service:
Process: 25532 ExecStart=/usr/share/vdsm/daemonAdapter -0 /dev/null -1 /dev/null -2 /dev/null /usr/share/vdsm/vdsm (code=exited, status=1/FAILURE)
Main PID: 25532 (code=exited, status=1/FAILURE); : 25577 (vdsmd_init_comm)
Snip from /var/log/mom.log
2015-10-21 21:19:04,556 - mom - INFO - MOM starting
2015-10-21 21:19:04,581 - mom.HostMonitor - INFO - Host Monitor starting
2015-10-21 21:19:04,581 - mom - INFO - hypervisor interface vdsmxmlrpc
2015-10-21 21:19:07,372 - mom - ERROR - Failed to initialize MOM threads
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/mom/__init__.py", line 29, in run
hypervisor_iface = self.get_hypervisor_interface()
File "/usr/lib/python2.7/site-packages/mom/__init__.py", line 216, in get_hypervisor_interface
module = __import__('mom.HypervisorInterfaces.' + name + 'Interface', None, None, name)
File "/usr/lib/python2.7/site-packages/mom/HypervisorInterfaces/vdsmxmlrpcInterface.py", line 24, in <module>
from vdsm import vdscli
File "/usr/lib/python2.7/site-packages/vdsm/vdscli.py", line 29, in <module>
from . import sslutils
File "/usr/lib/python2.7/site-packages/vdsm/sslutils.py", line 28, in <module>
from vdsm.utils import (
File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 66, in <module>
from ovirtnode import ovirtfunctions
File "/usr/lib/python2.7/site-packages/ovirtnode/ovirtfunctions.py", line 1867, in <module>
logger = setup_custom_logger()
File "/usr/lib/python2.7/site-packages/ovirtnode/ovirtfunctions.py", line 1858, in setup_custom_logger
handler = logging.FileHandler(log_file)
File "/usr/lib64/python2.7/logging/__init__.py", line 902, in __init__
File "/usr/lib64/python2.7/logging/__init__.py", line 925, in _open
stream = open(self.baseFilename, self.mode)
IOError: [Errno 13] Permission denied: '/tmp/ovirt.log'
Changing permissions from root:root to vdsm:kvm on /tmp/ovirt.log fixes the issue, and vdsm starts successfully.
Selinux is in permissive mode, but it was enforced during the installation of the components (no denials from the logs as far as I can tell).
Version-Release number of selected component (if applicable):
Apparently, the mere
from ovirtnode import ovirtfunctions
logs onto /tmp/ovirt.log.
If the first import happened to be in root context, we'd get this error.
This bug is not marked for z-stream, yet the milestone is for a z-stream version, therefore the milestone has been reset.
Please set the correct milestone or add the z-stream flag.
Created attachment 1085643 [details]
chown the /tmp/ovirt.log
When I remove ovirt.log from the tmp directory and reboot, vdsm starts successfully. Supervdsmd creates the temporary log file with root privileges, and when vdsm starts with user privileges, it changes ownership of /tmp/ovirt.log to vdsm:kvm.
This is an relict from old ovirt-node times.
The logging is setup when ovirtnode.ovirtfunctions is getting imported, which is not good.
Normally the import happens because of persistence, in that case ovirt.node.utils.fs.Config can be used these days, which should not suffer the import problems.
Target release should be placed once a package build is known to fix a issue. Since this bug is not modified, the target version has been reset. Please use target milestone to plan a fix for a oVirt release.
(In reply to Fabian Deutsch from comment #6)
> Normally the import happens because of persistence, in that case
> ovirt.node.utils.fs.Config can be used these days, which should not suffer
> the import problems.
Yes, forcing import from ovirt.node.utils.fs (commenting out importing from ovirtnode) in vdsm/utils.py does not prevent mom-vdsm and vdsm units from starting anymore.
Thanks Fabian, I caught this by accident, thought I should report it (even though I don't fully understand implications of it all, yet).
Ivan, cool, nice to hear that it worked.
Moving this over to vdsm where the patch resides.