Bug 1268223 - VDSM doesn't start due to failing dependency
Summary: VDSM doesn't start due to failing dependency
Alias: None
Product: vdsm
Classification: oVirt
Component: Services
Version: 4.17.0
Hardware: x86_64
OS: Linux
urgent vote
Target Milestone: ovirt-3.6.0-rc3
: 4.17.8
Assignee: Yaniv Bronhaim
QA Contact: Aharon Canan
Whiteboard: storage
Depends On:
TreeView+ depends on / blocked
Reported: 2015-10-02 08:20 UTC by Christopher Pereira
Modified: 2016-03-10 15:23 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Last Closed: 2015-10-07 16:08:03 UTC
oVirt Team: Storage
amureini: ovirt-3.6.0?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?

Attachments (Terms of Use)
vdsm.log (55.92 KB, text/plain)
2015-10-02 08:20 UTC, Christopher Pereira
no flags Details
ovirt-hosted-engine-setup-20151002051145-j0j7tn.log (287.31 KB, text/plain)
2015-10-02 08:21 UTC, Christopher Pereira
no flags Details
sosreport (7.99 MB, application/x-xz)
2015-10-02 14:33 UTC, Christopher Pereira
no flags Details

Description Christopher Pereira 2015-10-02 08:20:10 UTC
Created attachment 1079385 [details]

[ INFO  ] Stage: Transaction setup
[ INFO  ] Stage: Misc configuration
[ INFO  ] Stage: Package installation
[ INFO  ] Stage: Misc configuration
[ INFO  ] Configuring libvirt
[ INFO  ] Configuring VDSM
[ INFO  ] Starting vdsmd
[ ERROR ] Failed to execute stage 'Misc configuration': Failed to start service 'vdsmd'
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20151002051315.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination


Attached relevant logs


# rpm -qa |grep ovirt

Comment 1 Christopher Pereira 2015-10-02 08:21:44 UTC
Created attachment 1079386 [details]

Comment 2 Sandro Bonazzola 2015-10-02 08:45:39 UTC
Can you please attach full sos report?

Comment 3 Christopher Pereira 2015-10-02 14:33:59 UTC
Created attachment 1079462 [details]

Comment 4 Christopher Pereira 2015-10-04 04:06:39 UTC
VDSMd receives SIGTERM 15 (kill) during hosted-engine setup.
CGROUP related?

Comment 5 Sandro Bonazzola 2015-10-07 15:27:14 UTC
2015-10-02 05:13:15 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:941 execute-output: ('/bin/systemctl', 'start', 'vdsmd.service') stderr:
A dependency job for vdsmd.service failed. See 'journalctl -xn' for details.

SuperVDSM started correctly:
MainThread::DEBUG::2015-10-02 05:13:15,470::supervdsmServer::520::SuperVdsm.Server::(main) Making sure I'm root - SuperVdsm

But VDSM didn't start.
Looking at /var/log/messages and at systemd status looks like the following services failed to start:
mnt-datacenter.mount  loaded failed failed IMATRONIX DataCenter Mount
ix-datacenter.service loaded failed failed IMATRONIX DataCenter

mom process stopped earlier and never restart:
2015-10-02 03:07:04,596 - mom - INFO - MOM ending

In /var/log/messages I see:
Oct  2 04:01:13 h8 mount: Mount failed. Please check the log file for more details.
Oct  2 04:01:13 h8 systemd: Failed at step EXEC spawning /mnt/datacenter/bin/init-datacenter: Transport endpoint is not connected
Oct  2 04:01:13 h8 systemd: mnt-datacenter.mount mount process exited, code=exited status=1
Oct  2 04:01:13 h8 systemd: Unit mnt-datacenter.mount entered failed state.
Oct  2 04:01:13 h8 systemd: ix-datacenter.service: main process exited, code=exited, status=203/EXEC
Oct  2 04:01:13 h8 systemd: Failed to start IMATRONIX DataCenter.
Oct  2 04:01:13 h8 systemd: Dependency failed for Virtual Desktop Server Manager.
Oct  2 04:01:13 h8 systemd: Dependency failed for MOM instance configured for VDSM purposes.
Oct  2 04:01:13 h8 systemd: 
Oct  2 04:01:13 h8 systemd: 

I think that your disk mount failure is causing the vdsm dependency failure here.
Can you fix the monut issue and retry?
I'm moving this to VDSM component, allowing them to review the issue as well.

Comment 6 Christopher Pereira 2015-10-07 16:08:03 UTC
Could be, but the strange thing is that starting VDSMD via "systemctl start vdsmd" works fine, even when the other service is throwing an error because of the mount.

Closing for now as "NOTABUG", until I can confirm on another host (this one is in production now).

Thanks for your time, Sandro.

Comment 7 Sandro Bonazzola 2015-10-09 10:09:14 UTC
You're welcome Christopher.

Note You need to log in before you can comment on or make changes to this bug.