Bug 1268223 - VDSM doesn't start due to failing dependency
VDSM doesn't start due to failing dependency
Status: CLOSED NOTABUG
Product: vdsm
Classification: oVirt
Component: Services (Show other bugs)
4.17.0
x86_64 Linux
unspecified Severity urgent (vote)
: ovirt-3.6.0-rc3
: 4.17.8
Assigned To: Yaniv Bronhaim
Aharon Canan
storage
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2015-10-02 04:20 EDT by Christopher Pereira
Modified: 2016-03-10 10:23 EST (History)
5 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2015-10-07 12:08:03 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: Storage
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
amureini: ovirt‑3.6.0?
rule-engine: planning_ack?
rule-engine: devel_ack?
rule-engine: testing_ack?


Attachments (Terms of Use)
vdsm.log (55.92 KB, text/plain)
2015-10-02 04:20 EDT, Christopher Pereira
no flags Details
ovirt-hosted-engine-setup-20151002051145-j0j7tn.log (287.31 KB, text/plain)
2015-10-02 04:21 EDT, Christopher Pereira
no flags Details
sosreport (7.99 MB, application/x-xz)
2015-10-02 10:33 EDT, Christopher Pereira
no flags Details

  None (edit)
Description Christopher Pereira 2015-10-02 04:20:10 EDT
Created attachment 1079385 [details]
vdsm.log

[ INFO  ] Stage: Transaction setup
[ INFO  ] Stage: Misc configuration
[ INFO  ] Stage: Package installation
[ INFO  ] Stage: Misc configuration
[ INFO  ] Configuring libvirt
[ INFO  ] Configuring VDSM
[ INFO  ] Starting vdsmd
[ ERROR ] Failed to execute stage 'Misc configuration': Failed to start service 'vdsmd'
[ INFO  ] Stage: Clean up
[ INFO  ] Generating answer file '/var/lib/ovirt-hosted-engine-setup/answers/answers-20151002051315.conf'
[ INFO  ] Stage: Pre-termination
[ INFO  ] Stage: Termination

---

Attached relevant logs

---

# rpm -qa |grep ovirt
ovirt-setup-lib-1.0.0-0.0.master.20150812132738.git6a54bc0.el7.centos.noarch
ovirt-engine-appliance-20150802.0-1.el7.centos.noarch
ovirt-vmconsole-host-1.0.0-0.0.master.20150821105434.gite14b2f0.el7.noarch
ovirt-hosted-engine-setup-1.3.0-0.0.master.20150914144133.gitc63047c.el7.centos.noarch
libgovirt-0.3.1-3.el7.x86_64
ovirt-hosted-engine-ha-1.3.0-0.0.master.20150909150556.20150909150548.git9a2bd43.el7.noarch
ovirt-vmconsole-1.0.0-0.0.master.20150821105434.gite14b2f0.el7.noarch
ovirt-release36-001-0.5.beta.noarch
ovirt-host-deploy-1.4.0-0.0.master.20150806005708.git670e9c8.el7.noarch
ovirt-engine-sdk-python-3.6.0.2-1.el7.centos.noarch
Comment 1 Christopher Pereira 2015-10-02 04:21 EDT
Created attachment 1079386 [details]
ovirt-hosted-engine-setup-20151002051145-j0j7tn.log
Comment 2 Sandro Bonazzola 2015-10-02 04:45:39 EDT
Can you please attach full sos report?
Comment 3 Christopher Pereira 2015-10-02 10:33 EDT
Created attachment 1079462 [details]
sosreport
Comment 4 Christopher Pereira 2015-10-04 00:06:39 EDT
VDSMd receives SIGTERM 15 (kill) during hosted-engine setup.
CGROUP related?
Comment 5 Sandro Bonazzola 2015-10-07 11:27:14 EDT
2015-10-02 05:13:15 DEBUG otopi.plugins.otopi.services.systemd plugin.execute:941 execute-output: ('/bin/systemctl', 'start', 'vdsmd.service') stderr:
A dependency job for vdsmd.service failed. See 'journalctl -xn' for details.

SuperVDSM started correctly:
MainThread::DEBUG::2015-10-02 05:13:15,470::supervdsmServer::520::SuperVdsm.Server::(main) Making sure I'm root - SuperVdsm

But VDSM didn't start.
Looking at /var/log/messages and at systemd status looks like the following services failed to start:
mnt-datacenter.mount  loaded failed failed IMATRONIX DataCenter Mount
ix-datacenter.service loaded failed failed IMATRONIX DataCenter

mom process stopped earlier and never restart:
2015-10-02 03:07:04,596 - mom - INFO - MOM ending

In /var/log/messages I see:
Oct  2 04:01:13 h8 mount: Mount failed. Please check the log file for more details.
Oct  2 04:01:13 h8 systemd: Failed at step EXEC spawning /mnt/datacenter/bin/init-datacenter: Transport endpoint is not connected
Oct  2 04:01:13 h8 systemd: mnt-datacenter.mount mount process exited, code=exited status=1
Oct  2 04:01:13 h8 systemd: Unit mnt-datacenter.mount entered failed state.
Oct  2 04:01:13 h8 systemd: ix-datacenter.service: main process exited, code=exited, status=203/EXEC
Oct  2 04:01:13 h8 systemd: Failed to start IMATRONIX DataCenter.
Oct  2 04:01:13 h8 systemd: Dependency failed for Virtual Desktop Server Manager.
Oct  2 04:01:13 h8 systemd: Dependency failed for MOM instance configured for VDSM purposes.
Oct  2 04:01:13 h8 systemd: 
Oct  2 04:01:13 h8 systemd: 


I think that your disk mount failure is causing the vdsm dependency failure here.
Can you fix the monut issue and retry?
I'm moving this to VDSM component, allowing them to review the issue as well.
Comment 6 Christopher Pereira 2015-10-07 12:08:03 EDT
Could be, but the strange thing is that starting VDSMD via "systemctl start vdsmd" works fine, even when the other service is throwing an error because of the mount.

Closing for now as "NOTABUG", until I can confirm on another host (this one is in production now).

Thanks for your time, Sandro.
Comment 7 Sandro Bonazzola 2015-10-09 06:09:14 EDT
You're welcome Christopher.

Note You need to log in before you can comment on or make changes to this bug.