Bug 1120049 - RHEV-H: vdsm not starting due to connectivity.log root ownership
Summary: RHEV-H: vdsm not starting due to connectivity.log root ownership
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Red Hat Enterprise Virtualization Manager
Classification: Red Hat
Component: vdsm
Version: 3.4.0
Hardware: Unspecified
OS: Unspecified
urgent
urgent
Target Milestone: ---
: 3.5.0
Assignee: Douglas Schilling Landgraf
QA Contact: Martin Pavlik
URL:
Whiteboard: network
Depends On:
Blocks: rhev3.5beta 1156165
TreeView+ depends on / blocked
 
Reported: 2014-07-16 07:32 UTC by cshao
Modified: 2016-02-10 19:53 UTC (History)
23 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2015-02-16 13:39:59 UTC
oVirt Team: Network
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)
vdsm-reg.log (33.53 KB, text/plain)
2014-07-16 07:32 UTC, cshao
no flags Details
engine.log (90.49 KB, text/plain)
2014-07-16 07:33 UTC, cshao
no flags Details
supervdsm.log (29.36 KB, text/plain)
2014-07-17 12:58 UTC, Ying Cui
no flags Details


Links
System ID Private Priority Status Summary Last Updated
oVirt gerrit 30507 0 master MERGED connectivity log: add to spec with proper ownership Never
oVirt gerrit 30558 0 ovirt-3.5 MERGED connectivity log: add to spec with proper ownership Never

Description cshao 2014-07-16 07:32:14 UTC
Created attachment 918336 [details]
vdsm-reg.log

Description of problem:
Register RHEV-H to RHEV-M av10.1 & is37 failed.
Network error during communication with the host.

======================================================
libvir: Network Driver error : Network not found: no network with matching name 'vdsm-rhevm'
libvir: Network Driver error : Network not found: no network with matching name 'vdsm-rhevm'

Version-Release number of selected component (if applicable):
rhev-hypervisor6-6.5-20140715.0
ovirt-node-3.0.1-18.el6_5.13.noarch
vdsm-4.14.7-7.el6ev.x86_64
vdsm-reg-4.14.7-7.el6ev.noarch
RHEVM av10.1
RHEVM is37

How reproducible:
100%

Steps to Reproduce:
1. Install rhev-hypervisor6-6.5-20140715.0
2. Register to RHEVM.
3.

Actual results:
Register RHEV-H to RHEV-M av10.1& is37 failed. Network error during communication with the host.

Expected results:
Register to RHEVM can successful.

Additional info:
Add RHEV-H from RHEV-M side can successful.

Comment 1 cshao 2014-07-16 07:33:23 UTC
Created attachment 918337 [details]
engine.log

Comment 2 cshao 2014-07-16 07:35:57 UTC
1. No such issue with rhev-hypervisor6-6.5-20140624.0.el6ev build, so add keyword regression.
2. This bug block our testing, so add keyword testblodker.

Comment 6 Lior Vernia 2014-07-17 12:28:57 UTC
Could we get relevant VDSM log as well?

Comment 7 Ying Cui 2014-07-17 12:29:51 UTC
(In reply to shaochen from comment #2)
> 1. No such issue with rhev-hypervisor6-6.5-20140624.0.el6ev build, so add
> keyword regression.
> 2. This bug block our testing, so add keyword testblodker.

No such issue with rhev-hypervisor6-6.5-20140624.0.el6ev build with vdsm-4.14.7-3.el6ev.x86_64

Comment 8 Ying Cui 2014-07-17 12:57:07 UTC
register failed, vdsmd service is not running, there is no vdsm.log, provide supervdsm.log in this bug.

Comment 9 Ying Cui 2014-07-17 12:58:23 UTC
Created attachment 918698 [details]
supervdsm.log

Comment 10 Dan Kenigsberg 2014-07-17 14:22:04 UTC
Vdsm refuses to start reminds me of recent cpopen and python-pthreading bugs. Which versions of them do you have installed?

Comment 11 Ying Cui 2014-07-18 02:58:11 UTC
(In reply to Dan Kenigsberg from comment #10)
> Vdsm refuses to start reminds me of recent cpopen and python-pthreading
> bugs. Which versions of them do you have installed?

python-cpopen-1.3-2.el6_5.x86_64
python-pthreading-0.1.3-3.el6ev.noarch

Comment 12 Dan Kenigsberg 2014-07-20 20:29:52 UTC
Ying Cui, does vdsm start up when you try to start it explicitly? If not, could you log into the node as root, and run

# su - vdsm -s /bin/bash
$ /usr/share/vdsm/vdsm

this may provide more hints regarding the failure.

Comment 13 Ying Cui 2014-07-21 02:57:28 UTC
Try to start vdsmd, but it is still not running.
# /etc/init.d/vdsmd start 
vdsm: already running                                      [  OK  ]
vdsm start                                                 [  OK  ]
# /etc/init.d/vdsmd status
VDS daemon is not running, and its watchdog is running

# su - vdsm -s /bin/bash
$ /usr/share/vdsm/vdsm
Traceback (most recent call last):
  File "/usr/share/vdsm/vdsm", line 218, in <module>
    run(**argDict)
  File "/usr/share/vdsm/vdsm", line 92, in run
    lconfig.fileConfig(loggerConfFile, disable_existing_loggers=False)
  File "/usr/lib64/python2.6/logging/config.py", line 84, in fileConfig
  File "/usr/lib64/python2.6/logging/config.py", line 162, in _install_handlers
  File "/usr/lib64/python2.6/logging/handlers.py", line 377, in __init__
  File "/usr/lib64/python2.6/logging/__init__.py", line 827, in __init__
  File "/usr/lib64/python2.6/logging/__init__.py", line 846, in _open
IOError: [Errno 13] Permission denied: '/var/log/vdsm/connectivity.log'

$ ll /var/log/vdsm/connectivity.log 
-rw-r--r--. 1 root root 0 2014-07-16 06:31 /var/log/vdsm/connectivity.log

Comment 14 Ying Cui 2014-07-21 08:27:37 UTC
Virt QE helps to qa_ack+ this bug.

Comment 15 Eyal Edri 2014-07-22 05:41:02 UTC
danken - can we move this to MODIFIED since we reverted the change?

2 commits were reverted:
7a743d1 - connectivty log: more readable changes
5f0f1e6 - introduce connectivity 

or open another bug to verify the new fix?

Comment 16 Dan Kenigsberg 2014-07-22 07:38:03 UTC
We should fix this properly on 3.5 (with the linked patch), so I'm changing the released version instead.

Comment 17 Douglas Schilling Landgraf 2014-07-23 14:06:50 UTC
Moving to MODIFIED since next rebase should include this fix.

Comment 19 cshao 2014-07-24 05:55:44 UTC
Test version:
rhev-hypervisor6-6.5-20140723.0.el6ev
vdsm-4.14.11-5.el6ev.x86_64
ovirt-node-3.0.1-18.el6.14.noarch
RHEVM av10.5

Test version:
1. Install rhev-hypervisor6-6.5-20140715.0
2. Register to RHEVM.
3. Add RHEV-H from RHEV-M side.
4. ll /var/log/vdsm/connectivity.log 

Test result:
1. Both "register" and "add host from rhevm side" can successful.
2. But there is no connectivity.log under /var/log/vdsm path.
# ll /var/log/vdsm
total 7284
drwxr-xr-x. 2 vdsm kvm     4096 2014-07-24 02:32 backup
-rw-r--r--. 1 vdsm kvm        0 2014-07-24 02:56 metadata.log
-rw-r--r--. 1 vdsm kvm     1395 2014-07-24 05:50 mom.log
-rw-r--r--. 1 vdsm kvm     2184 2014-07-24 02:58 spm-lock.log
-rw-r--r--. 1 root root   18248 2014-07-24 05:50 supervdsm.log
-rw-r--r--. 1 vdsm kvm  7420427 2014-07-24 05:52 vdsm.log

Comment 20 Ying Cui 2014-07-24 07:09:20 UTC
This bug need to clone to rhev 3.4.1, then we will verify this bug on 3.4.1 build.

Comment 21 Martin Pavlik 2014-08-07 13:58:31 UTC
works with 
oVirt Node Hypervisor release 3.1.0_master (20140805.0) (Edited, Unsigned)

[root@dell-r210ii-06 ~]# ll /var/log/vdsm/connectivity.log 
-rw-r--r--. 1 vdsm kvm 472 2014-08-07 13:55 /var/log/vdsm/connectivity.log


Note You need to log in before you can comment on or make changes to this bug.