+++ This bug was initially created as a clone of Bug #1293617 +++
Description of problem:
[RHEV-H] - There is no execute permissions for other under /var/run/vdsm/netconf/ in rhev-h 7.2 with vdsm-4.16.31-1 and because of that, server can't be added to engine 3.5.7.
Calling 'Host.getCapabilities' failed
[root@orchid-vds2 vdsm]# ls -ld /var/run/vdsm/netconf/nets/
drwxrwx---. 2 root root 60 Dec 22 12:16 /var/run/vdsm/netconf/nets/
[root@orchid-vds2 vdsm]# ls -l /var/run/vdsm/netconf/nets/
total 4
-rw-rw----. 1 root root 138 Dec 22 12:16 rhevm
[root@orchid-vds2 ~]# tail -f /var/log/vdsm/vdsm.log
File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 501, in _serveRequest
File "/usr/share/vdsm/rpc/Bridge.py", line 271, in _dynamicMethod
File "/usr/share/vdsm/API.py", line 1266, in getCapabilities
File "/usr/share/vdsm/caps.py", line 602, in get
File "/usr/lib/python2.7/site-packages/vdsm/netinfo.py", line 757, in get
File "/usr/lib/python2.7/site-packages/vdsm/netinfo.py", line 711, in _libvirtNets2vdsm
File "/usr/lib/python2.7/site-packages/vdsm/netconfpersistence.py", line 438, in __init__
File "/usr/lib/python2.7/site-packages/vdsm/netconfpersistence.py", line 113, in __init__
File "/usr/lib/python2.7/site-packages/vdsm/netconfpersistence.py", line 137, in _getConfigs
OSError: [Errno 13] Permission denied: '/var/run/vdsm/netconf/nets/'
Version-Release number of selected component (if applicable):
ovirt-node-3.2.3-30.el7.noarch
vdsm-4.16.31-1.el7ev.x86_64
3.5.7-0.1.el6ev
How reproducible:
100
Steps to Reproduce:
1. Install clean Red Hat Enterprise Virtualization Hypervisor release 7.2 (20151218.1.el7ev)
2. Configure NIC with dhcp via TUI and register to engine 3.5.7
3. Approve server
Actual results:
Calling 'Host.getCapabilities' failed
OSError: [Errno 13] Permission denied: '/var/run/vdsm/netconf/nets/'
Expected results:
Should work as expected
--- Additional comment from RHEL Product and Program Management on 2015-12-22 19:04:04 IST ---
This request has been proposed as a blocker, but a release flag has
not been requested. Please set a release flag to ? to ensure we may
track this bug against the appropriate upcoming release, and reset
the blocker flag to ?.
--- Additional comment from shaochen on 2015-12-23 07:59:39 IST ---
I can reproduce this bug.
Test version:
rhev-hypervisor7-7.2-20151218.1
ovirt-node-3.2.3-30.el7.noarch
rhevm-3.5.7-0.1.el6ev
Test steps:
1. Install clean Red Hat Enterprise Virtualization Hypervisor release 7.2 (20151218.1.el7ev)
2. Configure NIC with dhcp via TUI and register to engine 3.5.7
3. Approve server
Test result:
Failed to up RHEV-H on RHEV-M side.
--- Additional comment from shaochen on 2015-12-23 09:07:17 IST ---
The same issue occurs on rhev-hypervisor6-6.7-20151218.1 (ovirt-node-3.2.3-30.el6.noarch) build.
--- Additional comment from shaochen on 2015-12-23 09:33:00 IST ---
(In reply to shaochen from comment #3)
> The same issue occurs on rhev-hypervisor6-6.7-20151218.1
> (ovirt-node-3.2.3-30.el6.noarch) build.
Add RHEV-H from RHEV-M side can succeed and RHEV-H can up.
--- Additional comment from Fabian Deutsch on 2015-12-23 11:46:25 IST ---
I am not sure why this is popping up in 3.5.7, but I am quite sure that this /var/run dir is created by vdsm itself:
From lib/vdsm/netconfpersistence.py
@staticmethod
def _setConfig(config, path):
dirPath = os.path.dirname(path)
try:
os.makedirs(dirPath) <-- HERE
except OSError as ose:
if errno.EEXIST != ose.errno:
raise
with open(path, 'w') as configurationFile:
json.dump(config, configurationFile)
Ido, am I understanding the code correct and shouldn't the permissions be set here?
--- Additional comment from Ido Barkan on 2015-12-23 12:04:39 IST ---
this is called in a lazy manner- meaning- only when something is indeed being written in the persistence directory. IIUC the problem here is to read the contents of the directory ('x' permission on the dir) and this should happen when VDSM is installed.
--- Additional comment from Fabian Deutsch on 2015-12-23 12:15:39 IST ---
Right - Let me elaborate, on RHEV-H 3.5.7 I see:
[root@dhcp-10-46 admin]# cat /etc/default/version
PRODUCT='Red Hat Enterprise Virtualization Hypervisor'
PRODUCT_SHORT='RHEV-H'
PRODUCT_CODE=rhevh
RECIPE_SHA256=8cb6d13d4e004993f75d1890e4475beb8957f272df4efd3757b8fc9191e24340
RECIPE_RPM=ovirt-node-plugin-vdsm-recipe-0.2.0-26.el7ev.noarch
PACKAGE=rhev-hypervisor
VERSION=7.2
RELEASE=20151218.1.el7ev
[root@dhcp-10-46 admin]# findmnt /var/run/
TARGET SOURCE FSTYPE OPTIONS
/run tmpfs tmpfs rw,nosuid,nodev,seclabel,mode=755
We are speaking about a problem reading from a file in /var/run - on RHEV-H /var/run is a tmpfs (I need to check RHEL-H), thus everything in /var/run needs to be created at runtime.
This indicates that the lazy path you mentioned above might be triggered.
[root@dhcp-10-46 admin]# find /var/run/vdsm -ls
22238 0 drwxr-xr-x 5 vdsm kvm 160 Dec 23 03:09 /var/run/vdsm
45964 0 drwxr-xr-x 2 vdsm kvm 60 Dec 23 03:09 /var/run/vdsm/lvm
45965 4 -rw-r--r-- 1 vdsm kvm 309 Dec 23 03:09 /var/run/vdsm/lvm/lvm.conf
51455 0 -rw-r--r-- 1 vdsm kvm 0 Dec 23 03:26 /var/run/vdsm/client.log
43538 0 -rw-r--r-- 1 root root 0 Dec 23 03:09 /var/run/vdsm/nets_restored
43514 0 srwxr-xr-x 1 vdsm kvm 0 Dec 23 03:09 /var/run/vdsm/svdsm.sock
22284 0 drwxr-xr-x 2 vdsm kvm 40 Dec 23 03:09 /var/run/vdsm/trackedInterfaces
22239 0 drwxr-xr-x 2 vdsm kvm 40 Dec 23 05:40 /var/run/vdsm/sourceRoutes
The snippet above shows the state before adding the host to RHEV-M - it shows that the netconf persistence dir is not there.
The following snippet shows the state after adding the host:
[root@dhcp-10-46 admin]# find /var/run/vdsm -ls
22238 0 drwxr-xr-x 6 vdsm kvm 180 Dec 23 2015 /var/run/vdsm
289628 0 drwxrwx--- 3 root root 60 Dec 23 2015 /var/run/vdsm/netconf
289629 0 drwxrwx--- 2 root root 60 Dec 23 2015 /var/run/vdsm/netconf/nets
289630 4 -rw-rw---- 1 root root 135 Dec 23 2015 /var/run/vdsm/netconf/nets/rhevm
45964 0 drwxr-xr-x 2 vdsm kvm 60 Dec 23 2015 /var/run/vdsm/lvm
45965 4 -rw-r--r-- 1 vdsm kvm 309 Dec 23 2015 /var/run/vdsm/lvm/lvm.conf
51455 0 -rw-r--r-- 1 vdsm kvm 0 Dec 23 2015 /var/run/vdsm/client.log
43538 0 -rw-r--r-- 1 root root 0 Dec 23 2015 /var/run/vdsm/nets_restored
43514 0 srwxr-xr-x 1 vdsm kvm 0 Dec 23 2015 /var/run/vdsm/svdsm.sock
22284 0 drwxr-xr-x 2 vdsm kvm 40 Dec 23 2015 /var/run/vdsm/trackedInterfaces
22239 0 drwxr-xr-x 2 vdsm kvm 40 Dec 23 2015 /var/run/vdsm/sourceRoutes
This shows that the /var/run/vdsm/netconf path was created during the add host flow. And I suppose that saving the network config (which triggers the lazy dir creation path) was part of this.
--- Additional comment from Fabian Deutsch on 2015-12-23 12:20:52 IST ---
Michael, can you try to reproduce this on RHEL 7.2 with vdsm from 3.5.7?
--- Additional comment from Michael Burman on 2015-12-23 13:17:40 IST ---
Working as expected with rhel 7.2(RHEL - 7.2 - 9.el7_2.1)
and vdsm 3.5.7 (vdsm-4.16.31-1.el7ev)
kernel version 3.10.0 - 327.4.4.el7.x86_64
--- Additional comment from Dan Kenigsberg on 2015-12-23 14:19:34 IST ---
I suppose that adding an explicit
os.chown(dirName, vdsm_uid, 0)
to lib/vdsm/netconfpersistence.py's _setConfig would hide this surprise change in umask.
--- Additional comment from Red Hat Bugzilla Rules Engine on 2015-12-23 15:17:54 IST ---
This bug is not marked for z-stream, yet the milestone is for a z-stream version, therefore the milestone has been reset.
Please set the correct milestone or add the z-stream flag.
--- Additional comment from Red Hat Bugzilla Rules Engine on 2015-12-23 15:27:29 IST ---
This bug is not marked for z-stream, yet the milestone is for a z-stream version, therefore the milestone has been reset.
Please set the correct milestone or add the z-stream flag.
--- Additional comment from Red Hat Bugzilla Rules Engine on 2015-12-23 15:28:08 IST ---
This bug is not marked for z-stream, yet the milestone is for a z-stream version, therefore the milestone has been reset.
Please set the correct milestone or add the z-stream flag.
Test version:
rhev-hypervisor7-7.2-20151229.1
ovirt-node-3.2.3-30.el7.noarch
vdsm-4.16.32-1.el7ev.x86_64
rhevm-3.5.7-0.1.el6ev
Test steps:
1. Install clean rhev-hypervisor7-7.2-20151229.1
2. Configure NIC with dhcp via TUI and register to engine 3.5.7
3. Approve server
Test result:
Register can succeed and RHEV-H can up.
Verified on - vdsm-4.16.32-1.el7ev.x86_64
3.5.7-0.1.el6ev
ovirt-node-3.2.3-30.el7.noarch
Red Hat Enterprise Virtualization Hypervisor release 7.2 (20151229.1.el7ev)
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory, and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://rhn.redhat.com/errata/RHBA-2016-0034.html