Created attachment 1108633 [details] Logs Description of problem: [RHEV-H] - There is no execute permissions for other under /var/run/vdsm/netconf/ in rhev-h 7.2 with vdsm-4.16.31-1 and because of that, server can't be added to engine 3.5.7. Calling 'Host.getCapabilities' failed [root@orchid-vds2 vdsm]# ls -ld /var/run/vdsm/netconf/nets/ drwxrwx---. 2 root root 60 Dec 22 12:16 /var/run/vdsm/netconf/nets/ [root@orchid-vds2 vdsm]# ls -l /var/run/vdsm/netconf/nets/ total 4 -rw-rw----. 1 root root 138 Dec 22 12:16 rhevm [root@orchid-vds2 ~]# tail -f /var/log/vdsm/vdsm.log File "/usr/lib/python2.7/site-packages/yajsonrpc/__init__.py", line 501, in _serveRequest File "/usr/share/vdsm/rpc/Bridge.py", line 271, in _dynamicMethod File "/usr/share/vdsm/API.py", line 1266, in getCapabilities File "/usr/share/vdsm/caps.py", line 602, in get File "/usr/lib/python2.7/site-packages/vdsm/netinfo.py", line 757, in get File "/usr/lib/python2.7/site-packages/vdsm/netinfo.py", line 711, in _libvirtNets2vdsm File "/usr/lib/python2.7/site-packages/vdsm/netconfpersistence.py", line 438, in __init__ File "/usr/lib/python2.7/site-packages/vdsm/netconfpersistence.py", line 113, in __init__ File "/usr/lib/python2.7/site-packages/vdsm/netconfpersistence.py", line 137, in _getConfigs OSError: [Errno 13] Permission denied: '/var/run/vdsm/netconf/nets/' Version-Release number of selected component (if applicable): ovirt-node-3.2.3-30.el7.noarch vdsm-4.16.31-1.el7ev.x86_64 3.5.7-0.1.el6ev How reproducible: 100 Steps to Reproduce: 1. Install clean Red Hat Enterprise Virtualization Hypervisor release 7.2 (20151218.1.el7ev) 2. Configure NIC with dhcp via TUI and register to engine 3.5.7 3. Approve server Actual results: Calling 'Host.getCapabilities' failed OSError: [Errno 13] Permission denied: '/var/run/vdsm/netconf/nets/' Expected results: Should work as expected
I can reproduce this bug. Test version: rhev-hypervisor7-7.2-20151218.1 ovirt-node-3.2.3-30.el7.noarch rhevm-3.5.7-0.1.el6ev Test steps: 1. Install clean Red Hat Enterprise Virtualization Hypervisor release 7.2 (20151218.1.el7ev) 2. Configure NIC with dhcp via TUI and register to engine 3.5.7 3. Approve server Test result: Failed to up RHEV-H on RHEV-M side.
The same issue occurs on rhev-hypervisor6-6.7-20151218.1 (ovirt-node-3.2.3-30.el6.noarch) build.
(In reply to shaochen from comment #3) > The same issue occurs on rhev-hypervisor6-6.7-20151218.1 > (ovirt-node-3.2.3-30.el6.noarch) build. Add RHEV-H from RHEV-M side can succeed and RHEV-H can up.
I am not sure why this is popping up in 3.5.7, but I am quite sure that this /var/run dir is created by vdsm itself: From lib/vdsm/netconfpersistence.py @staticmethod def _setConfig(config, path): dirPath = os.path.dirname(path) try: os.makedirs(dirPath) <-- HERE except OSError as ose: if errno.EEXIST != ose.errno: raise with open(path, 'w') as configurationFile: json.dump(config, configurationFile) Ido, am I understanding the code correct and shouldn't the permissions be set here?
this is called in a lazy manner- meaning- only when something is indeed being written in the persistence directory. IIUC the problem here is to read the contents of the directory ('x' permission on the dir) and this should happen when VDSM is installed.
Right - Let me elaborate, on RHEV-H 3.5.7 I see: [root@dhcp-10-46 admin]# cat /etc/default/version PRODUCT='Red Hat Enterprise Virtualization Hypervisor' PRODUCT_SHORT='RHEV-H' PRODUCT_CODE=rhevh RECIPE_SHA256=8cb6d13d4e004993f75d1890e4475beb8957f272df4efd3757b8fc9191e24340 RECIPE_RPM=ovirt-node-plugin-vdsm-recipe-0.2.0-26.el7ev.noarch PACKAGE=rhev-hypervisor VERSION=7.2 RELEASE=20151218.1.el7ev [root@dhcp-10-46 admin]# findmnt /var/run/ TARGET SOURCE FSTYPE OPTIONS /run tmpfs tmpfs rw,nosuid,nodev,seclabel,mode=755 We are speaking about a problem reading from a file in /var/run - on RHEV-H /var/run is a tmpfs (I need to check RHEL-H), thus everything in /var/run needs to be created at runtime. This indicates that the lazy path you mentioned above might be triggered. [root@dhcp-10-46 admin]# find /var/run/vdsm -ls 22238 0 drwxr-xr-x 5 vdsm kvm 160 Dec 23 03:09 /var/run/vdsm 45964 0 drwxr-xr-x 2 vdsm kvm 60 Dec 23 03:09 /var/run/vdsm/lvm 45965 4 -rw-r--r-- 1 vdsm kvm 309 Dec 23 03:09 /var/run/vdsm/lvm/lvm.conf 51455 0 -rw-r--r-- 1 vdsm kvm 0 Dec 23 03:26 /var/run/vdsm/client.log 43538 0 -rw-r--r-- 1 root root 0 Dec 23 03:09 /var/run/vdsm/nets_restored 43514 0 srwxr-xr-x 1 vdsm kvm 0 Dec 23 03:09 /var/run/vdsm/svdsm.sock 22284 0 drwxr-xr-x 2 vdsm kvm 40 Dec 23 03:09 /var/run/vdsm/trackedInterfaces 22239 0 drwxr-xr-x 2 vdsm kvm 40 Dec 23 05:40 /var/run/vdsm/sourceRoutes The snippet above shows the state before adding the host to RHEV-M - it shows that the netconf persistence dir is not there. The following snippet shows the state after adding the host: [root@dhcp-10-46 admin]# find /var/run/vdsm -ls 22238 0 drwxr-xr-x 6 vdsm kvm 180 Dec 23 2015 /var/run/vdsm 289628 0 drwxrwx--- 3 root root 60 Dec 23 2015 /var/run/vdsm/netconf 289629 0 drwxrwx--- 2 root root 60 Dec 23 2015 /var/run/vdsm/netconf/nets 289630 4 -rw-rw---- 1 root root 135 Dec 23 2015 /var/run/vdsm/netconf/nets/rhevm 45964 0 drwxr-xr-x 2 vdsm kvm 60 Dec 23 2015 /var/run/vdsm/lvm 45965 4 -rw-r--r-- 1 vdsm kvm 309 Dec 23 2015 /var/run/vdsm/lvm/lvm.conf 51455 0 -rw-r--r-- 1 vdsm kvm 0 Dec 23 2015 /var/run/vdsm/client.log 43538 0 -rw-r--r-- 1 root root 0 Dec 23 2015 /var/run/vdsm/nets_restored 43514 0 srwxr-xr-x 1 vdsm kvm 0 Dec 23 2015 /var/run/vdsm/svdsm.sock 22284 0 drwxr-xr-x 2 vdsm kvm 40 Dec 23 2015 /var/run/vdsm/trackedInterfaces 22239 0 drwxr-xr-x 2 vdsm kvm 40 Dec 23 2015 /var/run/vdsm/sourceRoutes This shows that the /var/run/vdsm/netconf path was created during the add host flow. And I suppose that saving the network config (which triggers the lazy dir creation path) was part of this.
Michael, can you try to reproduce this on RHEL 7.2 with vdsm from 3.5.7?
Working as expected with rhel 7.2(RHEL - 7.2 - 9.el7_2.1) and vdsm 3.5.7 (vdsm-4.16.31-1.el7ev) kernel version 3.10.0 - 327.4.4.el7.x86_64
I suppose that adding an explicit os.chown(dirName, vdsm_uid, 0) to lib/vdsm/netconfpersistence.py's _setConfig would hide this surprise change in umask.
RHEV build 3.6.2-3 released http://bob.eng.lab.tlv.redhat.com/builds/3.6/3.6.2-3/
Verified on - vdsm-4.17.15-0.el7ev.noarch 3.6.2-0.1.el6 ovirt-node-3.6.1-1.0.el7ev.noarch Red Hat Enterprise Virtualization Hypervisor (Beta) release 7.2 (20151229.0.el7ev)