| Summary: | cannot attach iso domain to engine 3.6.6. | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | [oVirt] ovirt-engine | Reporter: | Sven Kieske <s.kieske> | ||||||||||
| Component: | BLL.Storage | Assignee: | Nir Soffer <nsoffer> | ||||||||||
| Status: | CLOSED NOTABUG | QA Contact: | Aharon Canan <acanan> | ||||||||||
| Severity: | urgent | Docs Contact: | |||||||||||
| Priority: | unspecified | ||||||||||||
| Version: | 3.6.5.1 | CC: | amureini, bugs, nsoffer, s.kieske | ||||||||||
| Target Milestone: | ovirt-3.6.6 | Keywords: | Regression | ||||||||||
| Target Release: | --- | Flags: | amureini:
ovirt-3.6.z?
rule-engine: planning_ack? rule-engine: devel_ack? rule-engine: testing_ack? |
||||||||||
| Hardware: | x86_64 | ||||||||||||
| OS: | Linux | ||||||||||||
| Whiteboard: | |||||||||||||
| Fixed In Version: | Doc Type: | Bug Fix | |||||||||||
| Doc Text: | Story Points: | --- | |||||||||||
| Clone Of: | Environment: | ||||||||||||
| Last Closed: | 2016-04-29 07:03:21 UTC | Type: | Bug | ||||||||||
| Regression: | --- | Mount Type: | --- | ||||||||||
| Documentation: | --- | CRM: | |||||||||||
| Verified Versions: | Category: | --- | |||||||||||
| oVirt Team: | Storage | RHEL 7.3 requirements from Atomic Host: | |||||||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
| Attachments: |
|
||||||||||||
|
Description
Sven Kieske
2016-04-27 14:44:09 UTC
Created attachment 1151425 [details]
engine.log
I also did run the nfs-check.py from vdsm/contrib repo described here: https://www.ovirt.org/documentation/how-to/troubleshooting/troubleshooting-nfs-storage-issues/ result: python ./nfs-check.py 10.210.1.6:/home/iso1 Current hostname: vrnode0015.vrootdev - IP addr 10.210.2.20 Trying to /bin/mount -t nfs 10.210.1.6:/home/iso1... Executing NFS tests.. Removing vdsmTest file.. Status of tests [OK] Disconnecting from NFS Server.. Done! So I really don't know what's the problem. Maybe this is related to BZ 1320128 ? There seem to be some network issues in between Connections from engine to host (in engine and/or vdsm code, nothing related to the infrastructure). kind regards Sven Relevant error from VDSM:
jsonrpc.Executor/2::DEBUG::2016-04-27 16:28:55,378::clusterlock::140::Storage.Misc.excCmd::(acquire) /usr/bin/taskset --cpu-list 0-47 /usr/bin/sudo -n /usr/bin/setsid /usr/bin/ionice -c 1 -n 0 /usr/bin/su vdsm -s /bin/sh -c '/usr/libexec/vdsm/spmprotect.sh start 9ba22bfe-d760-48c9-b4a5-10149cb4ecb9 1 5 /rhev/data-center/mnt/10.210.1.6:_home_iso1/9ba22bfe-d760-48c9-b4a5-10149cb4ecb9/dom_md/leases 60000 10000 3 17433' (cwd /usr/libexec/vdsm)
jsonrpc.Executor/2::DEBUG::2016-04-27 16:28:55,387::clusterlock::140::Storage.Misc.excCmd::(acquire) FAILED: <err> = 'sudo: a password is required\n'; <rc> = 1
jsonrpc.Executor/2::ERROR::2016-04-27 16:28:55,387::task::866::Storage.TaskManager.Task::(_setError) Task=`9cf18f8d-c6a8-4e4a-ac8e-d9c4d786d3ae`::Unexpected error
Traceback (most recent call last):
File "/usr/share/vdsm/storage/task.py", line 873, in _run
return fn(*args, **kargs)
File "/usr/share/vdsm/logUtils.py", line 49, in wrapper
res = f(*args, **kwargs)
File "/usr/share/vdsm/storage/hsm.py", line 1210, in attachStorageDomain
pool.attachSD(sdUUID)
File "/usr/share/vdsm/storage/securable.py", line 77, in wrapper
return method(self, *args, **kwargs)
File "/usr/share/vdsm/storage/sp.py", line 943, in attachSD
dom.acquireClusterLock(self.id)
File "/usr/share/vdsm/storage/sd.py", line 565, in acquireClusterLock
self._clusterLock.acquire(hostID)
File "/usr/share/vdsm/storage/clusterlock.py", line 142, in acquire
raise se.AcquireLockFailure(self._sdUUID, rc, out, err)
AcquireLockFailure: Cannot obtain lock: u"id=9ba22bfe-d760-48c9-b4a5-10149cb4ecb9, rc=1, out=[], err=['sudo: a password is required']"
Can you please share your /etc/sudoers.d/50_vdsm file?
And the /etc/sudoers file too please, actually. Created attachment 1151722 [details]
/etc/sudoers from affected host running vdsm
This is the stock centos 7.2 sudoers file, I did not alter it
Created attachment 1151735 [details]
50_vdsm sudoer file, actually altered by me
I added the section Cmnd_Alias VDSM_NETWORK because I run a hook which uses "tc" for advanced traffic control of vms.
I hope this isn't problematic, this works with older vdsm and ovirt-engine versions.
Seems to be in order [unfortunately :-(]. Nir - can you take a look? Bug tickets must have version flags set prior to targeting them to a release. Please ask maintainer to set the correct version flags and only then set the target milestone. Sven,
In your vdsm log, we see that vdsm ask to run this command with sudo:
/usr/bin/setsid /usr/bin/ionice -c 1 -n 0 /usr/bin/su vdsm -s /bin/sh -c '/usr/libexec/vdsm/spmprotect.sh ...
But your /etc/sudoers.d/50_vdsm will allow only:
/usr/bin/setsid /usr/bin/ionice -c ? -n ? /bin/su vdsm -s /bin/sh -c /usr/libexec/vdsm/spmprotect.sh
Note "/bin/su" vs "/usr/bin/su"
Can you share the output of:
grep 'EXT_SU\b' /usr/lib/python2.7/site-packages/vdsm/constants.py
Did you modify the 50_vdsm file installed by vdsm, or copied from another machine?
I don't think modifying files installed by vdsm is a good idea. Your file
will be overwritten once you upgrade vdsm.
I think you want to put your private sudoers configuration in another file like:
/etc/sudoers.d/51_vdsm_hook_name
The contents would some somthing like this:
vdsm ALL=(ALL) NOPASSWD: /sbin/tc *
(In reply to Nir Soffer from comment #9) > Sven, > > In your vdsm log, we see that vdsm ask to run this command with sudo: > > /usr/bin/setsid /usr/bin/ionice -c 1 -n 0 /usr/bin/su vdsm -s /bin/sh -c > '/usr/libexec/vdsm/spmprotect.sh ... > > But your /etc/sudoers.d/50_vdsm will allow only: > > /usr/bin/setsid /usr/bin/ionice -c ? -n ? /bin/su vdsm -s /bin/sh -c > /usr/libexec/vdsm/spmprotect.sh > > Note "/bin/su" vs "/usr/bin/su" > > Can you share the output of: > > grep 'EXT_SU\b' /usr/lib/python2.7/site-packages/vdsm/constants.py > > Did you modify the 50_vdsm file installed by vdsm, or copied from another > machine? In fact I just checked the config management (I do basic setup with cobbler and later configuration via salt): It seems my colleagues found it a good idea to overwrite the whole vdsm config with an old one from vdsm-4.13.4-0, from EL6 machines hence the error. > I don't think modifying files installed by vdsm is a good idea. Your file > will be overwritten once you upgrade vdsm. > > I think you want to put your private sudoers configuration in another file > like: > > /etc/sudoers.d/51_vdsm_hook_name > > The contents would some somthing like this: > > vdsm ALL=(ALL) NOPASSWD: /sbin/tc * I will adapt your suggestion and put my own config in my own file and not fiddle with vdsm's sudoers file anymore. In fact on EL7 su is located under /usr/bin/su and tc is located under /usr/sbin/tc and the stock 50_vdsm sudoers file has the correct path to "su" in it, I just checked. I just reinstalled the host with your suggested changes to my sudoers files (taking stock 50_vdsm sudoers and my changes in a separate file) and I can confirm that iso domain attaching works again. Thanks for your great help and support, it's really appreciated! :) And sorry for all the noise, as this was apparently a configuration mistake on my side from the very beginning. kind regards Sven |