Bug 1560460
| Summary: | getFileStats fails on NFS domain in case or recursive symbolic link (e.g., using NetApp snapshots) | |||
|---|---|---|---|---|
| Product: | Red Hat Enterprise Virtualization Manager | Reporter: | Roman Hodain <rhodain> | |
| Component: | vdsm | Assignee: | Nir Soffer <nsoffer> | |
| Status: | CLOSED ERRATA | QA Contact: | Kevin Alon Goldblatt <kgoldbla> | |
| Severity: | medium | Docs Contact: | ||
| Priority: | medium | |||
| Version: | 4.1.9 | CC: | ebenahar, lsurette, nsoffer, rdlugyhe, rhodain, srevivo, tnisan, ycui | |
| Target Milestone: | ovirt-4.3.0 | Keywords: | ZStream | |
| Target Release: | 4.3.0 | Flags: | lsvaty:
testing_plan_complete-
|
|
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | v4.30.3 | Doc Type: | Bug Fix | |
| Doc Text: |
Previously, VDSM used stat() to implement islink() checks when using ioprocess to run commands. As a result, if a user or storage system created a recursive symbolic link inside the ISO storage domain, VDSM failed to report file information. In the current release, VDSM uses lstat() to implement islink() so it can report file information from recursive symbolic links.
|
Story Points: | --- | |
| Clone Of: | ||||
| : | 1612904 (view as bug list) | Environment: | ||
| Last Closed: | 2019-05-08 12:36:02 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | Storage | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | 1564515 | |||
| Bug Blocks: | 1612904 | |||
Roman, can you share output of "ls -lhR" in /rhev/data-center/mnt/server:_path for a faulty server using these recursive symlinks? We don't have enough data to understand how a recursive link was created inside a the iso domain images directory, but checking the source show that we indeed implement os.path.islink() in the wrong way in vdsm. Fixing this requires adding lstat() support in ioprocess, see bug 1564515. WARN: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:
[Found non-acked flags: '{'rhevm-4.2.z': '?'}', ]
For more info please contact: rhv-devops: Bug status wasn't changed from MODIFIED to ON_QA due to the following reason:
[Found non-acked flags: '{'rhevm-4.2.z': '?'}', ]
For more info please contact: rhv-devops
Verify according to https://bugzilla.redhat.com/show_bug.cgi?id=1612904#c13 Verified with the following code:
----------------------------------------
ovirt-engine-4.3.0-0.0.master.20181012165724.gitd25f971.el7.noarch
vdsm-4.30.0-640.git6fd8327.el7.x86_64
Verified with the following scenario:
----------------------------------------
Steps to Reproduce:
1. Go to NFS ISO domain and create a recursive symbolic link.
# pwd
/rhev/data-center/mnt/10.34.1.195:_exports_rheviso01/c217bcfa-ffba-4982-b2c3-4ed418a0cd1d/images/11111111-1111-1111-1111-111111111111
# ln -s ../ testLink
2. Try to get the list of isos in the WebAdmin >>>>> Works fine and no errors are reported
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:1077 sync2jira sync2jira |
Description of problem: NetApp snapshots feature may include symbolic links to NFS share which points to the same directory. This causes getFileStats to fail. Example: /rhev/data-center/mnt/<mountPoint>/.snapshot/daily.2018-02-20_0010 -> /rhev/data-center/mnt/<mountPoint>/<SD_UUID>/images/11111111-1111-1111-1111-111111111111/ Version-Release number of selected component (if applicable): python-ioprocess-0.16.1-1.el7ev.noarch ioprocess-0.16.1-1.el7ev.x86_64 How reproducible: 100% Steps to Reproduce: 1. Go to NFS ISO domain and create a recursive symbolic link. # pwd /rhev/data-center/mnt/10.34.1.195:_exports_rheviso01/c217bcfa-ffba-4982-b2c3-4ed418a0cd1d/images/11111111-1111-1111-1111-111111111111 # ln -s ../ testLink 2. Try to get the list of isos in the WebAdmin Actual results: 2018-03-26 10:08:03,596+0200 ERROR (jsonrpc/6) [storage.TaskManager.Task] (Task='70ba66b9-dd38-4042-9baa -d390a8c44305') Unexpected error (task:872) Traceback (most recent call last): File "/usr/share/vdsm/storage/task.py", line 879, in _run return fn(*args, **kargs) File "<string>", line 2, in getFileStats File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/share/vdsm/storage/hsm.py", line 2305, in getFileStats caseSensitive=caseSensitive) File "/usr/share/vdsm/storage/fileSD.py", line 496, in getFileList filesList = self.oop.simpleWalk(basedir) File "/usr/lib/python2.7/site-packages/vdsm/storage/outOfProcess.py", line 351, in simpleWalk files.extend(simpleWalk(ioproc, fullpath)) File "/usr/lib/python2.7/site-packages/vdsm/storage/outOfProcess.py", line 351, in simpleWalk files.extend(simpleWalk(ioproc, fullpath)) File "/usr/lib/python2.7/site-packages/vdsm/storage/outOfProcess.py", line 351, in simpleWalk files.extend(simpleWalk(ioproc, fullpath)) .... File "/usr/lib/python2.7/site-packages/vdsm/storage/outOfProcess.py", line 255, in isdir res = self._iop.stat(path) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 484, in stat resdict = self._sendCommand("stat", {"path": path}, self.timeout) File "/usr/lib/python2.7/site-packages/ioprocess/__init__.py", line 461, in _sendCommand raise OSError(errcode, errstr) OSError: [Errno 40] Too many levels of symbolic links .... 2018-03-26 10:08:03,600+0200 ERROR (jsonrpc/6) [storage.Dispatcher] FINISH getFileStats error=[Errno 40] Too many levels of symbolic links (dispatcher:85) Traceback (most recent call last): File "/usr/share/vdsm/storage/dispatcher.py", line 72, in wrapper result = ctask.prepare(func, *args, **kwargs) File "/usr/share/vdsm/storage/task.py", line 105, in wrapper return m(self, *a, **kw) File "/usr/share/vdsm/storage/task.py", line 1185, in prepare raise self.error OSError: [Errno 40] Too many levels of symbolic links Expected results: The symbolic link is skipped. Additional info: ioprocess tries to skip the symlinks: def simpleWalk(ioproc, path): files = [] for f in ioproc.listdir(path): fullpath = os.path.join(path, f) osPath = _IOProcessOs(ioproc).path if osPath.isdir(fullpath) and not osPath.islink(fullpath): files.extend(simpleWalk(ioproc, fullpath)) else: files.append(fullpath) return files The problem is that the islink function uses "stat" function from ioprocess c library which does use a stat from sys/stat.h which returns information about the target not about the link itself. So lstat function should be used instead.