Bug 1492865
| Summary: | [RHCeph 3.0/ 12.2.0-2.el7cp] test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) | ||
|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Ceph Storage | Reporter: | Vasu Kulkarni <vakulkar> |
| Component: | CephFS | Assignee: | Patrick Donnelly <pdonnell> |
| Status: | CLOSED ERRATA | QA Contact: | Vasu Kulkarni <vakulkar> |
| Severity: | high | Docs Contact: | |
| Priority: | high | ||
| Version: | 3.0 | CC: | ceph-eng-bugs, hnallurv, john.spray, kdreyer |
| Target Milestone: | rc | ||
| Target Release: | 3.0 | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| Whiteboard: | |||
| Fixed In Version: | RHEL: ceph-12.2.1-1.el7cp Ubuntu: ceph_12.2.1-2redhat1xenial | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2017-12-05 23:44:17 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | |||
| Bug Blocks: | 1492889 | ||
Verified in 12.2.1-21.el7cp Full FS regression run on above build is at http://pulpito.ceph.redhat.com/vasu-2017-10-18_22:32:22-fs-luminous---basic-multi 2017-10-19T17:17:34.084 INFO:teuthology.orchestra.run.clara004:Running: 'sudo umount -l -f /home/ubuntu/cephtest/mnt.3' 2017-10-19T17:17:34.154 INFO:teuthology.orchestra.run.clara004.stderr:umount: /home/ubuntu/cephtest/mnt.3: mountpoint not found 2017-10-19T17:17:34.155 INFO:teuthology.orchestra.run.clara004:Running: "stat --file-system '--printf=%T\n' -- /home/ubuntu/cephtest/mnt.3" 2017-10-19T17:17:34.214 INFO:teuthology.orchestra.run.clara004.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2017-10-19T17:17:34.215 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2017-10-19T17:17:34.215 INFO:teuthology.orchestra.run.clara004:Running: "stat --file-system '--printf=%T\n' -- /home/ubuntu/cephtest/mnt.3" 2017-10-19T17:17:34.275 INFO:teuthology.orchestra.run.clara004.stderr:stat: cannot read file system information for ‘/home/ubuntu/cephtest/mnt.3’: No such file or directory 2017-10-19T17:17:34.275 INFO:tasks.cephfs.fuse_mount:mount point does not exist: /home/ubuntu/cephtest/mnt.3 2017-10-19T17:17:34.275 INFO:teuthology.orchestra.run.clara004:Running: 'rm -rf /home/ubuntu/cephtest/mnt.3' 2017-10-19T17:17:34.336 INFO:teuthology.orchestra.run.clara004:Running: 'sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c \'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))\' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf' 2017-10-19T17:17:34.341 INFO:teuthology.orchestra.run.clara013:Running: 'sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c \'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))\' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf' 2017-10-19T17:17:34.454 INFO:tasks.cephfs_test_runner:test_client_release_bug (tasks.cephfs.test_client_limits.TestClientLimits) ... ok 2017-10-19T17:17:34.454 INFO:tasks.cephfs_test_runner: 2017-10-19T17:17:34.454 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2017-10-19T17:17:34.455 INFO:tasks.cephfs_test_runner:Ran 5 tests in 477.917s Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2017:3387 |
Description of problem: Test failed during FS Regression suite Patrick checked the traceback and confirmed this is issue with kernel not releasing the dentrie for ceph-fuse to cleanup its cache. Test: description: fs/basic_functional/{begin.yaml clusters/4-remote-clients.yaml mount/fuse.yaml overrides/{debug.yaml frag_enable.yaml no_client_pidfile.yaml whitelist_health.yaml RHEL 7.4/3.10.0-693.2.2.el7.x86_64 2017-09-17T01:44:16.920 INFO:tasks.cephfs_test_runner:====================================================================== 2017-09-17T01:44:16.921 INFO:tasks.cephfs_test_runner:ERROR: test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) 2017-09-17T01:44:16.921 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2017-09-17T01:44:16.921 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2017-09-17T01:44:16.921 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/gitlab.osas.lab.eng.rdu2.redhat.com_ceph_ceph_rh-luminous/qa/tasks/cephfs/test_client_limits.py", line 97, in test_client_pin 2017-09-17T01:44:16.921 INFO:tasks.cephfs_test_runner: self._test_client_pin(True) 2017-09-17T01:44:16.921 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/gitlab.osas.lab.eng.rdu2.redhat.com_ceph_ceph_rh-luminous/qa/tasks/cephfs/test_client_limits.py", line 89, in _test_client_pin 2017-09-17T01:44:16.922 INFO:tasks.cephfs_test_runner: reject_fn=lambda x: x < int(cache_size*.8)) 2017-09-17T01:44:16.922 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/gitlab.osas.lab.eng.rdu2.redhat.com_ceph_ceph_rh-luminous/qa/tasks/ceph_test_case.py", line 125, in wait_until_equal 2017-09-17T01:44:16.922 INFO:tasks.cephfs_test_runner: elapsed, expect_val, val 2017-09-17T01:44:16.922 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out after 600 seconds waiting for 80 (currently 202) 2017-09-17T01:44:16.922 INFO:tasks.cephfs_test_runner: 2017-09-17T01:44:16.922 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2017-09-17T01:44:16.922 INFO:tasks.cephfs_test_runner:Ran 3 tests in 848.134s 2017-09-17T01:44:16.923 INFO:tasks.cephfs_test_runner: 2017-09-17T01:44:16.923 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2017-09-17T01:44:16.923 INFO:tasks.cephfs_test_runner: 2017-09-17T01:44:16.923 INFO:tasks.cephfs_test_runner:====================================================================== 2017-09-17T01:44:16.923 INFO:tasks.cephfs_test_runner:ERROR: test_client_pin (tasks.cephfs.test_client_limits.TestClientLimits) 2017-09-17T01:44:16.923 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2017-09-17T01:44:16.923 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2017-09-17T01:44:16.923 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/gitlab.osas.lab.eng.rdu2.redhat.com_ceph_ceph_rh-luminous/qa/tasks/cephfs/test_client_limits.py", line 97, in test_client_pin 2017-09-17T01:44:16.924 INFO:tasks.cephfs_test_runner: self._test_client_pin(True) 2017-09-17T01:44:16.924 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/gitlab.osas.lab.eng.rdu2.redhat.com_ceph_ceph_rh-luminous/qa/tasks/cephfs/test_client_limits.py", line 89, in _test_client_pin 2017-09-17T01:44:16.924 INFO:tasks.cephfs_test_runner: reject_fn=lambda x: x < int(cache_size*.8)) 2017-09-17T01:44:16.924 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/gitlab.osas.lab.eng.rdu2.redhat.com_ceph_ceph_rh-luminous/qa/tasks/ceph_test_case.py", line 125, in wait_until_equal 2017-09-17T01:44:16.924 INFO:tasks.cephfs_test_runner: elapsed, expect_val, val 2017-09-17T01:44:16.924 INFO:tasks.cephfs_test_runner:RuntimeError: Timed out after 600 seconds waiting for 80 (currently 202) Full logs: http://magna002.ceph.redhat.com/vasu-2017-09-16_00:44:06-fs-luminous---basic-multi/274192/teuthology.log