Description of problem: Seeing lots of error message, when bench-write is running in parallel with enable/disable of Journaling Version-Release number of selected component (if applicable): ceph version 10.2.1-6.el7cp How reproducible: 2 Times Steps to Reproduce: 1. create an rbd image with layering, exclusive-lock,fast-diff, object-map, jounaling 2. start bench-write on the image. 3. when writes are happening disable the journaling feature. Actual results: Seeing lots of Error Message Expected results: Error message should not be shown. ----------------------------------------------------------------------------- # rbd bench-write -p cephfs_data --image img1 --io-size 10240 --io-pattern rand bench-write io_size 10240 io_threads 16 bytes 1073741824 pattern random SEC OPS OPS/SEC BYTES/SEC 2016-05-31 07:03:24.530310 7f0589efa700 -1 JournalMetadata: init: failed to watch journal(2) No such file or directory 2016-05-31 07:03:24.530368 7f05896f9700 -1 librbd::Journal: 0x7f057000a640 handle_initialized: failed to initialize journal: (2) No such file or directory 2016-05-31 07:03:24.530421 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to open journal: (2) No such file or directory 2016-05-31 07:03:24.530431 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to close journal: (2) No such file or directory 2016-05-31 07:03:24.542757 7f05896f9700 -1 librbd::ExclusiveLock: failed to acquire exclusive lock:(2) No such file or directory 2016-05-31 07:03:24.554547 7f0589efa700 -1 JournalMetadata: init: failed to watch journal(2) No such file or directory 2016-05-31 07:03:24.554593 7f05896f9700 -1 librbd::Journal: 0x7f057000a640 handle_initialized: failed to initialize journal: (2) No such file or directory 2016-05-31 07:03:24.554642 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to open journal: (2) No such file or directory 2016-05-31 07:03:24.554650 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to close journal: (2) No such file or directory 2016-05-31 07:03:24.560040 7f05896f9700 -1 librbd::ExclusiveLock: failed to acquire exclusive lock:(2) No such file or directory 2016-05-31 07:03:24.571100 7f0589efa700 -1 JournalMetadata: init: failed to watch journal(2) No such file or directory 2016-05-31 07:03:24.571151 7f05896f9700 -1 librbd::Journal: 0x7f057000a640 handle_initialized: failed to initialize journal: (2) No such file or directory 2016-05-31 07:03:24.571200 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to open journal: (2) No such file or directory 2016-05-31 07:03:24.571208 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to close journal: (2) No such file or directory 2016-05-31 07:03:24.576809 7f05896f9700 -1 librbd::ExclusiveLock: failed to acquire exclusive lock:(2) No such file or directory 2016-05-31 07:03:24.589047 7f0589efa700 -1 JournalMetadata: init: failed to watch journal(2) No such file or directory 2016-05-31 07:03:24.589097 7f05896f9700 -1 librbd::Journal: 0x7f057000a640 handle_initialized: failed to initialize journal: (2) No such file or directory 2016-05-31 07:03:24.589143 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to open journal: (2) No such file or directory 2016-05-31 07:03:24.589151 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to close journal: (2) No such file or directory 2016-05-31 07:03:24.602635 7f05896f9700 -1 librbd::ExclusiveLock: failed to acquire exclusive lock:(2) No such file or directory 2016-05-31 07:03:24.636691 7f0589efa700 -1 JournalMetadata: failed to locate client: 2016-05-31 07:03:24.636710 7f05896f9700 -1 librbd::Journal: 0x7f057000a640 handle_initialized: failed to initialize journal: (2) No such file or directory 2016-05-31 07:03:24.652423 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to open journal: (2) No such file or directory 2016-05-31 07:03:24.652434 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to close journal: (2) No such file or directory 2016-05-31 07:03:24.669508 7f05896f9700 -1 librbd::ExclusiveLock: failed to acquire exclusive lock:(2) No such file or directory 37 1 0.45 4595.82 38 1047 27.97 286434.85 39 1212 31.48 322309.09 40 1357 34.32 351454.88 2016-05-31 07:03:37.119280 7f0589efa700 -1 JournalMetadata: init: failed to watch journal(2) No such file or directory 2016-05-31 07:03:37.119302 7f05896f9700 -1 librbd::Journal: 0x7f0570008b20 handle_initialized: failed to initialize journal: (2) No such file or directory 2016-05-31 07:03:37.119337 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to open journal: (2) No such file or directory 2016-05-31 07:03:37.119343 7f05896f9700 -1 librbd::exclusive_lock::AcquireRequest: failed to close journal: (2) No such file or directory 2016-05-31 07:03:37.133198 7f05896f9700 -1 librbd::ExclusiveLock: failed to acquire exclusive lock:(2) No such file or directory ----------------------------------------------------------------------------- For the below log messages there is a BUG 1335032 https://bugzilla.redhat.com/show_bug.cgi?id=1335032 for i in {1..200}; do rbd feature enable cephfs_data/img1 journaling; sleep 2; rbd feature disable cephfs_data/img1 journaling; done 2016-05-31 07:03:24.417365 7f8574ff9700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:03:37.007677 7fa6677fe700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:03:49.975260 7fc307fff700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:04:05.286831 7f8a6bfff700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:04:05.388188 7f8a37fff700 -1 JournalMetadata: handle_watch_reset: failed to watch journal(2) No such file or directory 2016-05-31 07:04:05.489165 7f8a37fff700 -1 JournalMetadata: handle_watch_reset: failed to watch journal(2) No such file or directory 2016-05-31 07:04:05.590001 7f8a37fff700 -1 JournalMetadata: handle_watch_reset: failed to watch journal(2) No such file or directory 2016-05-31 07:04:21.811341 7f0b7e7fc700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:04:21.912928 7f0b567fc700 -1 JournalMetadata: handle_watch_reset: failed to watch journal(2) No such file or directory 2016-05-31 07:04:34.691536 7eff4ffff700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:04:50.387203 7fbd117fa700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:05:06.553524 7fbe8affd700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:05:21.178626 7fd4eeffd700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:05:34.122701 7effd77fe700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:05:48.703560 7fc04e7fc700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:06:03.779298 7fe744ff9700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:06:18.031368 7fa9a3fff700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:06:32.759689 7fe71cff9700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:06:45.695572 7f30e0ff9700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:06:58.470836 7f418a7fc700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:07:10.868918 7f74d37fe700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:07:25.661293 7f24776fd700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:07:39.784748 7fae3d7fa700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:07:54.513960 7f4658ff9700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:08:08.900208 7f7e1d7fa700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:08:20.790760 7f40d3fff700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:08:33.231866 7f46cdffb700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:08:48.871778 7fc6c4ef8700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:09:02.943667 7f95ea7fc700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected 2016-05-31 07:09:03.044967 7f95c27fc700 -1 JournalMetadata: handle_watch_reset: failed to watch journal(2) No such file or directory 2016-05-31 07:09:15.367820 7fbd526fb700 -1 JournalMetadata: journal watch error: (107) Transport endpoint is not connected
*** This bug has been marked as a duplicate of bug 1335032 ***
Upstream PR: https://github.com/ceph/ceph/pull/9440
Merged Jewel PR: https://github.com/ceph/ceph/pull/9610
We will take this change in as part of the rebase to ceph 10.2.2.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHBA-2016-1755.html