Bug 1347195 - Unable to Enable Journaling on an Image
Summary: Unable to Enable Journaling on an Image
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Ceph Storage
Classification: Red Hat
Component: RBD
Version: 2.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: rc
: 2.0
Assignee: Jason Dillaman
QA Contact: Tanay Ganguly
URL:
Whiteboard:
Depends On:
Blocks: 1343229
TreeView+ depends on / blocked
 
Reported: 2016-06-16 09:34 UTC by Tanay Ganguly
Modified: 2017-07-30 15:33 UTC (History)
5 users (show)

Fixed In Version: RHEL: ceph-10.2.2-3.el7cp Ubuntu: ceph_10.2.2-4redhat1xenial
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-08-23 19:41:52 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Ceph Project Bug Tracker 16348 0 None None None 2016-06-16 13:43:13 UTC
Red Hat Product Errata RHBA-2016:1755 0 normal SHIPPED_LIVE Red Hat Ceph Storage 2.0 bug fix and enhancement update 2016-08-23 23:23:52 UTC

Description Tanay Ganguly 2016-06-16 09:34:06 UTC
Description of problem:
Even after enabling Journal, i am seeing the Mirroring state as Disabled and the image is not getting synced to Slave Node.

Version-Release number of selected component (if applicable):
ceph version 10.2.2-1.el7cp
rbd-mirror-10.2.2-1.el7cp.x86_64

How reproducible:
Once

Steps to Reproduce:
While Reproducing BZ 1347174, same steps as the mentioned BUG

On one of my Image test2, mirroring state is showing as disabled.
Even after enabling the Journaling on the Image, its unable to sync with the Slave Node. Although Journaling feature is showing as enabled but the functionality is not working.

Master:
rbd image 'test2':
        size 10240 MB in 2560 objects
        order 22 (4096 kB objects)
        block_name_prefix: rbd_data.1111238e1f29
        format: 2
        features: layering, exclusive-lock, object-map, fast-diff, deep-flatten, journaling
        flags: 
        journal: 1111238e1f29
        mirroring state: disabled

Slave:
rbd: error opening image test22016-06-16 09:09:42.499601 7ff16cff9700 10 librbd::image::OpenRequest: handle_close_image: r=0
: (2) No such file or directory
2016-06-16 09:09:42.499609 7ff16cff9700 10 librbd::ImageState: 0x7ff19b28f070 handle_open: r=-2


Actual results:
Although Journal is showing enabled, but its not working as per expected. Its not syncing to Slave Node.

Expected results:
Journaling should get enabled and the image should sync.

Additional info:


----------------------------------------------------------------------------
Master:

systemctl status -l ceph-rbd-mirror@master
● ceph-rbd-mirror@master.service - Ceph rbd mirror daemon
   Loaded: loaded (/usr/lib/systemd/system/ceph-rbd-mirror@.service; disabled; vendor preset: disabled)
   Active: active (running) since Thu 2016-06-16 11:15:02 IST; 2h 52min ago
 Main PID: 66591 (rbd-mirror)
   CGroup: /system.slice/system-ceph\x2drbd\x2dmirror.slice/ceph-rbd-mirror@master.service
           └─66591 /usr/bin/rbd-mirror -f --cluster master --id master --setuser ceph --setgroup ceph

Jun 16 13:56:24 x.x rbd-mirror[66591]: 2016-06-16 13:56:24.254626 7f702f7fe700 -1 JournalMetadata: operator(): failed to watch journal(2) No such file or directory
Jun 16 13:56:24 x.x rbd-mirror[66591]: 2016-06-16 13:56:24.254646 7f702f7fe700 -1 JournalMetadata: failed to initialize immutable metadata: (2) No such file or directory
Jun 16 13:56:54 x.x rbd-mirror[66591]: 2016-06-16 13:56:54.245727 7f702f7fe700 -1 JournalMetadata: operator(): failed to watch journal(2) No such file or directory
Jun 16 13:56:54 x.x rbd-mirror[66591]: 2016-06-16 13:56:54.245738 7f702f7fe700 -1 JournalMetadata: failed to initialize immutable metadata: (2) No such file or directory
Jun 16 13:58:27 x.x rbd-mirror[66591]: 2016-06-16 13:58:27.540204 7f701cff9700 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x7f6fe8004b30 handle_get_remote_tag_class: failed to retrieve remote client: (2) No such file or directory
Jun 16 13:58:27 x.x rbd-mirror[66591]: 2016-06-16 13:58:27.540278 7f70508ee700 -1 rbd::mirror::ImageReplayer: 0x7f6fe8001ea0 [1/659a83ea-3ead-4cc1-8b5b-8c1ffb4f327b] operator(): start failed: (2) No such file or directory
Jun 16 13:58:27 x.x rbd-mirror[66591]: 2016-06-16 13:58:27.552084 7f702f7fe700 -1 JournalMetadata: operator(): failed to watch journal(2) No such file or directory
Jun 16 13:58:27 x.x rbd-mirror[66591]: 2016-06-16 13:58:27.552103 7f702f7fe700 -1 JournalMetadata: failed to initialize immutable metadata: (2) No such file or directory
Jun 16 13:58:57 x.x rbd-mirror[66591]: 2016-06-16 13:58:57.529338 7f702f7fe700 -1 JournalMetadata: operator(): failed to watch journal(2) No such file or directory
Jun 16 13:58:57 x.x rbd-mirror[66591]: 2016-06-16 13:58:57.529359 7f702f7fe700 -1 JournalMetadata: failed to initialize immutable metadata: (2) No such file or directory




Slave:

systemctl status -l ceph-rbd-mirror@slave
● ceph-rbd-mirror@slave.service - Ceph rbd mirror daemon
   Loaded: loaded (/usr/lib/systemd/system/ceph-rbd-mirror@.service; enabled; vendor preset: disabled)
   Active: active (running) since Wed 2016-06-15 16:53:29 UTC; 15h ago
 Main PID: 30993 (rbd-mirror)
   CGroup: /system.slice/system-ceph\x2drbd\x2dmirror.slice/ceph-rbd-mirror@slave.service
           └─30993 /usr/bin/rbd-mirror -f --cluster slave --id slave --setuser ceph --setgroup ceph

Jun 16 08:28:12 magna003 rbd-mirror[30993]: 2016-06-16 08:28:12.139102 7f8d41ffb700 -1 rbd::mirror::image_sync::ImageCopyRequest: 0x7f8cf000bab0 handle_object_copy: object copy failed: (2) No such file or directory
Jun 16 08:28:22 magna003 rbd-mirror[30993]: 2016-06-16 08:28:22.573271 7f8d41ffb700 -1 rbd::mirror::image_sync::ObjectCopyRequest: 0x7f8d04104100 handle_read_object: failed to read from remote object: (2) No such file or directory
Jun 16 08:28:22 magna003 rbd-mirror[30993]: 2016-06-16 08:28:22.573285 7f8d41ffb700 -1 rbd::mirror::image_sync::ImageCopyRequest: 0x7f8cf000bab0 handle_object_copy: object copy failed: (2) No such file or directory
Jun 16 08:28:22 magna003 rbd-mirror[30993]: 2016-06-16 08:28:22.573293 7f8d41ffb700 -1 rbd::mirror::ImageSync: 0x7f8cf0056210 handle_copy_image: failed to copy image: (2) No such file or directory
Jun 16 08:28:22 magna003 rbd-mirror[30993]: 2016-06-16 08:28:22.573297 7f8d41ffb700 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x7f8d0002a330 handle_image_sync: failed to sync remote image: (2) No such file or directory
Jun 16 08:28:22 magna003 rbd-mirror[30993]: 2016-06-16 08:28:22.883709 7f8d6981b700 -1 rbd::mirror::ImageReplayer: 0x7f8d000065c0 [1/659a83ea-3ead-4cc1-8b5b-8c1ffb4f327b] operator(): start failed: (2) No such file or directory
Jun 16 08:28:34 magna003 rbd-mirror[30993]: 2016-06-16 08:28:34.176266 7f8d41ffb700 -1 rbd::mirror::image_replayer::BootstrapRequest: 0x7f8d00010e50 handle_get_remote_tag_class: failed to retrieve remote client: (2) No such file or directory
Jun 16 08:28:34 magna003 rbd-mirror[30993]: 2016-06-16 08:28:34.176330 7f8d6981b700 -1 rbd::mirror::ImageReplayer: 0x7f8d000065c0 [1/659a83ea-3ead-4cc1-8b5b-8c1ffb4f327b] operator(): start failed: (2) No such file or directory
Jun 16 08:29:04 magna003 rbd-mirror[30993]: 2016-06-16 08:29:04.175564 7f8d58ef8700 -1 JournalMetadata: operator(): failed to watch journal(2) No such file or directory
Jun 16 08:29:04 magna003 rbd-mirror[30993]: 2016-06-16 08:29:04.175584 7f8d58ef8700 -1 JournalMetadata: failed to initialize immutable metadata: (2) No such file or directory

Comment 8 Jason Dillaman 2016-06-16 17:13:13 UTC
Upstream master branch PR: https://github.com/ceph/ceph/pull/9751

Comment 11 Tanay Ganguly 2016-07-07 07:41:07 UTC
Unable to reproduce this Issue.

Marking it as Verified.
ceph version 10.2.2-15.el7cp

Comment 13 errata-xmlrpc 2016-08-23 19:41:52 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-1755.html


Note You need to log in before you can comment on or make changes to this bug.