Red Hat Bugzilla – Bug 1076033
Unknown Key: <bricks> are reported when the glusterd was restarted
Last modified: 2016-01-14 01:50:16 EST
Description of problem: ======================= In a 2*2 clustered snapshot system, following errors were reported when the glusterd was restarted simultaneously on all the systems. [2014-03-13 04:58:46.748010] I [glusterd-store.c:2019:glusterd_restore_op_version] 0-glusterd: retrieved op-version: 3 [2014-03-13 04:58:46.764780] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-0 [2014-03-13 04:58:46.764844] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-1 [2014-03-13 04:58:46.764873] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-2 [2014-03-13 04:58:46.764897] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-3 [2014-03-13 04:58:47.733494] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-0 [2014-03-13 04:58:47.733595] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-1 [2014-03-13 04:58:47.733622] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-2 [2014-03-13 04:58:47.733644] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-3 [2014-03-13 04:58:47.758419] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-0 [2014-03-13 04:58:47.758459] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-1 [2014-03-13 04:58:47.758490] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-2 [2014-03-13 04:58:47.758514] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-3 [2014-03-13 04:58:47.778857] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-0 [2014-03-13 04:58:47.778899] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-1 [2014-03-13 04:58:47.778926] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-2 System had 188 snaps created from these bricks. Version-Release number of selected component (if applicable): ============================================================= glusterfs-3.4.1.2.snap.mar11.2014git-1.el6.x86_64 How reproducible: ================= 1/1 Steps to Reproduce: 1. Create multiple snaps for different volumes 2. Delete few snaps from different volume 3. Create the snaps with same name as deleted at step2 4. Restart the glusterd on all the machines simultaneously. While the graphs are reloaded these error messages were logged. Actual results: =============== [2014-03-13 04:58:54.621217] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-2 [2014-03-13 04:58:54.621244] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-3 [2014-03-13 04:58:54.648320] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-0 [2014-03-13 04:58:54.648436] E [glusterd-store.c:2567:glusterd_store_update_volinfo] 0-: Unknown key: brick-1 Expected results: ================= No errors should be reported.
This issue happens with regular volume as well, so it is not a snapshot bug
This doesn't affect anything other than logs. This happens always when glusterd starts up if volumes are present. This is a bug with our volume options restore code. Since this doesn't affect the operation of glusterd, I say its of low severity and the priority could also be reduced.
Marking priority as low as this bug doesn't affect any functionality
Needinfo still applies as per comment 14
This issue was eventually fixed with some patch for RHGS 3.1.0 or much earlier. Hence moving this bug to ON_QA
Verified this bug using the Version "glusterfs-3.7.1-12" with below steps. Steps: ~~~~~~ 1. Created multiple volumes (#4 Dis-Rep) using 2 node cluster. 2. Created multiple snaps for diff volumes. 3. Deleted fe snaps from diff volumes 4. Created snaps with same name as deleted as step-3 5. Restarted the glusterd on all the M/C's simultaneously 6. Verified the glusterd log for the "unknow key" key // Not found Fix is working. moving this Bug to verified state.
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2015-1845.html