Bug 1202388

Summary: [SNAPSHOT]: After a volume which has quota enabled is restored to a snap, attaching another node to the cluster is not successful
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: senaik
Component: snapshotAssignee: rjoseph
Status: CLOSED ERRATA QA Contact: Anil Shah <ashah>
Severity: high Docs Contact:
Priority: high    
Version: rhgs-3.0CC: annair, asengupt, asriram, asrivast, nsathyan, rcyriac, rhs-bugs, rjoseph, rkavunga, storage-qa-internal, vagarwal
Target Milestone: ---   
Target Release: RHGS 3.1.0   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard: SNAPSHOT
Fixed In Version: glusterfs-3.7.0-3.el6rhs Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 1202436 1204636 (view as bug list) Environment:
Last Closed: 2015-07-29 04:39:21 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1122377, 1219744    
Bug Blocks: 1191838, 1202436, 1202842, 1204636, 1223636    

Description senaik 2015-03-16 14:16:05 UTC
Description of problem:
=======================
After quota is enabled on the volume, restore the volume to any snapshot and then attach another node to the cluster. CLI shows the command is succesful but gluster peer status shows the newly added node is in 'Rejected' state


Version-Release number of selected component (if applicable):
============================================================
glusterfs 3.6.0.51

How reproducible:
================
2/2


Steps to Reproduce:
====================
1.Create a 6x2 dist rep volume and start it 

2.Fuse and NFS mount the volume 
enable quota and set server and client threads to 4 and 7 

3.Create some files and take snapshot and activate it.
Repeat the same steps at few intervals

4.Stop the volume and restore it to a particular snapshot.

5.Start the volume and access the files from the mount point. 

6.Now add another node to the cluster (snapshot12.lab.eng.blr.redhat.com)

[2015-03-16 13:30:10.157903]  : peer probe snapshot12.lab.eng.blr.redhat.com : SUCCESS

7. Check gluster peer status from the node where the probe command was executed

 gluster peer status
Number of Peers: 5

Hostname: rhs-arch-srv2.lab.eng.blr.redhat.com
Uuid: 1db55b58-30d0-4da1-8a91-3b795a430446
State: Peer in Cluster (Connected)

Hostname: rhs-arch-srv3.lab.eng.blr.redhat.com
Uuid: b3dcf788-7e08-45d3-8343-f032cd5b41fc
State: Peer in Cluster (Connected)

Hostname: rhs-arch-srv4.lab.eng.blr.redhat.com
Uuid: 12a1f3a6-4f96-4345-9198-bdc540792572
State: Peer in Cluster (Connected)

Hostname: snapshot11.lab.eng.blr.redhat.com
Uuid: e4509823-08ad-4e11-90db-38024fae926c
State: Peer in Cluster (Connected)

Hostname: snapshot12.lab.eng.blr.redhat.com
Uuid: 3ade540a-ff00-4334-b07f-8d43ce92b07d
State: Peer Rejected (Connected) -----------------------> shows Rejected state

8.The other nodes in the cluster do not show the newly added node in the cluster 

[root@rhs-arch-srv2 .unsupported]# gluster peer status
Number of Peers: 4

Hostname: 10.70.34.50
Uuid: 520168e7-b2c4-49c7-98f0-96d0a3acbbc8
State: Peer in Cluster (Connected)

Hostname: rhs-arch-srv3.lab.eng.blr.redhat.com
Uuid: b3dcf788-7e08-45d3-8343-f032cd5b41fc
State: Peer in Cluster (Connected)

Hostname: rhs-arch-srv4.lab.eng.blr.redhat.com
Uuid: 12a1f3a6-4f96-4345-9198-bdc540792572
State: Peer in Cluster (Connected)

Hostname: snapshot11.lab.eng.blr.redhat.com
Uuid: e4509823-08ad-4e11-90db-38024fae926c
State: Peer in Cluster (Connected)

9.Check the peer status on the newly added node 

[root@snapshot12 ~]# gluster peer status
Number of Peers: 1

Hostname: 10.70.34.50
Uuid: 520168e7-b2c4-49c7-98f0-96d0a3acbbc8
State: Peer Rejected (Connected)


-------------------Part of the log from 10.70.34.50--------------------
[2015-03-16 13:30:10.371769] I [glusterd-handler.c:2320:__glusterd_handle_incoming_friend_req] 0-glusterd: Received probe from uuid: 3ade540a-ff00-4334-b07f-8d43ce92b07d
[2015-03-16 13:30:10.372065] E [MSGID: 106012] [glusterd-utils.c:3531:glusterd_compare_friend_volume] 0-management: Cksums of quota configuration of volume vol0 differ. local cksum = 0, remote  cksum = 1271429248 on peer snapshot12.lab.e
ng.blr.redhat.com
[2015-03-16 13:30:10.372097] I [glusterd-handler.c:3372:glusterd_xfer_friend_add_resp] 0-glusterd: Responded to snapshot12.lab.eng.blr.redhat.com (0), ret: 0
--------------------------------------------------------------------

Actual results:
===============
After volume which has quota enabled is restored to a snapshot, attaching another node to the cluster is not successful

Expected results:
================
Attaching the node should be successful

Additional info:
===============
[root@inception ~]# gluster v i vol0
 
Volume Name: vol0
Type: Distributed-Replicate
Volume ID: b1ae2f52-4161-4200-a639-ba2c5aeeb8c5
Status: Started
Snap Volume: no
Number of Bricks: 6 x 2 = 12
Transport-type: tcp
Bricks:
Brick1: inception.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick1/b1
Brick2: rhs-arch-srv2.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick2/b1
Brick3: rhs-arch-srv3.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick3/b1
Brick4: rhs-arch-srv4.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick4/b1
Brick5: inception.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick5/b2
Brick6: rhs-arch-srv2.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick6/b2
Brick7: rhs-arch-srv3.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick7/b2
Brick8: rhs-arch-srv4.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick8/b2
Brick9: inception.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick9/b3
Brick10: rhs-arch-srv2.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick10/b3
Brick11: rhs-arch-srv3.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick11/b3
Brick12: rhs-arch-srv4.lab.eng.blr.redhat.com:/var/run/gluster/snaps/64e656d56fac44dcaf1dda4ed0b10dc6/brick12/b3
Options Reconfigured:
client.event-threads: 4
server.event-threads: 7
features.uss: enable
features.quota: on
performance.readdir-ahead: on
auto-delete: disable
snap-max-soft-limit: 90
snap-max-hard-limit: 256

Comment 3 senaik 2015-03-19 07:30:56 UTC
Could we please have the RCA for this bug updated

Comment 4 senaik 2015-03-20 06:21:48 UTC
If quota is enabled on the volume and if it is restored, then we cannot add another node to the cluster.

Comment 5 Sachin Pandit 2015-03-20 06:25:29 UTC
RCA : 
When we enable quota on a volume, the quota cksum file is created.
and is present in /var/lib/glusterd/vols/<volname>/, and the in memory
structure of volinfo is updated as well. When we take a snapshot we
copy the quota cksum file and along with that we take a copy of in memory
structure. During restore we are missing out to fill in the quota cksum
value. Hence after we restore the snapshot the cksum is updated to 0.
When we try to add a new node, the quota cksum mismatches and because
of that we fail to add a new node.

Comment 6 Sachin Pandit 2015-03-20 12:07:53 UTC
While trying out a workaround for this problem. I have hit any interesting issue. The issue is as follows:

1) Let us say we have 2 nodes and a volume (vol1) is spread across these 2 nodes.
2) We take a snapshot
3) We set some options on the volume "vol1"
   [a] Volume options are persisted in /var/lib/glusterd/vols/<volname>/info
   [b] snap volume info is saved in /var/lib/glusterd/snaps/<snapvol>/info 
   * Newly added option should be saved in [a] (mentioned above)
4) Peer probe to a new node.
5) Now check the "info file of snap volume (i.e mentioned in point 3.b)
   * Info file of snap volume contains the newly set option. Ideally the
     snap volume should not contain that option, as it is set after
      snapshot create.

* Because of this there will be a mismatch of volume checksum too.
* Until and unless the glusterd does not restart we are safe. Once glusterd
  is restarted on the new node, because of checksum mismatch the newly added
  node goes into a peer reject state.

Comment 7 Sachin Pandit 2015-03-21 06:01:01 UTC
There are 2 separate problems for this bug

1) After taking a snapshot if we set any volume option and if we add a new node,
   then ideally the new node should have the same "info" file of snap-volume
   compared to existing nodes. But, the problem here is along with existing
   "info" file contents, the new volume option is also being updated in "info"
   file of snap-volume, which should not be the case. Because of this checksum
   mismatch happens and peer goes into rejected state.
 
   * Workaround here would be that, after snapshot restore, we need to copy the
     /var/lib/glusterd/vols/<volname>/info file from existing node to new
     node and restart the glusterd on new node.

2) Let us imagine we have quota enabled on the volume and we take snapshot.
   During snapshot restore we fail to re-calculate the quota checksum and
   because of that quota checksum is initialized to zero. If we try to add
   a new node then quota_conf checksum mismatches and because of that peer
   will go into a rejected state

   * Workaround here would be that after snapshot restore, the glusterd needs
     to be restarted on existing nodes and then a peer probe has to be done on
     new node.

Comment 8 Pavithra 2015-03-23 07:02:33 UTC
Hi Rajesh,

I see this bug being added as a known issue for the 3.0.4 release. Please fill out the doc text.

Comment 10 Pavithra 2015-03-23 17:07:08 UTC
Hi Rajesh,

Can you please review the edited doc text for technical accuracy and sign off?

Comment 11 rjoseph 2015-03-24 07:15:52 UTC
doc-text seems fine to me.

Comment 12 senaik 2015-03-26 09:46:44 UTC
Workaround mentioned in Comment 7 is tested and works fine.

Comment 14 Anil Shah 2015-07-04 07:29:09 UTC
root@darkknightrises cron.d]# gluster snapshot create snap3 vol0
snapshot create: success: Snap snap3_GMT-2015.07.04-07.19.09 created successfully

[root@darkknightrises cron.d]# gluster snapshot activate snap3_GMT-2015.07.04-07.19.09
Snapshot activate: snap3_GMT-2015.07.04-07.19.09: Snap activated successfully
[root@darkknightrises cron.d]# gluster v stop vol0
Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y
volume stop: vol0: success
[root@darkknightrises cron.d]# gluster snapshot restore snap3_GMT-2015.07.04-07.19.09
Restore operation will replace the original volume with the snapshotted volume. Do you still want to continue? (y/n) y
Snapshot restore: snap3_GMT-2015.07.04-07.19.09: Snap restored successfully
[root@darkknightrises cron.d]# gluster v start vol0 
volume start: vol0: success


[root@darkknightrises cron.d]# gluster peer probe 10.70.47.151
peer probe: success. 
[root@darkknightrises cron.d]# gluster peer status
Number of Peers: 4

Hostname: 10.70.33.219
Uuid: e91bd1ef-38d5-4389-8db8-2a6528ccbb17
State: Peer in Cluster (Connected)

Hostname: 10.70.44.13
Uuid: 7e6e250c-b14d-4850-96c6-2a194a47b90e
State: Peer in Cluster (Connected)

Hostname: 10.70.33.225
Uuid: 0e2626da-55f5-4523-95c0-55eefb4e53a3
State: Peer in Cluster (Connected)

Hostname: 10.70.47.151
Uuid: 6f5126cf-312a-4274-8a09-380dd6d465ad
State: Peer in Cluster (Connected)


Bug verified on build glusterfs-3.7.1-7.el6rhs.x86_64.

Comment 16 errata-xmlrpc 2015-07-29 04:39:21 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2015-1495.html