Description of problem: ======================= While IO was in progress from client, started creating the snapshots in a loop. After a while when snapshot creation is still in progress, experimented to change the config values of volume and system config value. During that time hit the following errors: [root@snapshot-09 ~]# cat /var/log/glusterfs/etc-glusterfs-glusterd.vol.log | grep "Assertion failed" [2014-04-03 00:04:42.846154] E [glusterd-snapshot.c:3389:snap_max_hard_limit_set_commit] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(gd_mgmt_v3_commit_fn+0xae) [0x7f8a60cbf59e] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot+0x106) [0x7f8a60cba4f6] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot_config_commit+0x2c0) [0x7f8a60cb7b30]))) 0-: Assertion failed: volname [2014-04-03 00:04:44.255832] E [glusterd-snapshot.c:3389:snap_max_hard_limit_set_commit] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(gd_mgmt_v3_commit_fn+0xae) [0x7f8a60cbf59e] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot+0x106) [0x7f8a60cba4f6] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot_config_commit+0x2c0) [0x7f8a60cb7b30]))) 0-: Assertion failed: volname [2014-04-03 00:04:44.559529] E [glusterd-snapshot.c:3389:snap_max_hard_limit_set_commit] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(gd_mgmt_v3_commit_fn+0xae) [0x7f8a60cbf59e] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot+0x106) [0x7f8a60cba4f6] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot_config_commit+0x2c0) [0x7f8a60cb7b30]))) 0-: Assertion failed: volname [2014-04-03 00:04:44.847555] E [glusterd-snapshot.c:3389:snap_max_hard_limit_set_commit] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(gd_mgmt_v3_commit_fn+0xae) [0x7f8a60cbf59e] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot+0x106) [0x7f8a60cba4f6] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot_config_commit+0x2c0) [0x7f8a60cb7b30]))) 0-: Assertion failed: volname [2014-04-03 00:04:45.340511] E [glusterd-snapshot.c:3389:snap_max_hard_limit_set_commit] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(gd_mgmt_v3_commit_fn+0xae) [0x7f8a60cbf59e] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot+0x106) [0x7f8a60cba4f6] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot_config_commit+0x2c0) [0x7f8a60cb7b30]))) 0-: Assertion failed: volname [2014-04-03 00:04:45.650557] E [glusterd-snapshot.c:3389:snap_max_hard_limit_set_commit] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(gd_mgmt_v3_commit_fn+0xae) [0x7f8a60cbf59e] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot+0x106) [0x7f8a60cba4f6] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot_config_commit+0x2c0) [0x7f8a60cb7b30]))) 0-: Assertion failed: volname Version-Release number of selected component (if applicable): ============================================================== glusterfs-3.4.1.7.snap.mar27.2014git-1.el6.x86_64 Steps carried out: ================= 1. Create multiple volumes 2. Mount the volume (Fuse and NFS), start heavy IO 3. Start the creation of snaps in a loop. 4. While snap creation is in progress, try to set the config values for that particular volume. It fails as expected with "Another Transition is in progress". 5. While snap creation is in progress, try to set the config values for system. It is successful and reflects for all the volumes. Actual results: =============== Somewhere between step4 OR step5 the glusterd logs reported: [2014-04-03 00:04:45.954016] E [glusterd-snapshot.c:3389:snap_max_hard_limit_set_commit] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(gd_mgmt_v3_commit_fn+0xae) [0x7f8a60cbf59e] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot+0x106) [0x7f8a60cba4f6] (-->/usr/lib64/glusterfs/3.4.1.7.snap.mar27.2014git/xlator/mgmt/glusterd.so(glusterd_snapshot_config_commit+0x2c0) [0x7f8a60cb7b30]))) 0-: Assertion failed: volname Expected results: ================= Should not hit assertion failure.
Marking snapshot BZs to RHS 3.0.
The Patch which fixes this bug can be found in http://review.gluster.org/#/c/7458/
Setting flags required to add BZs to RHS 3.0 Errata
Please move this to ON_QA only when dependent is fixed.
Moved this Bug to ON_QA again, As dependent patch has been moved to ON_QA
Version: glusterfs 3.6.0.18 built on Jun 16 2014 ======== Repeated the steps as mentioned in Description , did not face any Assertion Failures. Marking the bug as 'Verified'
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. http://rhn.redhat.com/errata/RHEA-2014-1278.html