Bug 1278418

Summary: Spurious failure in bug-1275616.t
Product: [Community] GlusterFS Reporter: Avra Sengupta <asengupt>
Component: snapshotAssignee: Avra Sengupta <asengupt>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: mainlineCC: bugs
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.8rc2 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-06-16 13:43:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Avra Sengupta 2015-11-05 12:34:44 UTC 
Description of problem:
1. snapshot clone failure is indeed the reason for the failure.
2. snapshot clone has failed in pre-validation with the error that the brick of snap3 is not up and running.
3. snap3 was created, and subsequently started (because of activate-on-create being enabled), long before we tried to create a clone out of it.
4. The snap3's brick shows no failure logs, and thereby gives us no reason to believe that it did not start properly in the course of the testcase.
5. The brick connectivity happens a tad bit after the clone command is executed.


[2015-11-05 11:25:15.103233] E [MSGID: 106122] [glusterd-snapshot.c:2376:glusterd_snapshot_clone_prevalidate] 0-management: Failed to pre validate
[2015-11-05 11:25:15.103265] E [MSGID: 106443] [glusterd-snapshot.c:2398:glusterd_snapshot_clone_prevalidate] 0-management: One or more bricks are not running. Please run snapshot status command to see brick sta
tus.
Please start the stopped brick and then issue snapshot clone command 
[2015-11-05 11:25:15.103280] W [MSGID: 106443] [glusterd-snapshot.c:8398:glusterd_snapshot_prevalidate] 0-management: Snapshot clone pre-validation failed
[2015-11-05 11:25:15.103294] W [MSGID: 106122] [glusterd-mgmt.c:166:gd_mgmt_v3_pre_validate_fn] 0-management: Snapshot Prevalidate Failed
[2015-11-05 11:25:15.103305] E [MSGID: 106122] [glusterd-mgmt.c:820:glusterd_mgmt_v3_pre_validate] 0-management: Pre Validation failed for operation Snapshot on local node
[2015-11-05 11:25:15.103315] E [MSGID: 106122] [glusterd-mgmt.c:2166:glusterd_mgmt_v3_initiate_snap_phases] 0-management: Pre Validation Failed
[2015-11-05 11:25:15.103332] E [MSGID: 106027] [glusterd-snapshot.c:7946:glusterd_snapshot_clone_postvalidate] 0-management: unable to find clone clone1 volinfo
[2015-11-05 11:25:15.103342] W [MSGID: 106444] [glusterd-snapshot.c:8837:glusterd_snapshot_postvalidate] 0-management: Snapshot create post-validation failed
[2015-11-05 11:25:15.103352] W [MSGID: 106121] [glusterd-mgmt.c:323:gd_mgmt_v3_post_validate_fn] 0-management: postvalidate operation failed
[2015-11-05 11:25:15.103362] E [MSGID: 106121] [glusterd-mgmt.c:1585:glusterd_mgmt_v3_post_validate] 0-management: Post Validation failed for operation Snapshot on local node
[2015-11-05 11:25:15.103372] E [MSGID: 106122] [glusterd-mgmt.c:2286:glusterd_mgmt_v3_initiate_snap_phases] 0-management: Post Validation Failed
[2015-11-05 11:25:15.109994]:++++++++++ G_LOG:./tests/bugs/snapshot/bug-1275616.t: TEST: 42 42 149 snap_info_volume CLI Snaps Available patchy ++++++++++
[2015-11-05 11:25:15.239358]:++++++++++ G_LOG:./tests/bugs/snapshot/bug-1275616.t: TEST: 43 43 150 snap_config_volume CLI snap-max-hard-limit patchy ++++++++++
[2015-11-05 11:25:15.378255]:++++++++++ G_LOG:./tests/bugs/snapshot/bug-1275616.t: TEST: 45 45 200 snap_info_volume CLI Snaps Available clone1 ++++++++++
[2015-11-05 11:25:15.501970] E [MSGID: 106027] [glusterd-snapshot.c:3574:glusterd_snapshot_get_info_by_volume] 0-management: Volume (clone1) does not exist [Invalid argument]
[2015-11-05 11:25:15.502024] E [MSGID: 106027] [glusterd-snapshot.c:3766:glusterd_handle_snapshot_info] 0-management: Failed to get volume info of volume clone1 [Invalid argument]
[2015-11-05 11:25:15.502061] W [MSGID: 106063] [glusterd-snapshot.c:9082:glusterd_handle_snapshot_fn] 0-management: Snapshot info failed
[2015-11-05 11:25:15.510016]:++++++++++ G_LOG:./tests/bugs/snapshot/bug-1275616.t: TEST: 46 46 200 snap_config_volume CLI snap-max-hard-limit clone1 ++++++++++
[2015-11-05 11:25:15.639515] E [MSGID: 106060] [glusterd-snapshot.c:438:snap_max_limits_display_commit] 0-management: Volume (clone1) does not exist
[2015-11-05 11:25:15.639543] E [MSGID: 106090] [glusterd-snapshot.c:1446:glusterd_handle_snapshot_config] 0-management: snap-max-limit display commit failed.
[2015-11-05 11:25:15.639558] W [MSGID: 106045] [glusterd-snapshot.c:9101:glusterd_handle_snapshot_fn] 0-management: snapshot config failed
[2015-11-05 11:25:15.684746] I [glusterd-utils.c:4883:glusterd_set_brick_status] 0-glusterd: Setting brick slave28.cloud.gluster.org:/var/run/gluster/snaps/7db8306c170541eb98c02633407bf625/brick1 status to started


Version-Release number of selected component (if applicable):
mailine


How reproducible:
Spuriously on regression setup


Steps to Reproduce:
1.
2.
3.

Actual results:


Expected results:


Additional info:
Comment 1 Avra Sengupta 2015-11-06 10:48:35 UTC 
Fix at http://review.gluster.org/#/c/12516/
Comment 2 Niels de Vos 2016-06-16 13:43:12 UTC 
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.8.0, please open a new bug report.

glusterfs-3.8.0 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://blog.gluster.org/2016/06/glusterfs-3-8-released/
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user