This service will be undergoing maintenance at 00:00 UTC, 2016-08-01. It is expected to last about 1 hours

Bug 888752

Summary: "volume status" for single brick fails if brick is not on the server where peer command was issued.
Product: [Community] GlusterFS Reporter: Kaushal <kaushal>
Component: glusterdAssignee: Kaushal <kaushal>
Status: CLOSED CURRENTRELEASE QA Contact: SATHEESARAN <sasundar>
Severity: unspecified Docs Contact:
Priority: medium    
Version: mainlineCC: lancelotds, rwheeler, shmohan, shtripat
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: glusterfs-3.5.0 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 902214 1028878 (view as bug list) Environment:
Last Closed: 2014-04-17 07:40:04 EDT Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Bug Depends On:    
Bug Blocks: 877961, 902214, 918453, 1028878    

Description Kaushal 2012-12-19 07:11:08 EST
For "volume status" commands, the source glusterd depends on some keys to be set in the context dictionary to modify and merge the replies sent by other peers. These keys are set in the commit-op on the source.  
But in "volume status" for a single brick, with the brick on another peer, the commit-op finishes without setting the required keys, which prevents the replies from other peers from being merged properly and causes the command to fail.
Comment 1 Vijay Bellur 2012-12-27 02:40:56 EST
CHANGE: http://review.gluster.org/4347 (glusterd: "volume status" for remote brick fails on cli.) merged in master by Vijay Bellur (vbellur@redhat.com)
Comment 2 Jules Wang 2013-01-29 02:20:27 EST
please set bug status to MODIFIED
Comment 3 Shireesh 2013-03-08 05:26:33 EST
Moving back to ASSIGNED based on following discussion.

-------- Original Message --------
Subject: 	Re: Bug 888752 - "volume status" for single brick fails if brick is not on the server where peer command was issued.
Date: 	Thu, 7 Mar 2013 08:38:27 -0500 (EST)
From: 	Kaushal M <kaushal@redhat.com>
To: 	Shireesh Anjal <sanjal@redhat.com>
CC: 	Prasanth <pprakash@redhat.com>, Satheesaran Sundaramoorthi <sasundar@redhat.com>, Raghavendra Talur <rtalur@redhat.com>, Sahina Bose <sabose@redhat.com>, Shruti Sampat <ssampat@redhat.com>, Dustin Tsang <dtsang@redhat.com>, Matthew Mahoney <mmahoney@redhat.com>, Vijay Bellur <vbellur@redhat.com>


Found the problem. 

TL;DR: I didn't do the correct check for when to expect tasks.

The normal output & xml output code always expect tasks to be present when the normal status is requested, be it for the whole volume, just a brick or the nfs/shd processes. The xml generation code fails when it doesn't find tasks. The tasks are added only by the glusterd on the system where the command is issued, which doesn't happen when we request for the status of a brick on another machine or request for the status of specific process. Ideally, the code should expect for tasks only when the normal full status command is issued. In normal cli output we don't consider the failure of task output code, which allows it to complete successfully.

As a workaround you could,
i.  Use the full status command instead.
ii. Run the command on the system with the brick.
I know these are not ideal. I'll try to have a fix for this soon.

- Kaushal

----- Original Message -----
> From: "Shireesh Anjal" <sanjal@redhat.com>
> To: "Vijay Bellur" <vbellur@redhat.com>
> Cc: "Prasanth" <pprakash@redhat.com>, "Satheesaran Sundaramoorthi" <sasundar@redhat.com>, "Kaushal M"
> <kaushal@redhat.com>, "Raghavendra Talur" <rtalur@redhat.com>, "Sahina Bose" <sabose@redhat.com>, "Shruti Sampat"
> <ssampat@redhat.com>, "Dustin Tsang" <dtsang@redhat.com>, "Matthew Mahoney" <mmahoney@redhat.com>
> Sent: Thursday, March 7, 2013 6:01:15 PM
> Subject: Re: Bug 888752 - "volume status" for single brick fails if brick is not on the server where peer command was
> issued.
> 
... 
> Trying to prepare for my tomorrow's demo, I also hit this issue with
> upstream (glusterfs-3.4.0aphpa2). The problem is that it exits with
> return code 2 when we pass "--xml". Seems to work fine when "--xml"
> is
> not passed.
> 
> [root@localhost vdsm]# gluster volume status test1
> 10.70.37.127:/tmp/test1
> Status of volume: test1
> Gluster process                                         Port Online
>  Pid
> ------------------------------------------------------------------------------
> Brick 10.70.37.127:/tmp/test1                           49152 Y
>       20878
> 
> [root@localhost vdsm]# gluster volume status test1
> 10.70.37.127:/tmp/test1 --xml
> [root@localhost vdsm]# echo $?
> 2
> [root@localhost vdsm]#
> 
> You can check this on 10.70.37.124.
>
Comment 4 Anand Avati 2013-07-10 08:50:09 EDT
REVIEW: http://review.gluster.org/5308 (cli,glusterd: Fix when tasks are shown in 'volume status') posted (#2) for review on master by Kaushal M (kaushal@redhat.com)
Comment 5 Anand Avati 2013-07-15 01:35:47 EDT
REVIEW: http://review.gluster.org/5308 (cli,glusterd: Fix when tasks are shown in 'volume status') posted (#3) for review on master by Kaushal M (kaushal@redhat.com)
Comment 6 Anand Avati 2013-08-03 08:51:54 EDT
COMMIT: http://review.gluster.org/5308 committed in master by Anand Avati (avati@redhat.com) 
------
commit 572f5f0a85c97a4f90a33be87b96368a0d7e7a8e
Author: Kaushal M <kaushal@redhat.com>
Date:   Wed Jul 10 18:10:49 2013 +0530

    cli,glusterd: Fix when tasks are shown in 'volume status'
    
    Asynchronous tasks are shown in 'volume status' only for a normal volume
    status request for either all volumes or a single volume.
    
    Change-Id: I9d47101511776a179d213598782ca0bbdf32b8c2
    BUG: 888752
    Signed-off-by: Kaushal M <kaushal@redhat.com>
    Reviewed-on: http://review.gluster.org/5308
    Reviewed-by: Amar Tumballi <amarts@redhat.com>
    Tested-by: Gluster Build System <jenkins@build.gluster.com>
    Reviewed-by: Anand Avati <avati@redhat.com>
Comment 7 Niels de Vos 2014-04-17 07:40:04 EDT
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.5.0, please reopen this bug report.

glusterfs-3.5.0 has been announced on the Gluster Developers mailinglist [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] http://thread.gmane.org/gmane.comp.file-systems.gluster.devel/6137
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user