This service will be undergoing maintenance at 00:00 UTC, 2017-10-23 It is expected to last about 30 minutes
Bug 982471 - Dist-geo-rep: In geo rep status 'HEALTH' and 'UPTIME' is N/A and log gives error - 'Unable to fetch extra statusfor <master_vol> <slave_vol>'
Dist-geo-rep: In geo rep status 'HEALTH' and 'UPTIME' is N/A and log gives er...
Status: CLOSED ERRATA
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: geo-replication (Show other bugs)
2.1
x86_64 Linux
medium Severity urgent
: ---
: ---
Assigned To: Avra Sengupta
amainkar
: Regression
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2013-07-09 02:49 EDT by Rachana Patel
Modified: 2015-04-20 07:57 EDT (History)
6 users (show)

See Also:
Fixed In Version: glusterfs-3.4.0.23rhs-1
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2013-09-23 18:29:53 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Rachana Patel 2013-07-09 02:49:52 EDT
Description of problem:
 Dist-geo-rep: In geo rep status 'HEALTH' and 'UPTIME' is N/A and log gives error -  'Unable to fetch extra statusfor <master_vol> <slave_vol>'

Version-Release number of selected component (if applicable):
3.4.0.12rhs.beta3-1.el6rhs.x86_64

How reproducible:


Steps to Reproduce:
1. master was DHT volume(1x6), slave was DHT volume(1x3)
2. created data on master volume more than 80GB.
3. created geo rep session between master and slave, 
keep checking status

[root@hulk ~]#   gluster volume geo master1 10.70.43.147::slave1 status
NODE                        MASTER     SLAVE                   HEALTH    UPTIME       
--------------------------------------------------------------------------------------
ironman.blr.redhat.com      master1    10.70.43.147::slave1    Stable    00:26:44     
wall.blr.redhat.com         master1    10.70.43.147::slave1    Stable    01:54:20    


glusterd log says:-
[2013-07-08 14:59:41.815517] E [glusterd-geo-rep.c:2270:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master1(master), 10.70.43.147::slave1(slave)
[2013-07-08 15:00:27.981754] E [glusterd-geo-rep.c:2270:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master1(master), 10.70.43.147::slave1(slave)
[2013-07-08 15:00:36.049246] E [glusterd-geo-rep.c:2270:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master1(master), 10.70.43.147::slave1(slave)
[2013-07-08 15:00:41.752925] E [glusterd-geo-rep.c:2270:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master1(master), 10.70.43.147::slave1(slave)
[2013-07-08 15:01:26.045033] E [glusterd-geo-rep.c:2270:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master1(master), 10.70.43.147::slave1(slave)
[2013-07-08 15:01:49.681062] E [glusterd-geo-rep.c:2270:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master1(master), 10.70.43.147::slave1(slave)

Actual results:


Expected results:


Additional info:
Comment 2 Amar Tumballi 2013-08-01 06:29:31 EDT
not reproducible for us. Can you try with the latest build?
Comment 3 Rachana Patel 2013-08-13 05:00:20 EDT
not able to reproduce with latest build 3.4.0.18rhs-1.el6rhs.x86_64, hence moving to verified
Comment 4 Rachana Patel 2013-08-20 06:48:21 EDT
able to reproduce with 3.4.0.18rhs-1.el6rhs.x86_64
hence moving back to Assigned

log says
Unable to fetch extra statusfor master1(master)...

status file has status stable but status command shows N/A
Comment 5 M S Vishwanath Bhat 2013-08-20 07:13:21 EDT
I hit this with 20rhs-2 in few rather simple steps.

I rotated the logs using logrotate(8)

logrotate -f /etc/logrotate.d/glusterfs-georep

 
                                        MASTER: master  SLAVE: falcon::slave
 
NODE                         HEALTH    UPTIME      FILES SYNCD    FILES PENDING    BYTES PENDING    DELETES PENDING   
--------------------------------------------------------------------------------------------------------------------
spitfire.blr.redhat.com      Stable    01:22:50    1109           0                0Bytes           0                 
typhoon.blr.redhat.com       Stable    01:22:50    0              0                0Bytes           0                 
harrier.blr.redhat.com       Stable    01:22:50    1088           0                0Bytes           0                 
mustang.blr.redhat.com       N/A       N/A         N/A            N/A              N/A              N/A               
 
                                        MASTER: master  SLAVE: falcon::slave
 
NODE                         HEALTH    UPTIME      FILES SYNCD    FILES PENDING    BYTES PENDING    DELETES PENDING   
--------------------------------------------------------------------------------------------------------------------
spitfire.blr.redhat.com      Stable    01:22:55    1109           0                0Bytes           0                 
typhoon.blr.redhat.com       Stable    01:22:56    0              0                0Bytes           0                 
harrier.blr.redhat.com       Stable    01:22:56    1088           0                0Bytes           0                 
mustang.blr.redhat.com       N/A       N/A         N/A            N/A              N/A              N/A               


The status goes N/A on that node only. And I see lot of these error messages in the glusterd log.

[2013-08-20 10:44:05.870842] E [glusterd-geo-rep.c:2535:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master(master), falcon::slave(slave)
[2013-08-20 10:44:09.827399] E [glusterd-geo-rep.c:2535:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master(master), falcon::slave(slave)
[2013-08-20 10:44:13.836479] E [glusterd-geo-rep.c:2535:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master(master), falcon::slave(slave)
[2013-08-20 10:44:17.764012] E [glusterd-geo-rep.c:2535:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master(master), falcon::slave(slave)
[2013-08-20 10:44:21.738387] E [glusterd-geo-rep.c:2535:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master(master), falcon::slave(slave)
[2013-08-20 10:44:25.686593] E [glusterd-geo-rep.c:2535:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master(master), falcon::slave(slave)
[2013-08-20 10:44:29.688960] E [glusterd-geo-rep.c:2535:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master(master), falcon::slave(slave)
[2013-08-20 10:44:33.609573] E [glusterd-geo-rep.c:2535:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master(master), falcon::slave(slave)
[2013-08-20 10:44:37.586113] E [glusterd-geo-rep.c:2535:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master(master), falcon::slave(slave)
[2013-08-20 10:44:41.485594] E [glusterd-geo-rep.c:2535:glusterd_read_status_file] 0-: Unable to fetch extra statusfor master(master), falcon::slave(slave)
Comment 7 Rachana Patel 2013-09-08 09:48:47 EDT
unable to reproduce with 3.4.0.32rhs-1.el6_4.x86_64, hence marking it as verified
Comment 8 Scott Haines 2013-09-23 18:29:53 EDT
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html

Note You need to log in before you can comment on or make changes to this bug.