Bug 1285829 - AFR: 3-way-replication: Transport point not connected error messaged not displayed when one of the replica pair is down
AFR: 3-way-replication: Transport point not connected error messaged not disp...
Status: CLOSED CURRENTRELEASE
Product: GlusterFS
Classification: Community
Component: distribute (Show other bugs)
3.7.7
x86_64 Linux
unspecified Severity high
: ---
: ---
Assigned To: Sakshi
:
: 1308413 (view as bug list)
Depends On: 1185173
Blocks: 1279705 1308413
  Show dependency treegraph
 
Reported: 2015-11-26 10:21 EST by Sakshi
Modified: 2016-07-31 21:22 EDT (History)
7 users (show)

See Also:
Fixed In Version: glusterfs-3.7.9
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1185173
Environment:
Last Closed: 2016-04-15 05:48:28 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Sakshi 2015-11-26 10:21:45 EST
+++ This bug was initially created as a clone of Bug #1185173 +++

Description of problem:

When one of the replica pair is down, creating file from mount point should report transport point not connected error message 

Version-Release number of selected component (if applicable):

glusterfs 3.6.0.42

How reproducible:

100%

Steps to Reproduce:
1.create 2x3 distribute replicate volume
2.kill one of the replica pair of the volume
3.create files from mount point

Actual results:

[root@client glusterfs]# touch file{1..100}
touch: cannot touch `file3': No such file or directory
touch: cannot touch `file4': No such file or directory
touch: cannot touch `file7': No such file or directory

===============================
Mount logs /var/log/glusterfs/mnt-glusterfs-.log

[2015-01-23 11:52:17.053399] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1401870546
[2015-01-23 11:52:17.055207] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1401870546
[2015-01-23 11:52:17.055322] W [fuse-bridge.c:1937:fuse_create_cbk] 0-glusterfs-fuse: 13025: /file91 => -1 (No such file or directory)
[2015-01-23 11:52:17.055600] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1401870546
[2015-01-23 11:52:17.057228] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1401870546
[2015-01-23 11:52:17.080706] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1742334698
[2015-01-23 11:52:17.082387] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1742334698
[2015-01-23 11:52:17.084435] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1742334698
[2015-01-23 11:52:17.086271] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1742334698
[2015-01-23 11:52:17.086388] W [fuse-bridge.c:1937:fuse_create_cbk] 0-glusterfs-fuse: 13034: /file93 => -1 (No such file or directory)
[2015-01-23 11:52:17.086648] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1742334698
[2015-01-23 11:52:17.088220] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1742334698
[2015-01-23 11:52:17.156335] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1752000159
[2015-01-23 11:52:17.158209] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1752000159
[2015-01-23 11:52:17.160406] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1752000159
[2015-01-23 11:52:17.162200] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1752000159
[2015-01-23 11:52:17.162299] W [fuse-bridge.c:1937:fuse_create_cbk] 0-glusterfs-fuse: 13055: /file97 => -1 (No such file or directory)
[2015-01-23 11:52:17.162625] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1752000159
[2015-01-23 11:52:17.164215] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1752000159
[2015-01-23 11:52:17.209093] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1194566581
[2015-01-23 11:52:17.211148] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1194566581
[2015-01-23 11:52:17.213408] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1194566581
[2015-01-23 11:52:17.215112] W [dht-layout.c:180:dht_layout_search] 0-testvol-dht: no subvolume for hash (value) = 1194566581
[2015-01-23 11:52:17.215210] W [fuse-bridge.c:1937:fuse_create_cbk] 0-glusterfs-fuse: 13070: /file100 => -1 (No such file or directory)

=============================================================

Expected results:

Transport point not connected message should be displayed for the down replica pair 

Additional info:

Volume Name: testvol
Type: Distributed-Replicate
Volume ID: 7e154f4b-7ded-42f0-9a02-2d0ff734414b
Status: Started
Snap Volume: no
Number of Bricks: 2 x 3 = 6
Transport-type: tcp
Bricks:
Brick1: 10.70.47.143:/rhs/brick1/b1
Brick2: 10.70.47.145:/rhs/brick1/b2
Brick3: 10.70.47.150:/rhs/brick1/b3
Brick4: 10.70.47.151:/rhs/brick1/b4
Brick5: 10.70.47.143:/rhs/brick2/b5
Brick6: 10.70.47.145:/rhs/brick2/b6
Options Reconfigured:
performance.readdir-ahead: on
auto-delete: disable
snap-max-soft-limit: 90
snap-max-hard-limit: 256

==================================================

[root@node1 ~]# gluster v status testvol
Status of volume: testvol
Gluster process						Port	Online	Pid
------------------------------------------------------------------------------
Brick 10.70.47.143:/rhs/brick1/b1			N/A	N	N/A
Brick 10.70.47.145:/rhs/brick1/b2			N/A	N	N/A
Brick 10.70.47.150:/rhs/brick1/b3			N/A	N	N/A
Brick 10.70.47.151:/rhs/brick1/b4			49152	Y	7687
Brick 10.70.47.143:/rhs/brick2/b5			49153	Y	2463
Brick 10.70.47.145:/rhs/brick2/b6			49153	Y	3275
NFS Server on localhost					2049	Y	3047
Self-heal Daemon on localhost				N/A	Y	3004
NFS Server on 10.70.47.145				2049	Y	3762
Self-heal Daemon on 10.70.47.145			N/A	Y	3710
NFS Server on 10.70.47.151				2049	Y	8169
Self-heal Daemon on 10.70.47.151			N/A	Y	8108
NFS Server on 10.70.47.150				2049	Y	23171
Self-heal Daemon on 10.70.47.150			N/A	Y	23093
 
Task Status of Volume testvol
------------------------------------------------------------------------------
There are no active volume tasks


--- Additional comment from Pranith Kumar K on 2015-03-18 01:13:24 EDT ---

The problem seems to be with the errno
Comment 1 Vijay Bellur 2016-02-04 04:57:51 EST
REVIEW: http://review.gluster.org/13351 (dht: set proper errno when hashed subvol is not found) posted (#1) for review on release-3.7 by Sakshi Bansal
Comment 2 Sakshi 2016-02-15 01:41:53 EST
*** Bug 1308413 has been marked as a duplicate of this bug. ***
Comment 3 Vijay Bellur 2016-02-19 01:41:51 EST
REVIEW: http://review.gluster.org/13351 (dht: set proper errno when hashed subvol is not found) posted (#2) for review on release-3.7 by Sakshi Bansal
Comment 4 Vijay Bellur 2016-02-22 01:11:02 EST
COMMIT: http://review.gluster.org/13351 committed in release-3.7 by Raghavendra G (rgowdapp@redhat.com) 
------
commit e6d8e4fb469edf0e58b14ee02abb1ecd625eb48d
Author: Sakshi Bansal <sabansal@redhat.com>
Date:   Tue Nov 10 10:27:22 2015 +0530

    dht: set proper errno when hashed subvol is not found
    
    Backport of http://review.gluster.org/#/c/12559/
    
    > Change-Id: I0c4c72e2f5a9f8a7c60ef65251c596b54de89479
    > BUG: 1279705
    > Signed-off-by: Sakshi Bansal <sabansal@redhat.com>
    > Reviewed-on: http://review.gluster.org/12559
    > Tested-by: NetBSD Build System <jenkins@build.gluster.org>
    > Reviewed-by: Susant Palai <spalai@redhat.com>
    > Tested-by: Gluster Build System <jenkins@build.gluster.com>
    > Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
    
    Change-Id: I0c4c72e2f5a9f8a7c60ef65251c596b54de89479
    BUG: 1285829
    Signed-off-by: Sakshi Bansal <sabansal@redhat.com>
    Reviewed-on: http://review.gluster.org/13351
    CentOS-regression: Gluster Build System <jenkins@build.gluster.com>
    Smoke: Gluster Build System <jenkins@build.gluster.com>
    NetBSD-regression: NetBSD Build System <jenkins@build.gluster.org>
    Reviewed-by: Raghavendra G <rgowdapp@redhat.com>
    Tested-by: Raghavendra G <rgowdapp@redhat.com>
Comment 5 Kaushal 2016-04-19 03:24:46 EDT
This bug is getting closed because a release has been made available that should address the reported issue. In case the problem is still not fixed with glusterfs-3.7.9, please open a new bug report.

glusterfs-3.7.9 has been announced on the Gluster mailinglists [1], packages for several distributions should become available in the near future. Keep an eye on the Gluster Users mailinglist [2] and the update infrastructure for your distribution.

[1] https://www.gluster.org/pipermail/gluster-users/2016-March/025922.html
[2] http://thread.gmane.org/gmane.comp.file-systems.gluster.user

Note You need to log in before you can comment on or make changes to this bug.