Bug 1303045 - NFS+attach tier:IOs pause for some time during attach tier
NFS+attach tier:IOs pause for some time during attach tier
Status: ASSIGNED
Product: Red Hat Gluster Storage
Classification: Red Hat
Component: tier (Show other bugs)
3.1
Unspecified Unspecified
unspecified Severity unspecified
: ---
: ---
Assigned To: Mohammed Rafi KC
krishnaram Karthick
tier-fuse-nfs-samba
: ZStream
Depends On:
Blocks: 1358586 1268895
  Show dependency treegraph
 
Reported: 2016-01-29 07:15 EST by nchilaka
Modified: 2018-02-07 19:11 EST (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Known Issue
Doc Text:
When a tier is attached while I/O is occurring on an NFS mount, I/O pauses temporarily, usually for between 3 to 5 minutes. If I/O does not resume within 5 minutes, use the 'gluster volume start $VOLNAME force' command to resume I/O without interruption.
Story Points: ---
Clone Of:
: 1358586 (view as bug list)
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description nchilaka 2016-01-29 07:15:25 EST
When we attach a tier while running IOs from NFS mount, there is a temporary pause in the IOs. The pause was generally seen b/w 3-5 min

Raising this bug for tracking purpose
Comment 3 Mohammed Rafi KC 2016-02-12 07:58:52 EST
We have another issue (bug 1306194) related to the same area, So I think it is better to have one doc instead of two.

So the doc would be something like,

When a tier is attached while I/O is occurring on an NFS mount, I/O pauses temporarily, usually for between 3 to 5 minutes. If it is not resuming more than the expected time, executing the command gluster volume start volname force will help you to resume the i/0 with out any i/o interruption.
Comment 5 Mohammed Rafi KC 2016-02-15 00:07:45 EST
Looks good to me.
Comment 6 nchilaka 2016-02-17 01:18:30 EST
the workaround suggested for volume restart incase of IO not resuming works well except a small glitch that can be seen in form  of bug  bz#1309186 - file creates fail with " failed to open '<filename>': Too many levels of symbolic links for file create/write when restarting NFS using vol start force  has been raised
Comment 9 Mohammed Rafi KC 2016-07-11 03:21:22 EDT
I will try the following steps,

1)Reproducing the issue with 3.1.3
2)Measure the difference in time with add-brick on a regular volume and attach-tier on a tier volume
3)try to reproduce the issue with just restarting the gNFS without any attach-tier.
Comment 10 Mohammed Rafi KC 2016-07-21 09:06:07 EDT
Test result for comment9

1) I reproduced this issue in latest master, though it is 100% consistent it is very easy to reproduce.
2)For both attach-tier and add-brick, there is a delay of approx 3mnts to resume an operation. 
3)When I do a simple restart of nfs server, so far I haven't seen a huge delay to resume the operation

Note You need to log in before you can comment on or make changes to this bug.