| Summary: | Inode leaks found in data-self-heal | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Product: | Red Hat Gluster Storage | Reporter: | Pranith Kumar K <pkarampu> | ||||||
| Component: | replicate | Assignee: | Pranith Kumar K <pkarampu> | ||||||
| Status: | CLOSED ERRATA | QA Contact: | Nag Pavan Chilakam <nchilaka> | ||||||
| Severity: | unspecified | Docs Contact: | |||||||
| Priority: | unspecified | ||||||||
| Version: | rhgs-3.1 | CC: | asrivast, bugs, mchangir, ravishankar, rhinduja, rhs-bugs, storage-qa-internal | ||||||
| Target Milestone: | --- | Keywords: | Regression, ZStream | ||||||
| Target Release: | RHGS 3.1.3 | ||||||||
| Hardware: | Unspecified | ||||||||
| OS: | Unspecified | ||||||||
| Whiteboard: | |||||||||
| Fixed In Version: | glusterfs-3.7.9-3 | Doc Type: | Bug Fix | ||||||
| Doc Text: | Story Points: | --- | |||||||
| Clone Of: | 1329773 | Environment: | |||||||
| Last Closed: | 2016-06-23 05:19:54 UTC | Type: | Bug | ||||||
| Regression: | --- | Mount Type: | --- | ||||||
| Documentation: | --- | CRM: | |||||||
| Verified Versions: | Category: | --- | |||||||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||
| Cloudforms Team: | --- | Target Upstream Version: | |||||||
| Bug Depends On: | 1329773 | ||||||||
| Bug Blocks: | 1311817, 1329779 | ||||||||
| Attachments: |
|
||||||||
|
Description
Pranith Kumar K
2016-04-27 08:38:27 UTC
As there is no specific way of testing this functionally.
Talked with dev and did below testing.
Moving to verified with the available information
Had a 6 node setup
created 1x2 volume
fuse mnted the volume and created some files and copied linux tar
brought down brick-0
from two different mounts did following IOs
scped video files from my laptop to the volume mount
untarred kernel
In all had about 40GB of data to heal
Started the heal by bringing up the brick using force start.
While heal was going on continously(with sleep of 10s) issued heal info
Also from one mount in a loop used dd to create 400MB size files as long as heal is happening(crreated about 42 files)
the heal took about 1Hour to complete
I noticed the shd process memory consumption of both the bricks and didnt see any change in consumption. On an avg the cpu consumption from top command showed 1.1% usage
I noticed the shd process cpu usage of the source was b/w 50-90% during the healing
In 1x2 Volume:
Ran healinfo in loop while actual heal was going on
to test 1330881 - Inode leaks found in data-self-heal
there was about 40GB of data to be healed(one untarred lin kernel and others being folders containing many big video files
took about 1 Hr to heal complete data:
##############################
[root@dhcp35-191 ~]# gluster v status olia
Status of volume: olia
Gluster process TCP Port RDMA Port Online Pid
------------------------------------------------------------------------------
Brick 10.70.35.98:/rhs/brick4/olia N/A N/A N N/A
Brick 10.70.35.64:/rhs/brick4/olia 49170 0 Y 6547
NFS Server on localhost 2049 0 Y 20318
Self-heal Daemon on localhost N/A N/A Y 20326
NFS Server on 10.70.35.27 2049 0 Y 5753
Self-heal Daemon on 10.70.35.27 N/A N/A Y 5761
NFS Server on 10.70.35.114 2049 0 Y 5869
Self-heal Daemon on 10.70.35.114 N/A N/A Y 5877
NFS Server on 10.70.35.44 2049 0 Y 32066
Self-heal Daemon on 10.70.35.44 N/A N/A Y 32074
NFS Server on 10.70.35.98 2049 0 Y 4823
Self-heal Daemon on 10.70.35.98 N/A N/A Y 4832
NFS Server on 10.70.35.64 2049 0 Y 6574
Self-heal Daemon on 10.70.35.64 N/A N/A Y 6583
Task Status of Volume olia
------------------------------------------------------------------------------
There are no active volume tasks
Volume Name: olia
Type: Replicate
Volume ID: 6ad242d2-b0cb-441d-97a7-8fa2db693e05
Status: Started
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: 10.70.35.98:/rhs/brick4/olia
Brick2: 10.70.35.64:/rhs/brick4/olia
Options Reconfigured:
diagnostics.count-fop-hits: on
diagnostics.latency-measurement: on
performance.readdir-ahead: on
(turned on profiling)
Created attachment 1154497 [details]
qe validation logs
Created attachment 1154498 [details]
qe validation logs#2
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2016:1240 |