Bug 1409808
| Summary: | [Mdcache] clients being served wrong information about a file, can lead to file inconsistency | |||
|---|---|---|---|---|
| Product: | [Red Hat Storage] Red Hat Gluster Storage | Reporter: | Nag Pavan Chilakam <nchilaka> | |
| Component: | disperse | Assignee: | Poornima G <pgurusid> | |
| Status: | CLOSED ERRATA | QA Contact: | Nag Pavan Chilakam <nchilaka> | |
| Severity: | urgent | Docs Contact: | ||
| Priority: | unspecified | |||
| Version: | rhgs-3.2 | CC: | amukherj, rcyriac, rhinduja, rhs-bugs, storage-qa-internal | |
| Target Milestone: | --- | |||
| Target Release: | RHGS 3.2.0 | |||
| Hardware: | Unspecified | |||
| OS: | Unspecified | |||
| Whiteboard: | ||||
| Fixed In Version: | glusterfs-3.8.4-11 | Doc Type: | If docs needed, set a value | |
| Doc Text: | Story Points: | --- | ||
| Clone Of: | ||||
| : | 1410375 (view as bug list) | Environment: | ||
| Last Closed: | 2017-03-23 06:01:38 UTC | Type: | Bug | |
| Regression: | --- | Mount Type: | --- | |
| Documentation: | --- | CRM: | ||
| Verified Versions: | Category: | --- | ||
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
| Cloudforms Team: | --- | Target Upstream Version: | ||
| Embargoed: | ||||
| Bug Depends On: | ||||
| Bug Blocks: | 1351528, 1410375, 1410688 | |||
|
Description
Nag Pavan Chilakam
2017-01-03 13:44:25 UTC
I disable mdcache settings after which I don't see this problem hence seems to be with all likeliness mdcache issue I am seeing this issue even with fuse protocol, hence changing the title to something more generic(removing ganesha) This is seen with mdcache settings only Also, note that the reason it can skip someone's eye or observation with fuse protocol, is that, whenever we do the lookup say ll of the file, it gets the update information of the file(while dd is happening from other client) in fuse mount. However once file is completed with dd , then only the ll reflects to only the size of file on one brick . Whereas in ganesha, the ll doesnt always serve the an updated size information, probably with ganesha caching related stuff. Hence it is more easy for human eye to catch the discrepancy. upstream mainline patch http://review.gluster.org/16329 posted for review. QATP: ran the testcase which was mentioned while raising the bug, the case passed Other cases: saw a problem with truncate of file fixing atime as epoch starting time, raised a new bug https://bugzilla.redhat.com/show_bug.cgi?id=1411277 truncate setting accesstime of a file (atime) to unix epoch beginning time(01-01-1970) Also, noticed that there were some atime inconsistencies, like below on ganesha setup,with mdcache settings enabled when we access the file, the atime changes, on the local machine, but is not saved on the brick side, hence the atime is lost later. I will discuss and raise a new bug accordingly Moving to verified [root@dhcp35-37 ~]# gluster v status disperse Status of volume: disperse Gluster process TCP Port RDMA Port Online Pid ------------------------------------------------------------------------------ Brick 10.70.35.37:/rhs/brick2/disperse 49153 0 Y 20857 Brick 10.70.35.116:/rhs/brick2/disperse 49153 0 Y 18063 Brick 10.70.35.239:/rhs/brick2/disperse 49153 0 Y 18140 Brick 10.70.35.135:/rhs/brick2/disperse 49154 0 Y 4732 Brick 10.70.35.8:/rhs/brick2/disperse 49154 0 Y 4737 Brick 10.70.35.196:/rhs/brick2/disperse 49152 0 Y 17851 Brick 10.70.35.37:/rhs/brick3/disperse 49154 0 Y 20877 Brick 10.70.35.116:/rhs/brick3/disperse 49154 0 Y 18083 Brick 10.70.35.239:/rhs/brick3/disperse 49154 0 Y 18160 Brick 10.70.35.135:/rhs/brick3/disperse 49155 0 Y 4752 Brick 10.70.35.8:/rhs/brick3/disperse 49155 0 Y 4757 Brick 10.70.35.196:/rhs/brick3/disperse 49153 0 Y 17871 Self-heal Daemon on localhost N/A N/A Y 8229 Self-heal Daemon on 10.70.35.135 N/A N/A Y 5022 Self-heal Daemon on 10.70.35.196 N/A N/A Y 5010 Self-heal Daemon on 10.70.35.239 N/A N/A Y 5265 Self-heal Daemon on 10.70.35.8 N/A N/A Y 5042 Self-heal Daemon on 10.70.35.116 N/A N/A Y 5273 Task Status of Volume disperse ------------------------------------------------------------------------------ There are no active volume tasks [root@dhcp35-37 ~]# gluster v info disperse Volume Name: disperse Type: Distributed-Disperse Volume ID: 53f82e48-d215-4301-be59-5f3e58f8b846 Status: Started Snapshot Count: 0 Number of Bricks: 2 x (4 + 2) = 12 Transport-type: tcp Bricks: Brick1: 10.70.35.37:/rhs/brick2/disperse Brick2: 10.70.35.116:/rhs/brick2/disperse Brick3: 10.70.35.239:/rhs/brick2/disperse Brick4: 10.70.35.135:/rhs/brick2/disperse Brick5: 10.70.35.8:/rhs/brick2/disperse Brick6: 10.70.35.196:/rhs/brick2/disperse Brick7: 10.70.35.37:/rhs/brick3/disperse Brick8: 10.70.35.116:/rhs/brick3/disperse Brick9: 10.70.35.239:/rhs/brick3/disperse Brick10: 10.70.35.135:/rhs/brick3/disperse Brick11: 10.70.35.8:/rhs/brick3/disperse Brick12: 10.70.35.196:/rhs/brick3/disperse Options Reconfigured: cluster.shd-max-threads: 5 performance.md-cache-timeout: 600 performance.cache-invalidation: on performance.stat-prefetch: on features.cache-invalidation-timeout: 600 ganesha.enable: on features.cache-invalidation: off transport.address-family: inet performance.readdir-ahead: on nfs.disable: on nfs-ganesha: enable cluster.enable-shared-storage: enable [root@dhcp35-37 ~]# rpm -qa|egrep "ganesha|gluster" glusterfs-events-3.8.4-11.el7rhgs.x86_64 glusterfs-rdma-3.8.4-11.el7rhgs.x86_64 glusterfs-api-3.8.4-11.el7rhgs.x86_64 glusterfs-server-3.8.4-11.el7rhgs.x86_64 nfs-ganesha-gluster-2.4.1-3.el7rhgs.x86_64 nfs-ganesha-debuginfo-2.4.1-3.el7rhgs.x86_64 glusterfs-libs-3.8.4-11.el7rhgs.x86_64 glusterfs-cli-3.8.4-11.el7rhgs.x86_64 nfs-ganesha-2.4.1-3.el7rhgs.x86_64 glusterfs-3.8.4-11.el7rhgs.x86_64 glusterfs-fuse-3.8.4-11.el7rhgs.x86_64 glusterfs-debuginfo-3.8.4-11.el7rhgs.x86_64 glusterfs-client-xlators-3.8.4-11.el7rhgs.x86_64 python-gluster-3.8.4-11.el7rhgs.noarch glusterfs-ganesha-3.8.4-11.el7rhgs.x86_64 glusterfs-geo-replication-3.8.4-11.el7rhgs.x86_64 [root@dhcp35-37 ~]# TestCaseId:RHG3-11784 Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://rhn.redhat.com/errata/RHSA-2017-0486.html |