Bug 764752 (GLUSTER-3020)

Summary: Quota and marker can't coexist in the same process
Product: [Community] GlusterFS Reporter: Jeff Darcy <jdarcy>
Component: quotaAssignee: Junaid <junaid>
Status: CLOSED CURRENTRELEASE QA Contact:
Severity: low Docs Contact:
Priority: medium    
Version: mainlineCC: amarts, gluster-bugs, junaid, saurabh, vagarwal, vijay
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: 3.2.4 Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Jeff Darcy 2011-06-13 17:29:28 UTC
Changing xlator_dynload so it doesn't use RTLD_GLOBAL does *not* fix this, even though it seems like it should.  RTLD_DEEPRESOLVE might work, but it's not portable and its use generally seems to be discouraged for other reasons as well (e.g. incompatibility with malloc wrappers).  Renaming duplicate symbols seems like the best option overall.

Comment 1 Jeff Darcy 2011-06-13 19:41:41 UTC
Since some people might want to enforce quota on the server (enforcing on the client makes the system trivially susceptible to cheating), I tried running the quota translator there.  The result was segfaults in quota_lookup_cbk.  Looking deeper, it turns out that the cutting and pasting from quota to marker has resulted in several symbols with duplicate names:

      2 fini
      2 _fini
      2 init
      2 _init
      2 mem_acct_init
      2 quota_forget*
      2 quota_inode_loc_fill*
      2 quota_local_new*
      2 quota_loc_fill*
      2 reconfigure

Several of these are looked up using dlsym and accessed via the returned pointers, so they function properly despite the duplication.  The ones marked with asterisks are more problematic.  Because of the way we do dynamic loading, marker is loaded first and then quota resolves against the marker versions of the duplicate symbols.  Since e.g. marker's version of quota_local_new is doing completely different initialization on a completely different structure than what callers in quota expect, it should be unsurprising that subsequent use of the returned pointer from within quota results in a segfault.  The only surprise is that we get as far as quota_lookup_cbk before the crash.

Comment 2 Junaid 2011-06-20 12:46:36 UTC
*** Bug 2688 has been marked as a duplicate of this bug. ***

Comment 3 Anand Avati 2011-09-10 03:46:26 UTC
CHANGE: http://review.gluster.com/381 (This is to fix to bug marker translator and quota translator cannot co-exist in same process.) merged in master by Vijay Bellur (vijay)

Comment 4 Saurabh 2011-09-22 10:15:10 UTC
copied the quota information from client vol-file to the server vol-file.

and still able to do the filesystem operations on the mount point

glusterfs#10.1.12.134:one-br on /mnt/glfs type fuse (rw,allow_other,default_permissions,max_read=131072)
[root@Centos1 ~]# cd /mnt/glfs
[root@Centos1 glfs]# ls
a  dir  dir3  kill
[root@Centos1 glfs]# touch abc
[root@Centos1 glfs]# mkdir dir/dir-n
[root@Centos1 glfs]#