Bug 855303

Summary: geo-replication with large number of small static files will have E2BIG errors in logs.
Product: [Red Hat Storage] Red Hat Gluster Storage Reporter: Vijaykumar Koppad <vkoppad>
Component: geo-replicationAssignee: Venky Shankar <vshankar>
Status: CLOSED ERRATA QA Contact: Vijaykumar Koppad <vkoppad>
Severity: high Docs Contact:
Priority: unspecified    
Version: 2.0CC: aavati, bbandari, bugzilla, csaba, kkeithle, rhs-bugs, shaines
Target Milestone: ---   
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
: 855306 859173 (view as bug list) Environment:
Last Closed: 2013-09-23 22:38:37 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 855306, 859173    

Description Vijaykumar Koppad 2012-09-07 09:58:22 UTC 
Description of problem:
If the geo-replication started with large number of small static files (around 2000000 files) , you get errors saying E2BIG

Version-Release number of selected component (if applicable):RHS-2.0.z


How reproducible:Higher probabilty if you work with 


Steps to Reproduce:
1.create 2000000 files on the master.
2.Then start the geo-rep session between the master and slave. 
3.Then wait for the E2BIG errors to happen in log files. 
  
Actual results: You get E2BIG errors in geo-rep log files 


Expected results: shouldn't get such errors 


Additional info:
Comment 2 Csaba Henk 2012-10-17 16:51:10 UTC 
Vijaykumar, how does it differ from
https://bugzilla.redhat.com/show_bug.cgi?id=859173 ?
Comment 3 Nagilum 2013-05-14 10:02:42 UTC 
Backporting http://git.gluster.org/?p=glusterfs.git;a=commitdiff;h=7a2362d818baf7cae0ae54ffede436821491c876 
to glusterfs 3.3.1 gives me this patch (use "patch -p0 <resource.py.patch" to apply):

--- /usr/libexec/glusterfs/python/syncdaemon/resource.py        2013-04-30 18:04:24.000000000 +0000
+++ resource.py 2013-05-14 09:51:33.000000000 +0000
@@ -505,8 +505,14 @@
         if not files:
             raise GsyncdError("no files to sync")
         logging.debug("files: " + ", ".join(files))
-        argv = gconf.rsync_command.split() + ['-aRS', '--super', '--numeric-ids'] + files + list(args)
-        po = Popen(argv, stderr=subprocess.PIPE)
+        argv = gconf.rsync_command.split() + \
+               ['-aR0', '--files-from=-', '--super', '--numeric-ids', '--no-implied-dirs'] + \
+               ['.'] + list(args)
+        po = Popen(argv, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+        for f in files:
+            po.stdin.write(f)
+            po.stdin.write('\0')
+        po.stdin.close()
         po.wait()
         po.terminate_geterr(fail_on_err = False)
         return po
Comment 4 Nagilum 2013-05-14 10:15:52 UTC 
Adding kkeithle because since http://koji.fedoraproject.org/koji/packageinfo?packageID=5443 would probably profit from this fix.
Comment 5 Scott Haines 2013-09-23 22:38:37 UTC 
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html
Comment 6 Scott Haines 2013-09-23 22:41:27 UTC 
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html