Bug 855303 - geo-replication with large number of small static files will have E2BIG errors in logs.
Summary: geo-replication with large number of small static files will have E2BIG error...
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Gluster Storage
Classification: Red Hat Storage
Component: geo-replication
Version: 2.0
Hardware: x86_64
OS: Linux
unspecified
high
Target Milestone: ---
: ---
Assignee: Venky Shankar
QA Contact: Vijaykumar Koppad
URL:
Whiteboard:
Depends On:
Blocks: 855306 859173
TreeView+ depends on / blocked
 
Reported: 2012-09-07 09:58 UTC by Vijaykumar Koppad
Modified: 2014-08-25 00:49 UTC (History)
7 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 855306 859173 (view as bug list)
Environment:
Last Closed: 2013-09-23 22:38:37 UTC
Embargoed:


Attachments (Terms of Use)

Description Vijaykumar Koppad 2012-09-07 09:58:22 UTC
Description of problem:
If the geo-replication started with large number of small static files (around 2000000 files) , you get errors saying E2BIG

Version-Release number of selected component (if applicable):RHS-2.0.z


How reproducible:Higher probabilty if you work with 


Steps to Reproduce:
1.create 2000000 files on the master.
2.Then start the geo-rep session between the master and slave. 
3.Then wait for the E2BIG errors to happen in log files. 
  
Actual results: You get E2BIG errors in geo-rep log files 


Expected results: shouldn't get such errors 


Additional info:

Comment 2 Csaba Henk 2012-10-17 16:51:10 UTC
Vijaykumar, how does it differ from
https://bugzilla.redhat.com/show_bug.cgi?id=859173 ?

Comment 3 Nagilum 2013-05-14 10:02:42 UTC
Backporting http://git.gluster.org/?p=glusterfs.git;a=commitdiff;h=7a2362d818baf7cae0ae54ffede436821491c876 
to glusterfs 3.3.1 gives me this patch (use "patch -p0 <resource.py.patch" to apply):

--- /usr/libexec/glusterfs/python/syncdaemon/resource.py        2013-04-30 18:04:24.000000000 +0000
+++ resource.py 2013-05-14 09:51:33.000000000 +0000
@@ -505,8 +505,14 @@
         if not files:
             raise GsyncdError("no files to sync")
         logging.debug("files: " + ", ".join(files))
-        argv = gconf.rsync_command.split() + ['-aRS', '--super', '--numeric-ids'] + files + list(args)
-        po = Popen(argv, stderr=subprocess.PIPE)
+        argv = gconf.rsync_command.split() + \
+               ['-aR0', '--files-from=-', '--super', '--numeric-ids', '--no-implied-dirs'] + \
+               ['.'] + list(args)
+        po = Popen(argv, stdin=subprocess.PIPE, stderr=subprocess.PIPE)
+        for f in files:
+            po.stdin.write(f)
+            po.stdin.write('\0')
+        po.stdin.close()
         po.wait()
         po.terminate_geterr(fail_on_err = False)
         return po

Comment 4 Nagilum 2013-05-14 10:15:52 UTC
Adding kkeithle because since http://koji.fedoraproject.org/koji/packageinfo?packageID=5443 would probably profit from this fix.

Comment 5 Scott Haines 2013-09-23 22:38:37 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html

Comment 6 Scott Haines 2013-09-23 22:41:27 UTC
Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. 

For information on the advisory, and where to find the updated files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2013-1262.html


Note You need to log in before you can comment on or make changes to this bug.