Note: This bug is displayed in read-only format because
the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
DescriptionMason Loring Bliss
2022-01-21 22:19:10 UTC
1. Proposed title of this feature request
Improve defaults for sparse file handling.
2. Who is the customer behind the request?
Account: Allianz Technology SE / 888955
TAM customer: yes (Stefan Meyer / EMEA)
CSM customer: (not indicated)
Strategic: (yes)
3. What is the nature and description of the request?
rsync uses inadequate defaults for sparse file buffering, which can be
dramatically improved.
4. Why does the customer need this? (List the business requirements here)
Large file transfers are unacceptably slow without adjusting buffer
handling unfortunately, this requires a recompile.
5. How would the customer like to achieve this? (List the functional
requirements here)
Update buffer sizing. This was explored, for example, in this third-
party bug:
https://bugzilla.samba.org/show_bug.cgi?id=8512
With this:
---------------------------------------------------------------------------
% diff -u rsync.h-org rsync.h
--- rsync.h-org 2014-04-13 19:36:59.000000000 +0200
+++ rsync.h 2014-09-08 16:20:41.427973852 +0200
@@ -131,11 +131,11 @@
#define RSYNC_PORT 873
-#define SPARSE_WRITE_SIZE (1024)
-#define WRITE_SIZE (32*1024)
-#define CHUNK_SIZE (32*1024)
+#define SPARSE_WRITE_SIZE (128*1024)
+#define WRITE_SIZE (128*1024)
+#define CHUNK_SIZE (128*1024)
#define MAX_MAP_SIZE (256*1024)
-#define IO_BUFFER_SIZE (32*1024)
+#define IO_BUFFER_SIZE (128*1024)
#define MAX_BLOCK_SIZE ((int32)1 << 17)
---------------------------------------------------------------------------
With that, the user notes a factor of fifty increase. The customer
tested with only SPARSE_WRITE_SIZE bumped, but not bumped as far:
+#define SPARSE_WRITE_SIZE (32*1024)
...and noted significant performance impact:
---------------------------------------------------------------------------
without "-S"
sent 9,324,792,039 bytes received 42 bytes 90,973,581.28 bytes/sec
unpatched "-S"
sent 9,324,792,039 bytes received 42 bytes 8,190,419.04 bytes/sec
patched with "-S":
sent 9,324,792,039 bytes received 42 bytes 34,345,458.86 bytes/sec
---------------------------------------------------------------------------
6. For each functional requirement listed, specify how Red Hat and the
customer can test to confirm the requirement is successfully implemented.
Customer can verify performance for large sparse transfers.
7. Is there already an existing RFE upstream or in Red Hat Bugzilla?
Not that I've found.
8. Does the customer have any specific timeline dependencies and which
release would they like to target (i.e. RHEL8, RHEL9)?
Not specified.
9. Is the sales team involved in this request and do they have any
additional input?
Not as yet.
10. List any affected packages or components.
rsync
11. Would the customer be able to assist in testing this functionality if
implemented?
Yes.
Sorry for the delay here, I was hoping I would be able to reproduce this but after many attempts I don't see that much of a difference between sparse and non-sparse. Nevermind, the patch for setting up custom sparse-block size is in the upstream repo, only it is in the sort-of-unsupported rscyn-patches. I think it is ok to introduce this as a downstream change since this does not change any default usage and only for users who want to exploit the functionality could this feature make a difference.
Comment 17Frantisek Sumsal
2022-08-05 17:10:30 UTC
I have a suspicion this causes a regression when using rsync with this patch applied against an rsync daemon that doesn't support the --sparse-block option, see https://pagure.io/centos-infra/issue/872.
This change breaks compatibility with older version of rsync. Running rsync from CentOS 8 stream to CentOS 7 is not possible without special parameters.
Versions of components:
rsync-3.1.3-16.el8.x86_64
rsync-3.1.2-10.el7.x86_64
What happens when trying to rsync:
[backuppc@backup ~]$ rsync -vvvv root.sk:/etc/hosts /tmp/test/
cmd=<NULL> machine=devel.xxxx.sk user=root path=/etc/hosts
cmd[0]=ssh cmd[1]=-l cmd[2]=root cmd[3]=devel.xxxx.sk cmd[4]=rsync cmd[5]=--server cmd[6]=--sender cmd[7]=-vvvve.LsfxC cmd[8]=--sparse-block=1024 cmd[9]=. cmd[10]=/etc/hosts
opening connection using: ssh -l root devel.xxxx.sk rsync --server --sender -vvvve.LsfxC --sparse-block=1024 . /etc/hosts (11 args)
msg checking charset: UTF-8
rsync: on remote machine: --sparse-block=1024: unknown option
rsync error: syntax or usage error (code 1) at main.c(1568) [server=3.1.2]
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
[Receiver] _exit_cleanup(code=12, file=io.c, line=226): entered
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.3]
[Receiver] _exit_cleanup(code=12, file=io.c, line=226): about to call exit(12)
It is possible to workaround this problem by adding --sparse-block=0 to rsync parameters, but this is too complicated and breaks some backup programs.
Please, revert this change. Also doesn't help to update rsync on CentOS 7, because there are other distributions, which can use rsync without this parameter.
We (CentOS infra team) also confirm that we're suffering from that issue too, and that impacts the proper mirror workflow for CentOS Stream 8 and 9.
Can that package be removed from CentOS Stream 8 for the time being ?
I can confirm this behaviour.
CentOS Stream 8, rsync local 3.1.3-16
Without having set the opsion --sparse-block, I get this nonetheless:
rsync: on remote machine: --sparse-block=1024: unknown option
rsync error: requested action not supported (code 4) at clientserver.c(1556) [Receiver=3.0.9]
rsync: read error: Connection reset by peer (104)
rsync error: error in socket IO (code 10) at io.c(785) [sender=3.1.3]
Hi,
I am putting this here as a public comment - we know about the issue and the fix is already out - rsync-3.1.3-17.el8. You can either downgrade to rsync-3.1.3-15.el8 where the feature is not present or wait till the -17 version gets to the CentOS.
build: https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=2114622
I know this affects a lot of systems, no need to add more and more systems that are affected here as comments.
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.
For information on the advisory (Moderate: rsync security and enhancement update), and where to find the updated
files, follow the link below.
If the solution does not work for you, open a new bug report.
https://access.redhat.com/errata/RHSA-2022:7793
1. Proposed title of this feature request Improve defaults for sparse file handling. 2. Who is the customer behind the request? Account: Allianz Technology SE / 888955 TAM customer: yes (Stefan Meyer / EMEA) CSM customer: (not indicated) Strategic: (yes) 3. What is the nature and description of the request? rsync uses inadequate defaults for sparse file buffering, which can be dramatically improved. 4. Why does the customer need this? (List the business requirements here) Large file transfers are unacceptably slow without adjusting buffer handling unfortunately, this requires a recompile. 5. How would the customer like to achieve this? (List the functional requirements here) Update buffer sizing. This was explored, for example, in this third- party bug: https://bugzilla.samba.org/show_bug.cgi?id=8512 With this: --------------------------------------------------------------------------- % diff -u rsync.h-org rsync.h --- rsync.h-org 2014-04-13 19:36:59.000000000 +0200 +++ rsync.h 2014-09-08 16:20:41.427973852 +0200 @@ -131,11 +131,11 @@ #define RSYNC_PORT 873 -#define SPARSE_WRITE_SIZE (1024) -#define WRITE_SIZE (32*1024) -#define CHUNK_SIZE (32*1024) +#define SPARSE_WRITE_SIZE (128*1024) +#define WRITE_SIZE (128*1024) +#define CHUNK_SIZE (128*1024) #define MAX_MAP_SIZE (256*1024) -#define IO_BUFFER_SIZE (32*1024) +#define IO_BUFFER_SIZE (128*1024) #define MAX_BLOCK_SIZE ((int32)1 << 17) --------------------------------------------------------------------------- With that, the user notes a factor of fifty increase. The customer tested with only SPARSE_WRITE_SIZE bumped, but not bumped as far: +#define SPARSE_WRITE_SIZE (32*1024) ...and noted significant performance impact: --------------------------------------------------------------------------- without "-S" sent 9,324,792,039 bytes received 42 bytes 90,973,581.28 bytes/sec unpatched "-S" sent 9,324,792,039 bytes received 42 bytes 8,190,419.04 bytes/sec patched with "-S": sent 9,324,792,039 bytes received 42 bytes 34,345,458.86 bytes/sec --------------------------------------------------------------------------- 6. For each functional requirement listed, specify how Red Hat and the customer can test to confirm the requirement is successfully implemented. Customer can verify performance for large sparse transfers. 7. Is there already an existing RFE upstream or in Red Hat Bugzilla? Not that I've found. 8. Does the customer have any specific timeline dependencies and which release would they like to target (i.e. RHEL8, RHEL9)? Not specified. 9. Is the sales team involved in this request and do they have any additional input? Not as yet. 10. List any affected packages or components. rsync 11. Would the customer be able to assist in testing this functionality if implemented? Yes.