RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.
Bug 2043753 - [RFE] Improve defaults for sparse file buffering.
Summary: [RFE] Improve defaults for sparse file buffering.
Keywords:
Status: CLOSED ERRATA
Alias: None
Deadline: 2022-08-08
Product: Red Hat Enterprise Linux 8
Classification: Red Hat
Component: rsync
Version: 8.5
Hardware: All
OS: Linux
unspecified
medium
Target Milestone: rc
: ---
Assignee: Michal Ruprich
QA Contact: Ondrej Mejzlik
URL:
Whiteboard:
Depends On:
Blocks: 2089955
TreeView+ depends on / blocked
 
Reported: 2022-01-21 22:19 UTC by Mason Loring Bliss
Modified: 2022-11-08 12:52 UTC (History)
15 users (show)

Fixed In Version: rsync-3.1.3-17.el8
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2022-11-08 10:55:36 UTC
Type: Bug
Target Upstream Version:
Embargoed:


Attachments (Terms of Use)


Links
System ID Private Priority Status Summary Last Updated
Red Hat Issue Tracker RHELPLAN-109173 0 None None None 2022-01-21 22:23:03 UTC
Red Hat Product Errata RHSA-2022:7793 0 None None None 2022-11-08 10:56:06 UTC

Description Mason Loring Bliss 2022-01-21 22:19:10 UTC
1. Proposed title of this feature request

    Improve defaults for sparse file handling.

2. Who is the customer behind the request?

    Account: Allianz Technology SE / 888955
    TAM customer: yes (Stefan Meyer / EMEA)
    CSM customer: (not indicated)
    Strategic: (yes)

3. What is the nature and description of the request?

    rsync uses inadequate defaults for sparse file buffering, which can be
    dramatically improved.

4. Why does the customer need this? (List the business requirements here)

    Large file transfers are unacceptably slow without adjusting buffer
    handling unfortunately, this requires a recompile.


5. How would the customer like to achieve this? (List the functional
requirements here)

    Update buffer sizing. This was explored, for example, in this third-
    party bug:

        https://bugzilla.samba.org/show_bug.cgi?id=8512

    With this:

---------------------------------------------------------------------------
% diff -u rsync.h-org rsync.h
--- rsync.h-org 2014-04-13 19:36:59.000000000 +0200
+++ rsync.h     2014-09-08 16:20:41.427973852 +0200
@@ -131,11 +131,11 @@
 
 #define RSYNC_PORT 873
 
-#define SPARSE_WRITE_SIZE (1024)
-#define WRITE_SIZE (32*1024)
-#define CHUNK_SIZE (32*1024)
+#define SPARSE_WRITE_SIZE (128*1024)
+#define WRITE_SIZE (128*1024)
+#define CHUNK_SIZE (128*1024)
 #define MAX_MAP_SIZE (256*1024)
-#define IO_BUFFER_SIZE (32*1024)
+#define IO_BUFFER_SIZE (128*1024)
 #define MAX_BLOCK_SIZE ((int32)1 << 17)
---------------------------------------------------------------------------

    With that, the user notes a factor of fifty increase. The customer
    tested with only SPARSE_WRITE_SIZE bumped, but not bumped as far:

        +#define SPARSE_WRITE_SIZE (32*1024)

    ...and noted significant performance impact:

---------------------------------------------------------------------------
without "-S"
sent 9,324,792,039 bytes  received 42 bytes  90,973,581.28 bytes/sec

unpatched "-S"
sent 9,324,792,039 bytes  received 42 bytes  8,190,419.04 bytes/sec

patched with "-S":
sent 9,324,792,039 bytes  received 42 bytes  34,345,458.86 bytes/sec
---------------------------------------------------------------------------

6. For each functional requirement listed, specify how Red Hat and the
customer can test to confirm the requirement is successfully implemented.

    Customer can verify performance for large sparse transfers.

7. Is there already an existing RFE upstream or in Red Hat Bugzilla?

    Not that I've found.

8. Does the customer have any specific timeline dependencies and which
release would they like to target (i.e. RHEL8, RHEL9)?

    Not specified.

9. Is the sales team involved in this request and do they have any
additional input?

    Not as yet.

10. List any affected packages or components.

    rsync

11. Would the customer be able to assist in testing this functionality if
implemented?

    Yes.

Comment 2 Michal Ruprich 2022-05-13 13:20:11 UTC
Sorry for the delay here, I was hoping I would be able to reproduce this but after many attempts I don't see that much of a difference between sparse and non-sparse. Nevermind, the patch for setting up custom sparse-block size is in the upstream repo, only it is in the sort-of-unsupported rscyn-patches. I think it is ok to introduce this as a downstream change since this does not change any default usage and only for users who want to exploit the functionality could this feature make a difference.

Comment 17 Frantisek Sumsal 2022-08-05 17:10:30 UTC
I have a suspicion this causes a regression when using rsync with this patch applied against an rsync daemon that doesn't support the --sparse-block option, see https://pagure.io/centos-infra/issue/872.

Comment 18 Jan ONDREJ 2022-08-06 04:05:17 UTC
This change breaks compatibility with older version of rsync. Running rsync from CentOS 8 stream to CentOS 7 is not possible without special parameters.

Versions of components:
rsync-3.1.3-16.el8.x86_64
rsync-3.1.2-10.el7.x86_64

What happens when trying to rsync:
[backuppc@backup ~]$ rsync -vvvv root.sk:/etc/hosts /tmp/test/
cmd=<NULL> machine=devel.xxxx.sk user=root path=/etc/hosts
cmd[0]=ssh cmd[1]=-l cmd[2]=root cmd[3]=devel.xxxx.sk cmd[4]=rsync cmd[5]=--server cmd[6]=--sender cmd[7]=-vvvve.LsfxC cmd[8]=--sparse-block=1024 cmd[9]=. cmd[10]=/etc/hosts
opening connection using: ssh -l root devel.xxxx.sk rsync --server --sender -vvvve.LsfxC --sparse-block=1024 . /etc/hosts  (11 args)
msg checking charset: UTF-8
rsync: on remote machine: --sparse-block=1024: unknown option
rsync error: syntax or usage error (code 1) at main.c(1568) [server=3.1.2]
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
[Receiver] _exit_cleanup(code=12, file=io.c, line=226): entered
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.3]
[Receiver] _exit_cleanup(code=12, file=io.c, line=226): about to call exit(12)

It is possible to workaround this problem by adding --sparse-block=0 to rsync parameters, but this is too complicated and breaks some backup programs.

Please, revert this change. Also doesn't help to update rsync on CentOS 7, because there are other distributions, which can use rsync without this parameter.

Comment 19 Michal Ruprich 2022-08-06 09:22:22 UTC
I did not notice that this option is sent to the server every time. I am now working on a fix.

Comment 22 farrotin 2022-08-08 08:29:24 UTC
We (CentOS infra team) also confirm that we're suffering from that issue too, and that impacts the proper mirror workflow for CentOS Stream 8 and 9.
Can that package be removed from CentOS Stream 8 for the time being ?

Comment 27 Leon Fauster 2022-08-09 13:51:24 UTC
+1 also affected

Comment 28 rhn 2022-08-09 14:08:07 UTC
I can confirm this behaviour.

CentOS Stream 8, rsync local 3.1.3-16

Without having set the opsion --sparse-block, I get this nonetheless:

rsync: on remote machine: --sparse-block=1024: unknown option
rsync error: requested action not supported (code 4) at clientserver.c(1556) [Receiver=3.0.9]
rsync: read error: Connection reset by peer (104)
rsync error: error in socket IO (code 10) at io.c(785) [sender=3.1.3]

Comment 29 Marios Andreou 2022-08-10 06:36:17 UTC
adding info as i don't see it on the BZ already

this is hitting tripleo image build jobs on centos8 - upstream bug at https://bugs.launchpad.net/tripleo/+bug/1984035

workaround for now is to downgrade rsync to 3.1.3-14 https://review.rdoproject.org/r/c/config/+/44471

Comment 30 Michal Ruprich 2022-08-10 07:01:30 UTC
Hi,

I am putting this here as a public comment - we know about the issue and the fix is already out - rsync-3.1.3-17.el8. You can either downgrade to rsync-3.1.3-15.el8 where the feature is not present or wait till the -17 version gets to the CentOS.

build: https://brewweb.engineering.redhat.com/brew/buildinfo?buildID=2114622

I know this affects a lot of systems, no need to add more and more systems that are affected here as comments.

Comment 35 errata-xmlrpc 2022-11-08 10:55:36 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory (Moderate: rsync security and enhancement update), and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2022:7793


Note You need to log in before you can comment on or make changes to this bug.