Bug 1432899

Summary: [RFE] Rebase rsync to version 3.1.2
Product: Red Hat Enterprise Linux 7 Reporter: Kotresh HR <khiremat>
Component: rsyncAssignee: Michal Ruprich <mruprich>
Status: CLOSED ERRATA QA Contact: Martin Zelený <mzeleny>
Severity: urgent Docs Contact: Lenka Špačková <lkuprova>
Priority: medium    
Version: 7.4CC: carl, khiremat, ksrot, miabbott, Michael.Fenn, mruprich, omoris, pasik, rcyriac, salmy, thozza, walters
Target Milestone: rcKeywords: FutureFeature, Rebase
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: rsync-3.1.2-1.el7 Doc Type: Release Note
Doc Text:
_rsync_ rebased to version 3.1.2 The _rsync_ packages have been upgraded to upstream version 3.1.2, which provides a number of bug fixes and enhancements over the previous version. This update introduces the following output changes: * The default output format of numbers has been changed to 3-digit groups, for example, `1,234,567`. * The output of the "--progress" option has been changed; the following strings have been shortened: `xfer` to `xfr`, and `to-check` to `to-chk`. Notable enhancements in this version include: * I/O handling has been improved, which results in faster data transfers. * New "--info" and "--debug" options have been added for more fine-grained output. * The ability to synchronize nano-second modified times has been added. * New options, "--usermap", "--groupmap", and "--chown", have been added for manipulating file ownership during the copy operation. * A new "--preallocate" option has been added.
Story Points: ---
Clone Of:
: 1439072 (view as bug list) Environment:
Last Closed: 2018-04-10 10:06:32 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1212546, 1141673, 1388569, 1393543, 1420851, 1465928, 1491582    

Description Kotresh HR 2017-03-16 11:01:21 UTC
Description of problem:

RHEL doesn't have latest rsync version available (>=3.1.0).
The rsync version in RHEL is "3.0.9".

In RHGS, geo-replication extensively uses rsync to sync files between master volume and slave volume. Rsync version > 3.1.0 has lot of performance improvements and it could significantly benefit RHGS customers in syncing data faster.

Check the improvements here:
https://download.samba.org/pub/rsync/src/rsync-3.1.0-NEWS

One of the use cases that we have seen where it significantly performs better is that, it provides new option "--ignore-missing-args" which ignore the errors on source like ENOENT and hence saves lot of retries in geo-replication which otherwise involved multiple retries before failure. This is a common scenarios where the workload involves lots of UNLINKS and CREATES.


Version-Release number of selected component (if applicable):
RHEL 7.4

How reproducible:
Always

Steps to Reproduce:
1. 
2.
3.

Actual results:
Available rsync version is 3.0.9

Expected results:
RHGS requires rsync version to be > 3.1.0 to perform better.


Additional info:

Comment 2 Kotresh HR 2017-03-16 11:14:02 UTC
Hi Steve Almy,

Please provide pm_ack if this request is feasible to be addressed.

Comment 12 Tomáš Hozza 2017-07-21 14:19:18 UTC
*** Bug 1141673 has been marked as a duplicate of this bug. ***

Comment 21 Martin Zelený 2017-09-05 08:27:07 UTC
We will be rebasing to rsync version 3.1.2

Comment 23 Colin Walters 2017-10-09 16:48:58 UTC
This reintroduces a dependency on perl, which we would really like not to have for RHEL Atomic Host.

I believe it's due to this change:

-chmod -x support/*

Which if you look at `git annotate` was introduced for a reason.

commit 67362e2c3e4f363e3080a8a4f9465e8b7408225e
Author:     Simo Sorce <simo>
AuthorDate: Fri Apr 4 21:10:36 2008 +0000
Commit:     Simo Sorce <simo>
CommitDate: Fri Apr 4 21:10:36 2008 +0000

    Fix example executable to not generate dependencies

Comment 24 Michal Ruprich 2017-10-10 06:01:47 UTC
Hi Colin,

I tried to come up with a 'cleaner' way to remove the executable permissions on the support dir but to be honest I didn't notice that this line was there due to perl dependencies. Is perl as a dependency bad on Atomic Host?

Comment 25 Colin Walters 2017-10-10 13:46:46 UTC
(In reply to Michal Ruprich from comment #24)
> Is perl as a dependency bad on Atomic Host?

Yes.  We're trying to make it smaller.

Comment 26 Michal Ruprich 2017-10-10 13:56:46 UTC
Ok, I have edited the spec file and the dependency on perl is gone. There is a scratch build in brew, but I will wait with the final build until we are sure that will be the last build.

Comment 27 Colin Walters 2017-10-16 14:16:16 UTC
This rsync rebase looks like it breaks the Atomic Host CI jobs which use Ansible's `synchronize` module, which apparently uses `rsync --compress` by default, which now results in e.g.:

```
09:25:51 TASK: [Fetch results] ********************************************************* 
09:25:51 failed: [10.8.245.118 -> 127.0.0.1] => {"cmd": "rsync --delay-updates -F --compress --archive --rsh 'ssh -i /home/jenkins/workspace/atomic-host-jobs-treecompose-improved-sanity-test-rhelah-autobrew-next/atomic-ci-jobs/project/config/keys/ci-factory -S none -o StrictHostKeyChecking=no' --rsync-path=\"sudo rsync\" --out-format='<<CHANGED>>%i %n%L' \"root.245.118:/var/qe/\" \"/home/jenkins/workspace/atomic-host-jobs-treecompose-improved-sanity-test-rhelah-autobrew-next\"", "failed": true, "rc": 12}
09:25:51 msg: rsync: This rsync lacks old-style --compress due to its external zlib.  Try -zz.
09:25:51 rsync error: syntax or usage error (code 1) at main.c(1567) [server=3.1.2]
09:25:51 rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
09:25:51 rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.2]
```

I'm just going to adjust our jobs to set `compress: False` which should work around this, but I suspect our jobs aren't going to be the only things impacted by this.

Comment 28 Karel Srot 2017-10-16 14:28:31 UTC
Colin,
what rsync package version are you using? It should be fixed with the latest rsync-3.1.2-3.el7. Could you please confirm?

Comment 30 Michal Ruprich 2017-11-16 12:56:44 UTC
The list of changes and fixes is rather huge but I will try to point out the most important changes. Keeping the needinfo as a reminder.

Comment 32 Michal Ruprich 2017-12-21 07:44:43 UTC
Hi Lenka,

ok sorry about the url, I didn't realize that. The text looks really good.

Thank you.

Comment 33 Martin Zelený 2018-02-06 08:11:13 UTC
Successfully verified by  Case Run #16738622 of TR#319399 (rhel-7) and Case Run #16738623 of TR#320953 (rhel-alt-7) of TC#563276 /CoreOS/rsync/Sanity/check-rsync-3-1-2-rebase

[   PASS   ] :: RESULT: Test output numbers in 3-digit groups and output strings of --progress option
[   PASS   ] :: RESULT: Test new --info option
[   PASS   ] :: RESULT: Test new --debug option
[   PASS   ] :: RESULT: Test --preallocate option
[   PASS   ] :: RESULT: Test --usermap --groupmap --chown options

Comment 36 errata-xmlrpc 2018-04-10 10:06:32 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHEA-2018:0692