Bug 827429
Summary: | rsync --sparse cannot copy /var/log/lastlog on x86_64 server | |||
---|---|---|---|---|
Product: | Red Hat Enterprise Linux 6 | Reporter: | Joaquin <narebeestjes> | |
Component: | rsync | Assignee: | Michal Ruprich <mruprich> | |
Status: | CLOSED WONTFIX | QA Contact: | BaseOS QE Security Team <qe-baseos-security> | |
Severity: | high | Docs Contact: | ||
Priority: | unspecified | |||
Version: | 6.3 | CC: | andy, brian.murrell, dapospis, ebenes, mvermaes, narebeestjes, pablo.iranzo, thozza, william | |
Target Milestone: | rc | |||
Target Release: | --- | |||
Hardware: | All | |||
OS: | Linux | |||
Whiteboard: | ||||
Fixed In Version: | Doc Type: | If docs needed, set a value | ||
Doc Text: | Story Points: | --- | ||
Clone Of: | 156809 | |||
: | 1236520 (view as bug list) | Environment: | ||
Last Closed: | 2017-09-06 08:56:22 UTC | Type: | --- | |
Regression: | --- | Mount Type: | --- | |
Documentation: | --- | CRM: | ||
Verified Versions: | Category: | --- | ||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | ||
Cloudforms Team: | --- | Target Upstream Version: | ||
Embargoed: | ||||
Bug Depends On: | 525545, 771286, 951564 | |||
Bug Blocks: |
Description
Joaquin
2012-06-01 13:13:10 UTC
This request was evaluated by Red Hat Product Management for inclusion in the current release of Red Hat Enterprise Linux. Because the affected component is not scheduled to be updated in the current release, Red Hat is unable to address this request at this time. Red Hat invites you to ask your support representative to propose this request, if appropriate, in the next release of Red Hat Enterprise Linux. So fixing this missed RHEL 6.4 and 6.5. Can we please have this fixed for the next (or current even!) release of RHEL? Could you please confirm whether this bug is fixed or not in the current RHEL-6 version of rsync? Still not working on an updated RHEL 6.5, dry run works, actual sync not: # cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.5 (Santiago) # rpm -qi rsync Name : rsync Relocations: (not relocatable) Version : 3.0.6 Vendor: Red Hat, Inc. Release : 9.el6_4.1 Build Date: Wed 23 Oct 2013 10:31:58 AM CEST .. # rsync --dry-run --rsh=ssh --sparse -avz root@<other-rhel-6>:/var/log/lastlog ./ receiving incremental file list lastlog sent 14 bytes received 43 bytes 8.77 bytes/sec total size is 500825445128 speedup is 8786411318.04 (DRY RUN) # rsync --rsh=ssh --sparse -avz root@<other-rhel-6-or-7>:/var/log/lastlog ./ receiving incremental file list lastlog ^CKilled by signal 2. rsync error: unexplained error (code 255) at rsync.c(544) [generator=3.0.6] rsync error: received SIGUSR1 (code 19) at main.c(1285) [receiver=3.0.6] Still takes forever without even returning the "file size". Using a RHEL 7 server immediately returns the "file size" and, after some time, the file: # rsync --version rsync version 3.0.9 protocol version 30 .. # rsync --rsh=ssh --sparse -avz root@<my-rhel6.5-machine>:/var/log/lastlog ./ .. sent 30 bytes received 279 bytes 41.20 bytes/sec total size is 146000 speedup is 472.49 # ls -al lastlog -rw-r--r--. 1 root root 146000 Jul 10 13:24 lastlog Joaquin, thank you very much for the info. Now we see that on RHEL-7, it successfully transferred cca 140kB, while on RHEL-6, we see (thanks to the dry-run) that it tried to transfer cca 500GiB. BTW, an important information about the version is the update number behind the dash, e.g. -12 in rsync-3.0.6-12.el6, visible in "rpm -q" or the whole output of "rpm -qi", but now it is not important anymore since the difference in the file size is the information that we missed before. One solution would be to change the lastlog file implementation, which is probably not going to happen soon, see bug 525545. Another way would be to change the rsync implementation of sparse files, see bug 771286. A workaround could be to reduce the size of the lastlog (probably by changing the high UIDs for those users and recreating/deleting the file). Apparently this is not fixed with rsync 3.0.9? To be clear: The above 140kB (system with only local users) and 500GiB (system using AD for non local users) are different files. Maybe I made the same mistake before as I did today (get lastlog from a machine where the reported size was not too big). For me an exclusion of /var/log/lastlog resolved the issue at the time and at the moment I'm not using an rsync based backup solution. But maybe there are people who need this to work? To be complete: RHEL 6.5: ========= # rpm -qi rsync Name : rsync Relocations: (not relocatable) Version : 3.0.6 Vendor: Red Hat, Inc. Release : 9.el6_4.1 Build Date: Wed 23 Oct 2013 10:31:58 AM CEST ... The "500GiB" lastlog used above on another machine (CentOS 6.5, same rsync version): centos6.5# du -hs /var/log/lastlog 40K /var/log/lastlog centos6.5# ls -l /var/log/lastlog -rw-r--r--. 1 root root 500825445128 Jul 8 15:28 /var/log/lastlog rsync on RHEL 7 getting file from CentOS 6.5 ============================================ Now retrieving that file by rsync using the RHEL 7 machine: rhel7# rpm -qi rsync Name : rsync Version : 3.0.9 Release : 11.el7 Architecture: x86_64 .. rhel7# rsync --dry-run --rsh=ssh --sparse -avz root@<centos6.5>:/va/log/lastlog ./ ... sent 14 bytes received 43 bytes 8.77 bytes/sec total size is 500825445128 speedup is 8786411318.04 (DRY RUN) But the actual transfer might try to download everything (it does not return the size immediately and takes forever). rhel7# rsync --rsh=ssh --sparse -avz root@<centos6.5>:/var/log/lastlog ./ .. receiving incremental file list lastlog ^CKilled by signal 2. rsync error: unexplained error (code 255) at rsync.c(551) [generator=3.0.9] rsync error: received SIGUSR1 (code 19) at main.c(1298) [receiver=3.0.9] RHEL 7 -> RHEL 7: ================= Get the local /var/log/lastlog on RHEL 7 (also using AD for non local users): rhel7# ls -l /var/log/lastlog -rw-r--r--. 1 root root 500825445128 Jul 14 11:47 /var/log/lastlog rhel7# du -ks /var/log/lastlog 24 /var/log/lastlog rhel7# time rsync --dry-run --rsh=ssh --sparse -avz root@localhost:/var/log/lastlog ./ root@localhost's password: receiving incremental file list lastlog sent 14 bytes received 43 bytes 10.36 bytes/sec total size is 500825445128 speedup is 8786411318.04 (DRY RUN) real 10m4.280s user 0m0.188s sys 0m0.615s rhel7# time rsync --rsh=ssh --sparse -avz root@localhost:/var/log/lastlog ./ root@localhost's password: receiving incremental file list lastlog unexpected tag 25 [receiver] rsync error: error in rsync protocol data stream (code 12) at io.c(1141) [receiver=3.0.9] rsync: connection unexpectedly closed (37 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(605) [generator=3.0.9] real 10m3.797s user 1m20.166s sys 0m0.039s rsync on RHEL 6.5 getting lastlog from RHEL 7 ============================================= Get the same file (from this RHEL 7 machine) using RHEL 6.5: rhel6.5# time rsync --dry-run --rsh=ssh --sparse -avz root@<rhel7>:/var/log/lastlog ./ root@<rhel7> password: receiving incremental file list lastlog sent 14 bytes received 43 bytes 10.36 bytes/sec total size is 500825445128 speedup is 8786411318.04 (DRY RUN) real 10m4.889s user 0m0.287s sys 0m0.311s rhel6.5# time rsync --rsh=ssh --sparse -avz root@<rhel7>:/var/log/lastlog ./ root@<rhel7> password: receiving incremental file list lastlog unexpected tag 25 [receiver] rsync error: error in rsync protocol data stream (code 12) at io.c(1134) [receiver=3.0.6] rsync: connection unexpectedly closed (37 bytes received so far) [generator] rsync error: error in rsync protocol data stream (code 12) at io.c(600) [generator=3.0.6] real 10m5.969s user 0m0.604s sys 0m0.558s Thanks for the update. The details are consistent with what we have discussed. I think that you are right that it would be nice to have this resolved for other users even when you have worked around it. One thing is the implementation of transferring sparse files in rsync which influences e.g. transferring virtual machine images as well. The other thing is the size/implementation of the lastlog file. Regarding comment 12 that says it will not be fixed soon, there is a way -- there is a bug 951564 filed for Fedora and the fix could get into RHEL in the future. Red Hat Enterprise Linux 6 transitioned to the Production 3 Phase on May 10, 2017. During the Production 3 Phase, Critical impact Security Advisories (RHSAs) and selected Urgent Priority Bug Fix Advisories (RHBAs) may be released as they become available. The official life cycle policy can be reviewed here: http://redhat.com/rhel/lifecycle This issue does not appear to meet the inclusion criteria for the Production Phase 3 and will be marked as CLOSED/WONTFIX. If this remains a critical requirement, please contact Red Hat Customer Support to request a re-evaluation of the issue, citing a clear business justification. Red Hat Customer Support can be contacted via the Red Hat Customer Portal at the following URL: https://access.redhat.com |