Bug 1212546
Summary: | rsync-out fails to synchronize filesystem with remote filesystem | ||
---|---|---|---|
Product: | [Community] Virtualization Tools | Reporter: | Hu Zhang <huzhan> |
Component: | libguestfs | Assignee: | Richard W.M. Jones <rjones> |
Status: | NEW --- | QA Contact: | |
Severity: | medium | Docs Contact: | |
Priority: | medium | ||
Version: | unspecified | CC: | ksrot, leiwang, luhliari, mruprich, mzeleny, ovasik, ptoscano, rjones, ssorce, virt-maint, wshi, yoguo |
Target Milestone: | --- | ||
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Unspecified | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | Type: | Bug | |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Bug Depends On: | 1432899 | ||
Bug Blocks: | 1224209 |
Is libguestfs-rsync installed (from usual Fedora yum repos)? I tried the above commands, and it worked OK for me with: rsync-3.1.1-3.fc21.x86_64 libguestfs-1.29.34-1.fc23.x86_64 libguestfs-rsync-1.29.34-1.fc23.x86_64 However I didn't try this on Fedora 22 yet (maybe rsync has been broken). Also I didn't try running rsync as root. With rsync running as root, and the hostdest directory also owned by
root, I get a slightly different error:
><fs> rsync-out /dest rsync://192.168.0.175:2999/dest archive:true
libguestfs: error: rsync_out: '/dest' to 'rsync://192.168.0.175:2999/dest': rsync: recv_generator: mkdir "dest" (in dest) failed: Permission denied (13)
*** Skipping any contents from this failed directory ***
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1]
Not completely convinced this isn't something to do with rsync
itself.
(In reply to Richard W.M. Jones from comment #1) > Is libguestfs-rsync installed (from usual Fedora yum repos)? Yes, the version is libguestfs-rsync-1.29.33-1.fc22.x86_64 rsync-3.1.1-6.fc22.x86_64 libguestfs-rsync-1.29.33-1.fc22.x86_64 libguestfs-1.29.33-1.fc22.x86_64 On fedora22. 1. Try with "test", a non-system user. The "hostdest" directory has the permission 775. $ pwd /home/test $ ll drwxrwxr-x. 2 test test 6 May 6 10:42 hostdest drwxrwxr-x. 2 test test 6 May 6 10:13 hostsrc ### Now the "hostdest" does not have the o+w permission. Try with this, I will have a same result as yours, and it's same on rhel7.1. ><fs> rsync-out /dest rsync://test.11.53:33335/dest/ archive:true libguestfs: error: rsync_out: '/dest' to 'rsync://test.11.53:33335/dest/': rsync: recv_generator: mkdir "dest" (in dest) failed: Permission denied (13) *** Skipping any contents from this failed directory *** rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1] 2. Try with "test", a non-system user. The "hostdest" directory has the permission 777. $ pwd /home/test $ ll drwxrwxrwx. 2 test test 6 May 6 10:42 hostdest drwxrwxr-x. 2 test test 6 May 6 10:13 hostsrc Try with this, I will get the result like: ><fs> rsync-out /dest rsync://test.11.53:33335/dest/ archive:true libguestfs: error: rsync_out: '/dest' to 'rsync://test.11.53:33335/dest/': rsync: chgrp "dest" (in dest) failed: Operation not permitted (1) rsync: chgrp "dest/.rsyncout_test.txt.Z2mwvR" (in dest) failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1] $ ll hostdest/ total 0 drwx------. 2 nobody nobody 30 May 6 10:10 dest $ ll hostdest/dest/ ls: cannot open directory hostdest/dest/: Permission denied ### You can see the permission of "dest". $ su # ll hostdest/dest/ total 0 -rw-------. 1 nobody nobody 0 May 6 11:01 rsyncout_test.txt ### So, the file is copied out successfully, but the permission is not reasonable. 3. Let try the same scenario as step2 on rhel7.1. $ pwd /home/test $ ll drwxrwxrwx. 2 test test 6 May 6 10:42 hostdest drwxrwxr-x. 2 test test 6 May 6 10:13 hostsrc ><fs> rsync-out /dest rsync://test.106.30:33335/dest/ archive:true ><fs> q $ ll hostdest/ total 0 drwxr-xr-x. 2 nobody nobody 30 May 6 10:34 dest $ ll hostdest/dest/ total 0 -rw-r--r--. 1 nobody nobody 0 May 6 10:34 rsyncout_test.txt Compared with the same files in step2, the permissions of them are different. In summary, the permissions of the files copied out via rsync-out are not reasonable on fedora22. Hello. I'm facing exactly the same issue with rsync when syncing the newly generated filesystem content to / during the stage2 bootstrap with f21 sources. The issue was not present with f19 sources. So, it looks like something changed in rsync in the meantime. (In reply to Jaromír Cápík from comment #6) > Hello. I'm facing exactly the same issue with rsync when syncing the newly > generated filesystem content to / during the stage2 bootstrap with f21 > sources. The issue was not present with f19 sources. So, it looks like > something changed in rsync in the meantime. I pretty much convinced myself that this was a problem in rsync. In any case, libguestfs doesn't do anything more complex than running rsync. If you wish to debug this, you can see the exact commands it runs in this file: https://github.com/libguestfs/libguestfs/blob/master/daemon/rsync.c How about changing the component to rsync then? Well we don't really understand what the bug is. Moving it around isn't going to help until someone does that work. In that case at least CCing rsync maintainers could help, right? NOTE: In my case the problem appears even when rsync-ing local directories only. We do the following: rsync -av ./ / NOTE2: I just added --ignore-existing and it passed. After running the libguestfs automated cases, all of the rsync-out cases failed with the following versions:
libguestfs-1.36.10-3.el7.x86_64
libguestfs-rsync-1.36.10-3.el7.x86_64
rsync-3.1.2-4.el7.x86_64
On rhel7.5 host and the user is 'root'
Steps:
1. Prepare the file on the host
# rm -fr /tmp/hostsrc
# rm -fr /tmp/hostdest
# mkdir /tmp/hostsrc
# mkdir /tmp/hostdest
# chmod 777 /tmp/hostdest
# touch /tmp/hostsrc/rsyncin_test.txt
2. Configure the rsyncd service.
# echo "port = 33335
pid file = rsyncd.pid
[src]
path = /tmp/hostsrc
comment = source
use chroot = false
read only = true
[dest]
path = /tmp/hostdest
comment = destination
use chroot = false
read only = false
" > /tmp/rsyncd.conf
# rsync --daemon --config=/tmp/rsyncd.conf
3. guestfish --network -a $GSTEST_image_path
><fs> run
><fs> mount-options noatime $mount_device /
><fs> rm-rf /dest
><fs> rm-rf /src
><fs> mkdir /src
><fs> mkdir /dest
><fs> touch /dest/rsyncout_test.txt
><fs> rsync-out /dest rsync://root@$ipaddr_of_host:33335/dest/ archive:true
libguestfs: error: rsync_out: '/dest' to 'rsync://root.144.58:33335/dest/': rsync: chgrp "dest" (in dest) failed: Operation not permitted (1)
rsync: chgrp "dest/.rsyncout_test.txt.d8OIl2" (in dest) failed: Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]
So the same issue still exists in rsync version 3.1.2(#1432899).
Hi rjones, I also think it's a problem in rsync with -a option(archive:true in guestfish). -a option is a set of -rlptgoD. I had a try to use rsync command in local host. e.g. #guestmount -a misc.ext2.raw -m /dev/vol_test/vol_file /mnt/test #cd /mnt/test #mkdir dest #touch dest/rsyncout_test.txt #rsync -rlptoD dest rsync://root.73.37:33335/dest/ It works well. I just removed the -g option. The previous error also was displayed "chgrp ... failed" in guestfish according to the comment#14. I just want to inquire whether the -g option can be removed in libguestfs source code, in case we can work around this problem in libguestfs autotest. (In reply to Michal Ruprich from comment #22) > You should add the 'fake super = yes' option to the destination module. I > didn't understand this at first as well so I asked the upstream some time > ago and I received a nice answer here: > > https://bugzilla.samba.org/show_bug.cgi?id=11414 > > Basically even if you run rsync as root it will not run under root when a > transfer is initiated. So this error has nothing to do with SELinux but > simply with the fact that only root is able to set group on a file. > > Also if the original group from the sender side is missing in the conf file > on receiving side, it will be set to nobody. You should add uid and gid > options to the module. This is described in more detail in 'man rsyncd.conf'. > > Hope this helps. > > Regrads, > Michal It works fine with 'fake super = yes' option to the destination module. Thanks a lot.I will look at the rsyncd.conf carefully. |
Description of problem: rsync-out fails to synchronize filesystem with remote filesystem Version-Release number of selected component (if applicable): kernel-4.0.0-0.rc1.git0.1.fc22.x86_64 libguestfs-1.29.33-1.fc22.x86_64 How reproducible: Always Steps to Reproduce: 1. Prepare the file on the host # rm -fr /tmp/hostsrc # rm -fr /tmp/hostdest # mkdir /tmp/hostsrc # mkdir /tmp/hostdest # chmod 777 /tmp/hostdest # touch /tmp/hostsrc/rsyncin_test.txt 2. Configure the rsyncd service. # echo "port = 33335 pid file = rsyncd.pid [src] path = /tmp/hostsrc comment = source use chroot = false read only = true [dest] path = /tmp/hostdest comment = destination use chroot = false read only = false " > /tmp/rsyncd.conf # rsync --daemon --config=/tmp/rsyncd.conf 3. guestfish --network -a $GSTEST_image_path ><fs> run ><fs> mount-options noatime $mount_device / ><fs> rm-rf /dest ><fs> rm-rf /src ><fs> mkdir /src ><fs> mkdir /dest ><fs> touch /dest/rsyncout_test.txt ><fs> rsync-out /dest rsync://root@$ipaddr_of_host:33335/dest/ archive:true Actual results: libguestfs: error: rsync_out: '/dest' to 'rsync://root.10.241:33335/dest/': rsync: chgrp "dest" (in dest) failed: Operation not permitted (1) rsync: chgrp "dest/.rsyncout_test.txt.AUEUIM" (in dest) failed: Operation not permitted (1) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1] Expected results: rsync-out can synchronize filesystem with remote filesystem successfully Additional info: