Bug 1212546 - rsync-out fails to synchronize filesystem with remote filesystem
rsync-out fails to synchronize filesystem with remote filesystem
Status: NEW
Product: Virtualization Tools
Classification: Community
Component: libguestfs (Show other bugs)
unspecified
Unspecified Unspecified
medium Severity medium
: ---
: ---
Assigned To: Richard W.M. Jones
:
Depends On:
Blocks: fedora-bootstrap
  Show dependency treegraph
 
Reported: 2015-04-16 11:33 EDT by Hu Zhang
Modified: 2017-05-02 02:53 EDT (History)
11 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed:
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Hu Zhang 2015-04-16 11:33:56 EDT
Description of problem:
rsync-out fails to synchronize filesystem with remote filesystem

Version-Release number of selected component (if applicable):
kernel-4.0.0-0.rc1.git0.1.fc22.x86_64
libguestfs-1.29.33-1.fc22.x86_64

How reproducible:
Always

Steps to Reproduce:
1. Prepare the file on the host
# rm -fr /tmp/hostsrc
# rm -fr /tmp/hostdest
# mkdir /tmp/hostsrc
# mkdir /tmp/hostdest
# chmod 777 /tmp/hostdest
# touch /tmp/hostsrc/rsyncin_test.txt

2. Configure the rsyncd service.
# echo "port = 33335
pid file = rsyncd.pid
[src]
path = /tmp/hostsrc
comment = source
use chroot = false
read only = true
[dest]
path = /tmp/hostdest
comment = destination
use chroot = false
read only = false
" > /tmp/rsyncd.conf

# rsync --daemon --config=/tmp/rsyncd.conf

3. guestfish --network -a $GSTEST_image_path
><fs> run
><fs> mount-options noatime $mount_device /
><fs> rm-rf /dest
><fs> rm-rf /src
><fs> mkdir /src
><fs> mkdir /dest
><fs> touch /dest/rsyncout_test.txt
><fs> rsync-out /dest rsync://root@$ipaddr_of_host:33335/dest/ archive:true

Actual results:
libguestfs: error: rsync_out: '/dest' to 'rsync://root@10.66.10.241:33335/dest/': rsync: chgrp "dest" (in dest) failed: Operation not permitted (1)
rsync: chgrp "dest/.rsyncout_test.txt.AUEUIM" (in dest) failed: Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1]

Expected results:
rsync-out can synchronize filesystem with remote filesystem successfully

Additional info:
Comment 1 Richard W.M. Jones 2015-04-16 11:44:21 EDT
Is libguestfs-rsync installed (from usual Fedora yum repos)?
Comment 2 Richard W.M. Jones 2015-04-16 11:49:19 EDT
I tried the above commands, and it worked OK for me with:

rsync-3.1.1-3.fc21.x86_64
libguestfs-1.29.34-1.fc23.x86_64
libguestfs-rsync-1.29.34-1.fc23.x86_64

However I didn't try this on Fedora 22 yet (maybe rsync has
been broken).

Also I didn't try running rsync as root.
Comment 3 Richard W.M. Jones 2015-04-16 11:56:38 EDT
With rsync running as root, and the hostdest directory also owned by
root, I get a slightly different error:

><fs> rsync-out /dest rsync://192.168.0.175:2999/dest archive:true
libguestfs: error: rsync_out: '/dest' to 'rsync://192.168.0.175:2999/dest': rsync: recv_generator: mkdir "dest" (in dest) failed: Permission denied (13)
*** Skipping any contents from this failed directory ***
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1]

Not completely convinced this isn't something to do with rsync
itself.
Comment 4 Hu Zhang 2015-05-05 22:21:48 EDT
(In reply to Richard W.M. Jones from comment #1)
> Is libguestfs-rsync installed (from usual Fedora yum repos)?

Yes, the version is libguestfs-rsync-1.29.33-1.fc22.x86_64
Comment 5 Hu Zhang 2015-05-05 23:20:17 EDT
rsync-3.1.1-6.fc22.x86_64
libguestfs-rsync-1.29.33-1.fc22.x86_64
libguestfs-1.29.33-1.fc22.x86_64

On fedora22.
1. Try with "test", a non-system user. The "hostdest" directory has the permission 775.
$ pwd
/home/test
$ ll
drwxrwxr-x. 2 test test          6 May  6 10:42 hostdest
drwxrwxr-x. 2 test test          6 May  6 10:13 hostsrc

### Now the "hostdest" does not have the o+w permission. 

Try with this, I will have a same result as yours, and it's same on rhel7.1.
><fs> rsync-out /dest rsync://test@10.66.11.53:33335/dest/ archive:true
libguestfs: error: rsync_out: '/dest' to 'rsync://test@10.66.11.53:33335/dest/': rsync: recv_generator: mkdir "dest" (in dest) failed: Permission denied (13)
*** Skipping any contents from this failed directory ***
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1]

2. Try with "test", a non-system user. The "hostdest" directory has the permission 777.
$ pwd
/home/test
$ ll
drwxrwxrwx. 2 test test          6 May  6 10:42 hostdest
drwxrwxr-x. 2 test test          6 May  6 10:13 hostsrc

Try with this, I will get the result like:
><fs> rsync-out /dest rsync://test@10.66.11.53:33335/dest/ archive:true
libguestfs: error: rsync_out: '/dest' to 'rsync://test@10.66.11.53:33335/dest/': rsync: chgrp "dest" (in dest) failed: Operation not permitted (1)
rsync: chgrp "dest/.rsyncout_test.txt.Z2mwvR" (in dest) failed: Operation not permitted (1)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1165) [sender=3.1.1]

$ ll hostdest/
total 0
drwx------. 2 nobody nobody 30 May  6 10:10 dest

$ ll hostdest/dest/
ls: cannot open directory hostdest/dest/: Permission denied
### You can see the permission of "dest".

$ su
# ll hostdest/dest/
total 0
-rw-------. 1 nobody nobody 0 May  6 11:01 rsyncout_test.txt

### So, the file is copied out successfully, but the permission is not reasonable.

3. Let try the same scenario as step2 on rhel7.1.
$ pwd
/home/test
$ ll
drwxrwxrwx. 2 test test          6 May  6 10:42 hostdest
drwxrwxr-x. 2 test test          6 May  6 10:13 hostsrc

><fs> rsync-out /dest rsync://test@10.66.106.30:33335/dest/ archive:true 
><fs> q

$ ll hostdest/
total 0
drwxr-xr-x. 2 nobody nobody 30 May  6 10:34 dest
$ ll hostdest/dest/
total 0
-rw-r--r--. 1 nobody nobody 0 May  6 10:34 rsyncout_test.txt

Compared with the same files in step2, the permissions of them are different.
In summary, the permissions of the files copied out via rsync-out are not reasonable on fedora22.
Comment 6 Jaromír Cápík 2015-07-14 17:15:22 EDT
Hello. I'm facing exactly the same issue with rsync when syncing the newly generated filesystem content to / during the stage2 bootstrap with f21 sources. The issue was not present with f19 sources. So, it looks like something changed in rsync in the meantime.
Comment 7 Richard W.M. Jones 2015-07-14 17:39:52 EDT
(In reply to Jaromír Cápík from comment #6)
> Hello. I'm facing exactly the same issue with rsync when syncing the newly
> generated filesystem content to / during the stage2 bootstrap with f21
> sources. The issue was not present with f19 sources. So, it looks like
> something changed in rsync in the meantime.

I pretty much convinced myself that this was a problem in rsync.
In any case, libguestfs doesn't do anything more complex than
running rsync.  If you wish to debug this, you can see the exact
commands it runs in this file:

https://github.com/libguestfs/libguestfs/blob/master/daemon/rsync.c
Comment 8 Jaromír Cápík 2015-07-15 09:22:26 EDT
How about changing the component to rsync then?
Comment 9 Richard W.M. Jones 2015-07-15 10:08:34 EDT
Well we don't really understand what the bug is.  Moving it around
isn't going to help until someone does that work.
Comment 10 Jaromír Cápík 2015-07-15 10:32:23 EDT
In that case at least CCing rsync maintainers could help, right?
Comment 11 Jaromír Cápík 2015-07-15 10:39:19 EDT
NOTE: In my case the problem appears even when rsync-ing local directories only.

We do the following:

  rsync -av ./ /
Comment 12 Jaromír Cápík 2015-07-15 12:15:49 EDT
NOTE2: I just added --ignore-existing and it passed.

Note You need to log in before you can comment on or make changes to this bug.