Description of problem: Restoring a F19 tar archive using either F19 Live Media or Rescue Media removes all extended file attributes and corrupts many file user and group names. Version-Release number of selected component (if applicable): /usr/bin/tar-2:1.26-24.fc19.x86_64 /usr/lib64/libacl.so.1.1.0.fc19.x86_64 /usr/lib64/libc-2.17.so.fc19.x86_64 (glibc-2.17.11.fc19.x86_64) How reproducible: Always happens in the same way. Steps to Reproduce: 1.Create a tar archive of F19 (tar -cv -f $TAPE -X $EXCLUDE --one-file-system --acls --xattrs / /boot /home /var/lib/pgsql/data /tmp 2> /tmp/tarerr.out) 2.Boot Live or Rescue Media 3.tar xvf /dev/st0 to restore F19 Actual results: F19 boots but functions poorly due to file attribute corruption and disfunction. Absolutely all extended file attributes (acls and xattrs) are lost. For example: # rpm -Va | grep "\.""\."P (caPability change) ........P /usr/libexec/pt_chown (glibc-common) ........P /usr/bin/gnome-keyring-daemon (gnome-keyring) ........P /usr/bin/rcp (rsh) ........P /usr/bin/rlogin (rsh) ........P /usr/bin/rsh (rsh) ........P /usr/bin/systemd-detect-virt (systemd) ........P /usr/bin/ping (iputils) ........P /usr/bin/ping6 (iputils) ........P /usr/sbin/arping (iputils) ........P /usr/sbin/clockdiff (iputils) ........P /usr/sbin/suexec (httpd) ........P /usr/sbin/mtr (mtr) In addition, many file/directory user names and groups are corrupted: Eg: # lt -lt -d /var/lib/chrony drwxr-xr-x. 2 977 977 4096 Aug 13 16:07 chrony instead of: drwxr-xr-x. 2 chrony chrony 4096 Aug 13 16:07 chrony Where 977 is a non-existant user and group. This results in messages in /var/log/messages that chrony does have have permission to access /var/lib/977/driftfile. Other similar exmples are to numerous to mention. Expected results: Additional info: The only "work around" is to scan all file attributes and repair those effected (user names, group names, suids, guids, acls and xattrs).
The files tar, libacl.so.1.1.0 and libc-2.17.so are all smaller in the F19 Live and Rescue Media than in the full F19 OS. The libacl file is actually 7% smaller. I used the F19-x86_64-DESK-20130806.iso respin from http://alt.fedoraproject.org/pub/alt/live-respins/. The 3 files are identical in size in the Live Media when compared to those in the Rescue Media
Craig, thanks for the report. Try to restore with --xattrs --acls please. Extended attributes are not restored by default in F19+ (as the acls/selinux/xattr patch became upstream). If you don't like this behavior, you still may use the TAR_OPTIONS='--xattrs --selinux --acls' environment variable. According to group/user name corruptions. Be careful that the same users/groups exist on the running system and that they have the _same_ UIDs/GIDs. -- I doubt the problem is in libacl.so* size difference (it is probably different stripping level).
> According to group/user name corruptions. Be careful that the same > users/groups exist on the running system and that they have the _same_ > UIDs/GIDs. That was not 100% correct statement, sorry. The username and groupname should have precedence before user ID and group ID during archive extraction. Thus the UID/GID change among system should not be big a problem .. the problem with 'chrony' user is probably that its account is not present on the live media. If this is not the case, we should really observe why the user/group information are not restored properly. Pavel
Hi Pavel I tried (and I think I tried this once before): tar --acls --xattrs xvf /dev/st0 but with the same result, namely: [root@server lib]# rpm -Va | grep "\.""\."P ........P /usr/bin/rcp ........P /usr/bin/rlogin ........P /usr/bin/rsh ........P /usr/sbin/suexec ........P /usr/bin/systemd-detect-virt ........P /usr/bin/ping ........P /usr/bin/ping6 ........P /usr/sbin/arping ........P /usr/sbin/clockdiff ........P /usr/libexec/pt_chown ........P /usr/sbin/mtr ........P /usr/bin/gnome-keyring-daemon However, with the user/group names I found that for user/groupnames that existed on both the live media and the full OS: where the userid or groupid on the live media were different to those in the the full OS, the user/groupnames on the files restored to the full OS were replaced with the user/groupids from the live media. So, before: [root@server ~]# ls -lt /var/lib (full OS) total 260 drwxr-xr-x+ 2 root root 4096 Aug 14 02:18 alsa drwxr-x---+ 2 root slocate 4096 Aug 14 02:04 mlocate drwxr-xr-x+ 2 root root 4096 Aug 14 01:49 PackageKit drwxr-xr-x+ 2 root root 4096 Aug 14 01:49 xkb drwxr-xr-x+ 2 root root 4096 Aug 14 01:49 NetworkManager drwxrwx--T+ 11 gdm gdm 4096 Aug 14 01:49 gdm drwxr-xr-x+ 6 root root 4096 Aug 14 01:49 nfs drwx------+ 2 rpc rpc 4096 Aug 14 01:49 rpcbind drwxr-xr-x+ 2 root root 4096 Aug 14 01:49 rpm -rw-------. 1 root root 4096 Aug 14 01:49 random-seed drwxr-xr-x+ 2 chrony chrony 4096 Aug 14 01:48 chrony drwxr-xr-x+ 6 root root 4096 Aug 14 01:47 yum -rw-r--r-- 1 root root 210 Aug 14 01:29 error -rw-r--r--. 1 root root 1355 Aug 13 11:32 logrotate.status drwx------+ 2 postfix root 4096 Aug 10 17:41 postfix drwxr-xr-x+ 2 root root 4096 Aug 10 17:41 misc drwxr-xr-x+ 2 root root 4096 Aug 10 17:41 alternatives drwx------+ 2 root root 4096 Aug 10 11:02 udisks drwxr-xr-x+ 3 root root 4096 Aug 8 09:58 var drwxr-xr-x+ 3 root root 4096 Aug 8 09:55 usr drwxr-xr-x+ 2 unbound unbound 4096 Aug 1 03:10 unbound drwx------+ 2 apache apache 4096 Jul 31 14:50 dav drwxr-xr-x+ 2 root root 4096 Jul 26 01:13 initramfs drwxr-xr-x+ 5 root root 4096 Jul 25 21:01 pear drwxr-xr-x+ 2 root root 4096 Jul 23 23:36 bluetooth drwxr-xr-x+ 2 root root 4096 Jul 23 19:12 hp drwxr-xr-x+ 3 root root 4096 Jul 22 15:04 php drwxr-xr-x+ 3 colord colord 4096 Jul 19 06:32 colord drwxr-xr-x+ 6 root root 4096 Jul 19 00:06 sss drwxr-xr-x+ 4 root root 4096 Jul 15 09:54 texmf drwxr-xr-x+ 2 root root 4096 Jul 15 09:34 prelink drwxr-xr-x+ 2 root root 4096 Jul 14 14:47 freeipmi drwxr-xr-x+ 4 root root 4096 Jul 14 14:41 hsqldb drwxr-xr-x+ 2 root root 4096 Jul 14 14:39 sepolgen drwx------+ 4 postgres postgres 4096 Jul 14 14:38 pgsql drwxr-x---+ 3 root polkitd 4096 Jul 14 14:36 polkit-1 drwxr-xr-x+ 3 root root 4096 Jul 8 16:56 games drwxr-xr-x+ 3 root root 4096 Jul 8 16:56 rpm-state drwxr-xr-x+ 2 root root 4096 Jul 3 01:02 os-prober drwxr-xr-x+ 2 root root 4096 Jul 2 20:14 dhclient drwxr-xr-x+ 5 root root 4096 Jun 24 23:31 systemd drwxr-xr-x+ 2 root root 4096 Jun 22 09:15 selinux drwxr-xr-x+ 2 root root 4096 Jun 18 06:23 dbus drwx------+ 3 root root 4096 Jun 12 23:17 authconfig drwxr-xr-x+ 4 root root 4096 Jun 12 01:16 AccountsService drwxr-xr-x+ 2 root root 4096 Jun 11 21:27 dnsmasq drwx------+ 2 tss tss 4096 Jun 2 21:42 tpm drwxr-xr-x+ 4 root root 4096 May 31 16:07 stateless drwx------+ 2 pulse pulse 4096 May 31 12:36 pulse drwxr-xr-x+ 2 root root 4096 May 21 22:32 setroubleshoot drwxr-xr-x+ 4 root root 4096 May 6 16:09 net-snmp drwxr-xr-x+ 3 root root 4096 Apr 30 03:27 dirmngr drwxr-xr-x+ 3 root root 4096 Apr 17 19:48 samba drwx------+ 2 root root 4096 Apr 3 23:47 rsyslog drwxr-x---+ 6 backuppc root 4096 Apr 1 03:50 BackupPC drwx------+ 2 root root 4096 Mar 28 21:11 udisks2 drwxr-xr-x+ 2 root root 4096 Mar 27 00:14 plymouth drwxr-xr-x+ 2 root root 4096 Mar 20 22:10 upower drwxr-xr-x+ 3 root root 4096 Mar 9 02:19 color drwxr-xr-x+ 2 webalizer root 4096 Feb 19 09:38 webalizer drwxr-xr-x+ 2 root root 4096 Feb 14 13:53 cs drwxrwx---+ 2 root root 4096 Sep 9 2010 DeviceKit-disks drwxr-xr-x+ 2 root root 4096 Sep 9 2010 DeviceKit-power drwxr-xr-x+ 2 root root 4096 Mar 25 2010 readahead drwxrwx---+ 2 root polkituser 4096 Aug 18 2009 PolicyKit But now after running tar from the live media: [root@server lib]# ls -lt (full OS) total 256 drwxr-xr-x+ 2 root root 4096 Aug 14 00:34 alsa drwxr-xr-x+ 2 root root 4096 Aug 14 00:33 rpm drwxr-xr-x+ 2 root root 4096 Aug 14 00:29 PackageKit drwxr-xr-x+ 2 root root 4096 Aug 14 00:28 xkb drwxrwx--T+ 11 gdm gdm 4096 Aug 14 00:28 gdm drwxr-xr-x+ 2 root root 4096 Aug 14 00:28 NetworkManager drwxr-xr-x+ 6 root root 4096 Aug 14 00:28 nfs -rw-------. 1 root root 4096 Aug 14 00:28 random-seed drwxr-xr-x+ 2 997 996 4096 Aug 13 19:34 chrony drwx------+ 2 rpc rpc 4096 Aug 13 13:49 rpcbind drwxr-x---+ 2 root slocate 4096 Aug 13 11:32 mlocate -rw-r--r--. 1 root root 1355 Aug 13 11:32 logrotate.status drwxr-xr-x+ 6 root root 4096 Aug 13 07:52 yum drwx------+ 2 postfix root 4096 Aug 10 17:41 postfix drwxr-xr-x+ 2 root root 4096 Aug 10 17:41 misc drwxr-xr-x+ 2 root root 4096 Aug 10 17:41 alternatives drwx------+ 2 root root 4096 Aug 10 11:02 udisks drwxr-xr-x+ 3 root root 4096 Aug 8 09:58 var drwxr-xr-x+ 3 root root 4096 Aug 8 09:55 usr drwxr-xr-x+ 2 996 995 4096 Aug 1 03:10 unbound drwx------+ 2 apache apache 4096 Jul 31 14:50 dav drwxr-xr-x+ 2 root root 4096 Jul 26 01:13 initramfs drwxr-xr-x+ 5 root root 4096 Jul 25 21:01 pear drwxr-xr-x+ 2 root root 4096 Jul 23 23:36 bluetooth drwxr-xr-x+ 2 root root 4096 Jul 23 19:12 hp drwxr-xr-x+ 3 root root 4096 Jul 22 15:04 php drwxr-xr-x+ 3 998 998 4096 Jul 19 06:32 colord drwxr-xr-x+ 6 root root 4096 Jul 19 00:06 sss drwxr-xr-x+ 4 root root 4096 Jul 15 09:54 texmf drwxr-xr-x+ 2 root root 4096 Jul 15 09:34 prelink drwxr-xr-x+ 2 root root 4096 Jul 14 14:47 freeipmi drwxr-xr-x+ 4 root root 4096 Jul 14 14:41 hsqldb drwxr-xr-x+ 2 root root 4096 Jul 14 14:39 sepolgen drwx------+ 4 postgres postgres 4096 Jul 14 14:38 pgsql drwxr-x---+ 3 root 999 4096 Jul 14 14:36 polkit-1 drwxr-xr-x+ 3 root root 4096 Jul 8 16:56 games drwxr-xr-x+ 3 root root 4096 Jul 8 16:56 rpm-state drwxr-xr-x+ 2 root root 4096 Jul 3 01:02 os-prober drwxr-xr-x+ 2 root root 4096 Jul 2 20:14 dhclient drwxr-xr-x+ 5 root root 4096 Jun 24 23:31 systemd drwxr-xr-x+ 2 root root 4096 Jun 22 09:15 selinux drwxr-xr-x+ 2 root root 4096 Jun 18 06:23 dbus drwx------+ 3 root root 4096 Jun 12 23:17 authconfig drwxr-xr-x+ 4 root root 4096 Jun 12 01:16 AccountsService drwxr-xr-x+ 2 root root 4096 Jun 11 21:27 dnsmasq drwx------+ 2 tss tss 4096 Jun 2 21:42 tpm drwxr-xr-x+ 4 root root 4096 May 31 16:07 stateless drwx------+ 2 992 992 4096 May 31 12:36 pulse drwxr-xr-x+ 2 root root 4096 May 21 22:32 setroubleshoot drwxr-xr-x+ 4 root root 4096 May 6 16:09 net-snmp drwxr-xr-x+ 3 root root 4096 Apr 30 03:27 dirmngr drwxr-xr-x+ 3 root root 4096 Apr 17 19:48 samba drwx------+ 2 root root 4096 Apr 3 23:47 rsyslog drwxr-x---+ 6 backuppc root 4096 Apr 1 03:50 BackupPC drwx------+ 2 root root 4096 Mar 28 21:11 udisks2 drwxr-xr-x+ 2 root root 4096 Mar 27 00:14 plymouth drwxr-xr-x+ 2 root root 4096 Mar 20 22:10 upower drwxr-xr-x+ 3 root root 4096 Mar 9 02:19 color drwxr-xr-x+ 2 webalizer root 4096 Feb 19 09:38 webalizer drwxr-xr-x+ 2 root root 4096 Feb 14 13:53 cs drwxrwx---+ 2 root root 4096 Sep 9 2010 DeviceKit-disks drwxr-xr-x+ 2 root root 4096 Sep 9 2010 DeviceKit-power drwxr-xr-x+ 2 root root 4096 Mar 25 2010 readahead drwxrwx---+ 2 root polkituser 4096 Aug 18 2009 PolicyKit and you can see the user/groupids in the 900s corresponding to those belonging to the use/groupnames on the live media rather than those on the full OS. This seems to be the behavior that you were alluding to in your first message. I did not expect it because I thought all the file meta data would come from the tar archive. Do you know of any way of getting the file user/groupnames off the tar archive instead of picking up the user/groupids from the live media in these cases? Or is there no way of preventing this from happening. Just to be clear, all of the files (including chrony) in /var/lib that picked up the 'strange' user/groupids do exist in the live media. In my original bug report I said chrony had a user and group of 977 (from memory), that should have been 997 996 (getting senile!) The /var/lib files from the live media are listed below: ls -lt total 196 drwxr-xr-x. 2 root root 4096 Aug 13 10:01 NetworkManager drwxr-xr-x. 2 root root 4096 Aug 13 10:01 xkb drwx------. 2 rpc rpc 4096 Aug 13 10:01 rpcbind -rw-------. 1 root root 512 Aug 13 10:01 random-seed drwxr-xr-x. 2 root root 4096 Aug 13 10:01 dhclient drwxr-xr-x. 2 root root 4096 Aug 9 10:41 alsa drwxr-xr-x. 2 chrony chrony 4096 Aug 9 10:41 chrony drwxr-xr-x. 3 colord colord 4096 Aug 9 10:36 colord drwxr-xr-x. 2 root root 4096 Aug 6 08:26 rpm drwx------. 3 root root 4096 Aug 6 08:26 authconfig drwxr-xr-x. 6 root root 4096 Aug 6 08:26 yum drwxr-xr-x. 2 root root 4096 Aug 6 08:17 alternatives drwxrwx--T. 3 gdm gdm 4096 Aug 6 08:17 gdm drwxr-xr-x. 2 root root 4096 Aug 6 08:16 prelink drwxr-xr-x. 12 root root 4096 Aug 6 08:14 libvirt drwxr-xr-x. 5 root root 4096 Aug 6 08:14 nfs drwxr-xr-x. 3 root root 4096 Aug 6 08:13 samba drwxr-xr-x. 2 root root 4096 Aug 6 08:13 plymouth drwxr-xr-x. 4 root root 4096 Aug 6 08:13 stateless drwxr-xr-x. 6 root root 4096 Aug 6 08:13 sss drwxr-xr-x. 2 unbound unbound 4096 Aug 6 08:13 unbound drwxr-xr-x. 8 root root 4096 Aug 6 08:12 iscsi drwxr-xr-x. 4 root root 4096 Aug 6 08:12 AccountsService drwxr-xr-x. 3 root root 4096 Aug 6 08:12 rpm-state drwxr-x---. 3 root polkitd 4096 Aug 6 08:12 polkit-1 drwxr-xr-x. 4 root root 4096 Aug 6 08:12 systemd drwxr-xr-x. 4 root root 4096 Aug 6 08:12 net-snmp drwxr-xr-x. 3 root root 4096 Aug 6 08:12 color drwxr-xr-x. 2 root root 4096 Jul 25 13:13 initramfs drwxr-xr-x. 2 root root 4096 Jul 24 23:44 glusterd drwxr-xr-x. 2 root root 4096 Jul 23 11:36 bluetooth drwxr-xr-x. 2 root root 4096 Jul 10 07:34 corosync -rw-r--r--. 1 root root 0 Jul 10 03:39 logrotate.status drwxr-xr-x. 2 root root 4096 Jul 8 04:56 games drwxr-xr-x. 2 root root 4096 Jul 8 04:56 misc drwxr-xr-x. 2 root root 4096 Jul 2 13:02 os-prober drwxr-xr-x. 2 root root 4096 Jul 2 09:40 lldpad drwxr-xr-x. 2 root root 4096 Jun 22 09:33 PackageKit drwxr-xr-x. 2 root root 4096 Jun 21 21:15 selinux drwxr-xr-x. 2 root root 4096 Jun 17 18:23 dbus drwxr-xr-x. 2 root root 4096 Jun 11 09:27 dnsmasq drwx------. 2 tss tss 4096 Jun 2 09:42 tpm drwx------. 2 pulse pulse 4096 May 31 00:36 pulse drwxr-xr-x. 2 root root 4096 May 21 10:32 setroubleshoot drwxr-x---. 2 root slocate 4096 Apr 10 12:57 mlocate drwx------. 2 root root 4096 Apr 3 11:47 rsyslog drwx------. 2 root root 4096 Mar 28 09:11 udisks2 drwxr-xr-x. 2 root root 4096 Mar 20 10:10 upower drwxr-xr-x. 2 root root 4096 Mar 5 10:21 fprint drwxr-xr-x. 2 root root 4096 Feb 18 03:19 sheepdog Thanks so much for your help - Craig
> I tried (and I think I tried this once before): > > tar --acls --xattrs xvf /dev/st0 Well, I see — for restoring of capabilities, you need to specify also --xattrs-include='security.capability'. The problem I re-observed here is that the --xattrs & related options are still not documented in Fedora (as I backported original upstream changes not containing original documentation). Thanks for this - I filled a bug #996753. > [...] > > Do you know of any way of getting the file user/groupnames off the tar > archive instead of picking up the user/groupids from the live media in > these cases? Or is there no way of preventing this from happening. Unpack only files you need again on target system? Fix the UID for 'chrony' by usermod/groupmod (if there is no clash)? Overall, there seems to be no easy solution for this. [OT] Most "efficient" advice from me is either (a) don't reinstall, just upgrade Fedora or (b) reinstall & restore only what you really need to (data & configuration, not system). [/OT] Pavel
> Well, I see — for restoring of capabilities, you need to specify also --xattrs-include='security.capability'. Yes, that worked for me. Thanks - secret information. > Most "efficient" advice from me is either (a) don't reinstall, just upgrade Fedora or (b) reinstall & restore only what you really need to (data & configuration, not system). The problem here is that I am reformatting and partitioning the HDD, and restoring the full F19 OS from a tar tape for disaster recovery eg HDD failure. I would really rather have the file user and group straight off the tape than have tar making use of the /etc/group and similar files on the active system. Was this not the former behavior tar? Craig
> The problem here is that I am reformatting and partitioning the HDD, and > restoring the full F19 OS from a tar tape for disaster recovery eg HDD > failure. I would really rather have the file user and group straight off > the tape than have tar making use of the /etc/group and similar files on the > active system. Then the --numeric-owner is your friend. But note that if there is a UID/GID clash between source/target systems, this will make even more mess. > Was this not the former behavior tar? Can't tell you. At least latest upstream git repository shows that the numeric-owner option was there in that time. I am closing this bugreport as it is not a bug but feel free to continue discussing or reopen if you feel this is really bug.
> At least latest upstream git repository shows I meant the oldest commit :) in upstream repo.
OK, thanks Pavel. With your guidance, I used: tar --numeric-owner --acls --xattrs --xattrs-include='security.capability' -xvf /dev/st0, to perform a full system restore that preserved all of the original file attributes. I realize that I will have take care that the u/gids correspond to the right u/g names. Hope it was not wasted time for you. Cheers - Craig