Hide Forgot
Description of problem: Copying files using "tar" drops file capabilities. It appears this tool does not even provide any options to deal with such attributes or ACLs or ... Guess how many system backups are actually not working correctly? Version-Release number of selected component (if applicable): tar-1.26-2.fc16.x86_64 How reproducible: 100% Steps to Reproduce: 1. Select some files that have capabilities set: # cd /usr/bin # getcap -v rcp rlogin rsh rcp = cap_net_bind_service+ep rlogin = cap_net_bind_service+ep rsh = cap_net_bind_service+ep 2. Copy files to some other place: # rm -fr /tmp/foo # mkdir /tmp/foo # tar cf - rcp rlogin rsh | ( cd /tmp/foo ; tar xpvf - ) rcp rlogin rsh 3. Check capabilities on the copies. In all cases we get: # getcap -v /tmp/foo/* /tmp/foo/rcp /tmp/foo/rlogin /tmp/foo/rsh I. e., the (needed for correct function) capabilities are lost. Actual results: Copies have their capabilities lost. Expected results: There should be a way that all tools that deal with archiving, restoring and copying of files not only include the easily visible ownership and file permission information, but also the more delicate and less visible features like capabilities. See bugs # 771134 and 693731
At the moment, there is not even acl/xattrs support in upstream tar ... and we use our patch for it - using posix tar format. However - as POSIX.1-2001 format has extended header, it is probably doable within it. It would still be good to be compatible with star ( http://cdrecord.berlios.de/private/man/star/star.4.html ) if using this format.
Interestingly, in Fedora 16, strace of "tar -cf foo.tar --xattrs /bin/ping" shows this: fgetxattr(4, "system.posix_acl_access", 0x7fff81e827f0, 132) = -1 ENODATA (No data available) fgetxattr(4, "security.selinux", "system_u:object_r:ping_exec_t:s0", 255) = 33 flistxattr(4, 0x1bbd590, 1024) = 37 fgetxattr(4, "security.capability", "\x01\x00\x00\x02\x00 \x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00", 1024) = 20 fgetxattr(4, "security.selinux", "system_u:object_r:ping_exec_t:s0", 1024) = 33 But the corresponding extract only shows this: setxattr("ping", "system.posix_acl_access", "\x02\x00\x00\x00\x01\x00\x07\x00\xff\xff\xff\xff\x04\x00\x05\x00\xff\xff\xff\xff \x00\x05\x00\xff\xff\xff\xff", 28, 0) = 0 setxattr("ping", "security.selinux", "system_u:object_r:ping_exec_t:s0", 33, 0) = This looks odd because acls are being set when none are read and the security.capability xattr is read but not saved. Perhaps the xattrs patch just has a straightforward bug? (I haven't checked yet.)
(In reply to comment #2) > Perhaps the xattrs patch just has a straightforward bug? Fedora tar does not save/restore all extended attributes. There is a hard-wired list of namespaces we are interested in. It may be worth to consider a more generic approach as taken e.g. in Gentoo Linux: https://bugs.gentoo.org/show_bug.cgi?id=382067#c9
*** Bug 832494 has been marked as a duplicate of this bug. ***
We certainly need the security.capability since we turn this on by default in Fedora 16/17. Ondrej can you at least change the tar patch to support both SELinux and capability security attributes?
As Kamil said, we will probably completely remove the namespace limits, similar to the gentoo approach. This is not the first time someone needs some category to be added. For F16/F17, just adding security.capability to stored namespaces is probably the right way. I'll make that change now...
tar-1.26-6.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/tar-1.26-6.fc17
tar-1.26-3.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/tar-1.26-3.fc16
Package tar-1.26-3.fc16: * should fix your issue, * was pushed to the Fedora 16 testing repository, * should be available at your local mirror within two days. Update it with: # su -c 'yum update --enablerepo=updates-testing tar-1.26-3.fc16' as soon as you are able to. Please go to the following url: https://admin.fedoraproject.org/updates/FEDORA-2012-9561/tar-1.26-3.fc16 then log in and leave karma (feedback).
tar-1.26-6.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.
tar-1.26-7.fc17 has been submitted as an update for Fedora 17. https://admin.fedoraproject.org/updates/tar-1.26-7.fc17
tar-1.26-4.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/tar-1.26-4.fc16
tar-1.26-3.fc16 has been pushed to the Fedora 16 stable repository. If problems still persist, please make note of it in this bug report.
tar-1.26-7.fc17 has been pushed to the Fedora 17 stable repository. If problems still persist, please make note of it in this bug report.
I just tested tar-1.26-3.fc16.x86_64, and it does not fix the bug. It seems as if the patch was omitted from the rpm. If I unpack the source rpm for tar-1.26-2.fc16 and the new rpm for tar-1.26-3.fc16, here is the diff: bash$ diff -qr tar* Only in tar-1.26-3.fc16.src.rpm: tar-1.26-update-with-change-directory.patch Files tar-1.26-2.fc16.src.rpm/tar.1 and tar-1.26-3.fc16.src.rpm/tar.1 differ Files tar-1.26-2.fc16.src.rpm/tar.spec and tar-1.26-3.fc16.src.rpm/tar.spec differ As you can see, there is no patch related to the file capabilities. Did somebody forget to put it in the rpm? Regards, Andy
Capability support is part of tar-1.24-xattrs.patch distribution patch, so no new patch was required. However, due to order of fchown() and xattrs restoring, capabilities were reset by fchown(), you need to update to https://admin.fedoraproject.org/updates/tar-1.26-4.fc16 - as you can see in comment #12
Ah, damned, in f16 update the relevant part of the xattr patch was not modified. Will make an update soon.
tar-1.26-5.fc16 has been submitted as an update for Fedora 16. https://admin.fedoraproject.org/updates/tar-1.26-5.fc16
tar-1.26-5.fc16 has been pushed to the Fedora 16 stable repository. If problems still persist, please make note of it in this bug report.
It stopped working with tar-1.26-27.fc19.x86_64: [root@hermes ~]# mkdir /tmp/rhbz771927 [root@hermes ~]# cd /tmp/rhbz771927 [root@hermes rhbz771927]# mkdir src dst [root@hermes rhbz771927]# touch src/foo [root@hermes rhbz771927]# setcap cap_net_admin,cap_net_raw+ep src/foo [root@hermes rhbz771927]# getcap src/foo src/foo = cap_net_admin,cap_net_raw+ep [root@hermes rhbz771927]# tar -c --selinux --acls --xattrs -f - -C src foo | \ tar -x --selinux --acls --xattrs -f - -C dst [root@hermes rhbz771927]# getcap dst/foo [root@hermes rhbz771927]#
Hi Cristian, tar does not extract extended attributes other than user.* by default. This are usually not really "extended attributes", but other features which use extended attributes as its storage place. Usually, non-root users are unable to set these. Thus, capabilities are stored under 'security.capabilities', if you are curious - try to run `getfattr -m. /usr/bin/ping`. The correct extraction should be (in f19+) like this: tar -x --selinux --acls --xattrs-include='*' -f - -C dst Feel free to reopen if that does not work for you, Pavel
This behaviour is totally unexpected for the user. The man page does not provide any help here; it does not include any example either. OK, the info page says: By default, all extended attributes are stored into the archive. ... On the other hand, during extracting only the 'user.*' domain is extracted by default. Such asymmetric behaviour is always a pain in the neck. Why cannot we have symmetric behaviour, such that when run as root user the tar restore will just do "the right thing" and restore all settings that were archived? When running as non-root user, proper "Cannot setcap: Operation not permitted" (or similar) errors should be emitted. This way we at least would know that certain parts of the original file settings were not restored. It make sense to me to explicitly enable support for acls and xattrs, but anything beyond this level should be optional and only be needed if you want to tweak the behaviour. The default behaviour should not only be to "make the backup process as complete as possible by default" (see info page in the '--xattrs' section) - we should also do this upon restore. Needing just "--xattrs" for one operation and another set of options for the reverse operation is really poor design. Note that the info page also fails to provide an example how to correctly restore an archive with all potentially contained information and settings.
(In reply to Wolfgang Denk from comment #22) > This behaviour is totally unexpected for the user. The man page does > not provide any help here; it does not include any example either. That is truth. Manual page (which is a little bit enhanced --help output, jfyi) should shortly say what will be extracted (I'll mention the user.* prefix). > OK, the info page says: > > By default, all extended attributes are stored into the > archive. ... On the other hand, during extracting only > the 'user.*' domain is extracted by default. > > Such asymmetric behaviour is always a pain in the neck. Why cannot we > have symmetric behaviour, such that when run as root user the tar > restore will just do "the right thing" and restore all settings that > were archived? I don't see extracting capabilities by default as "doing the right thing". User should not expect capabilities to be restored; the relation to --xattrs is JUST implementation detail. Once something like --capabilities option comes into tar, the relation to --xattrs disappears. But, here and .... > When running as non-root user, proper "Cannot setcap: > Operation not permitted" (or similar) errors should be emitted. > This way we at least would know that certain parts of the original > file settings were not restored. ... even here: It seems to me you are asking for some --extract-all option (which is asking for separate bugreport, good point btw). > It make sense to me to explicitly enable support for acls and xattrs, > but anything beyond this level should be optional and only be needed > if you want to tweak the behaviour. And what about capabilities? How these differ from acls? We need to incorporate libcap.so (blind idea) and use --capabilities.. > The default behaviour should not only be to "make the backup process as > complete as possible by default" (see info page in the '--xattrs' section) - > we should also do this upon restore. The reason we store capabilities also in --xattrs input is to not cause catastrophic scenarios for users. Just to not loose data. > Needing just "--xattrs" for one operation and another set of options for the > reverse operation is really poor design. Don't expect --xattrs store/restore capabilities and it will be fine. Consider storing *all* xattrs (including capabilities) as safety belt (you are still able to restore everything, right?). > Note that the info page also fails to provide an example how to correctly > restore an archive with all potentially contained information and settings. That is again good point which should result in separate bugzilla request for something like --extract-all. Pavel
(In reply to Pavel Raiskup from comment #23) > > > Such asymmetric behaviour is always a pain in the neck. Why cannot we > > have symmetric behaviour, such that when run as root user the tar > > restore will just do "the right thing" and restore all settings that > > were archived? > > I don't see extracting capabilities by default as "doing the right thing". > User should not expect capabilities to be restored; the relation to --xattrs is What makes you think so? Users expect to have "exact copies" restored, including all properties the file had when it was backed up. I think there is a pretty significan majority of user (eventually even an overwhelming one) who just expect that tar backup and tar restore are truely inverse operations, and that they can be used to identically restore a copy of file archived before. I don't know how many documents exist that claim that something like tar -cf - | ( cd somewhere; tar -xpf - ) can be used for reliable copies. I don;t remember how many sysadmins I've seen using this method to copy root file systems - and it has been working perfectly fine for decades. Now we have ACLs and XATTRs and Capabilities and whatever, and things break down. This would still be somewhat acceptable if there were clear warnings like "this file has additional properties that are not backed up" and especially error message like "unable to restore this or that property". But what we have now is something that silently works, but the result is not what the user expects. > > When running as non-root user, proper "Cannot setcap: > > Operation not permitted" (or similar) errors should be emitted. > > This way we at least would know that certain parts of the original > > file settings were not restored. > > ... even here: It seems to me you are asking for some --extract-all option > (which is asking for separate bugreport, good point btw). No. This is actually NOT what I'm asking for. I'm asking to make this behaviour the DEFAULT.. > > It make sense to me to explicitly enable support for acls and xattrs, > > but anything beyond this level should be optional and only be needed > > if you want to tweak the behaviour. > > And what about capabilities? How these differ from acls? We need to > incorporate libcap.so (blind idea) and use --capabilities.. In this context I don't care what the name of such properties is. I'm playing the role of an end user here, who just wants to archive and restore a file with all properties it has - the user normally does not even care if there are ACLs or XATTRs or Capabilities assigned for the file - he just expects that he can make a backup, nd that restoring from the backup will give him an axact copy with the very same behaviour. It is already a major pain that we explicitly have to enable support for such features, and it would be indeed nice if "--preserve" would automatically preserve ALL properties, not only time and owner. And restore ALL properties, of course. > > The default behaviour should not only be to "make the backup process as > > complete as possible by default" (see info page in the '--xattrs' section) - > > we should also do this upon restore. > > The reason we store capabilities also in --xattrs input is to not cause > catastrophic scenarios for users. Just to not loose data. But this is the very problem: we do lose date on restore, and silently, without even a warning. Restoring a file from an archive will NOT create an identical copy any more. > > Needing just "--xattrs" for one operation and another set of options for the > > reverse operation is really poor design. > > Don't expect --xattrs store/restore capabilities and it will be fine. > Consider > storing *all* xattrs (including capabilities) as safety belt (you are still > able > to restore everything, right?). As mentioned before, I'm not focussed on capabilities here - for the average user, this is just one of the rarely used, usually not even noticed properties a file may have. Archiving and restoring should always preserve such information to the extend possible, and issue clear messages when any such information is lost (at archival) or cannot be restored (at restore). Just my $ 0.02, of course...
This bug is "RFE: tar should support file capabilities" and our default tar (GNU tar) supports it. I see your concerns regarding the default behavior, Wolfgang. Sadly, there is IMO not much we can do for your request via this resolved bugreport. > tar -cf - | ( cd somewhere; tar -xpf - ) 100% expectation that this will work the same way among all systems is bad. You always must take into account where the 'tar -p' command is run. The problem is that what must be stored/restored by default is not defined (even the -p option is not defined), so you can't expect anything. As tar is marked LEGACY by POSIX, I would say that there is high chance that this will never get changed. My 2 cents, as Fedora & GNU tar user: I don't want to produce tarballs having non standard attributes inside BY DEFAULT (it causes unnecessary troubles to users from other distributions, or non-Linux users - they must deal with warnings, etc.). We have a options for that if I know what I am doing. If you wan't to store/extract literally everything by default, please could you raise a bug for it? Whether we should or should not do it is more like principal question we may not answer in isolation. I can raise an upstream question (I mean for all tar implementations like GNU tar, star, libarchive and others?) though I am not sure I am good person to protect that idea.. So probably somebody more interested to protect that should raise it..? Pavel
(In reply to Pavel Raiskup from comment #21) > Hi Cristian, tar does not extract extended attributes other than user.* > by default. This are usually not really "extended attributes", but other > features which use extended attributes as its storage place. Usually, > non-root users are unable to set these. Thus, capabilities are stored under > 'security.capabilities', if you are curious - try to run > `getfattr -m. /usr/bin/ping`. I don't see why tar should care how those extended attributes are interpreted. It should archive & extract all of them, since it's an archiver. > The correct extraction should be (in f19+) like this: > tar -x --selinux --acls --xattrs-include='*' -f - -C dst For the record, that command works. (In reply to Wolfgang Denk from comment #24) > > I don't know how many documents exist that claim that something like > > tar -cf - | ( cd somewhere; tar -xpf - ) > > can be used for reliable copies. I don;t remember how many sysadmins > I've seen using this method to copy root file systems - and it has > been working perfectly fine for decades. By the way, the -p parameter isn't needed for sysadmins because: -p, --preserve-permissions, --same-permissions extract information about file permissions (default for superuser) Something similar should be done for capabilities. They should be restored by default at least by the superuser. (In reply to Pavel Raiskup from comment #25) > This bug is "RFE: tar should support file capabilities" and our default tar > (GNU tar) supports it. I see your concerns regarding the default behavior, > Wolfgang. Sadly, there is IMO not much we can do for your request via this > resolved bugreport. So another bug report is needed? > > tar -cf - | ( cd somewhere; tar -xpf - ) > > 100% expectation that this will work the same way among all systems is bad. > You always must take into account where the 'tar -p' command is run. > > The problem is that what must be stored/restored by default is not defined > (even the -p option is not defined), so you can't expect anything. As tar is > marked LEGACY by POSIX, I would say that there is high chance that this will > never get changed. By the way, what other archivers that support capabilities (properly) could be used?
(In reply to Cristian Ciupitu from comment #26) > (In reply to Pavel Raiskup from comment #21) > > Hi Cristian, tar does not extract extended attributes other than user.* > > by default. This are usually not really "extended attributes", but other > > features which use extended attributes as its storage place. Usually, > > non-root users are unable to set these. Thus, capabilities are stored under > > 'security.capabilities', if you are curious - try to run > > `getfattr -m. /usr/bin/ping`. > > I don't see why tar should care how those extended attributes are > interpreted. It should archive & extract all of them, since it's an > archiver. Full agreement here. This is what the average user would expect. > > can be used for reliable copies. I don;t remember how many sysadmins > > I've seen using this method to copy root file systems - and it has > > been working perfectly fine for decades. > > By the way, the -p parameter isn't needed for sysadmins because: > > -p, --preserve-permissions, --same-permissions > extract information about file permissions > (default for superuser) > > Something similar should be done for capabilities. They should be > restored by default at least by the superuser. Agreed. > (In reply to Pavel Raiskup from comment #25) > > This bug is "RFE: tar should support file capabilities" and our default tar > > (GNU tar) supports it. I see your concerns regarding the default behavior, > > Wolfgang. Sadly, there is IMO not much we can do for your request via this > > resolved bugreport. > > So another bug report is needed? This is my understanding, too. > By the way, what other archivers that support capabilities (properly) > could be used? Properly? None...
Sorry, but simply archiving the binary "security.capability" extended attribute is a very bad idea. Please take a look in "include/uapi/linux/capability.h" from the linux kernel. The developers write there: 20 /* User-level do most of the mapping between kernel and user 21 capabilities based on the version tag given by the kernel. The 22 kernel might be somewhat backwards compatible, but don't bet on 23 it. */ and later we find that there already 3 (in words: three) dirrent versions of the binary data for the extended attribute. Soome kernel versions are only supporting some of these versions: 31 #define _LINUX_CAPABILITY_VERSION_1 0x19980330 34 #define _LINUX_CAPABILITY_VERSION_2 0x20071026 /* deprecated - use v3 */ 37 #define _LINUX_CAPABILITY_VERSION_3 0x20080522 Version 1 is supported for kernel 2.6.24 and above. Version 2 is supported for kernel 2.6.25 and above. Dunno from which kernel version the capability version 3 is supported. You see, it is quite simple to get into problems if you only have an archive with the binary representation of the capabilities. Currently I am porting my patches to GNU tar 1.26 to GNU tar 1.27. These patches include a proposal how to deal with file capabilities (AFAIK no currecnt tar implementation can properly process them) - with a backward compatibility layer for archives that have been created with binary file capabilities (letting libcap doing the conversion twice to get a file system capability of the right version).
Created attachment 849830 [details] Patch to support file capabilities
Note that my above patch needs the patches from Bug #1052876 to be applied before.
What is the status of the above patch? Has this been incorporated into any Fedora distribution?
(In reply to Ani Sinha from comment #31) > What is the status of the above patch? Well I am not sure which patch you mean -- if the latest one, I am not sure whether this was proposed upstream or not, Markus? If not, Please, try to propose your work as you are original author -- that will make 100% probability of getting this into (not only) Fedora the fastest way. > Has this been incorporated into any Fedora distribution? No, we really need to have this _upstream first_. Until then, I am not going to risk we will keep downstream incompatibilities forever. Historically, that way it was done for xattrs itself and it was quite a few work to get it upstream (and it could be seen as quite offensive to GNU tar upstream). No need to repeat the history - GNU tar folks are pretty communicative and tar itself had several releases in latest years. This is the way to go. Pavel
(In reply to Pavel Raiskup from comment #32) > (In reply to Ani Sinha from comment #31) > > What is the status of the above patch? > > Well I am not sure which patch you mean -- I am talking about the patch in comment#29 which supports file capabilities in GNU tar. I would really like to see this feature pushed upstream. Markus, please enlighten us. thanks.
I see. Notice that we have to get the patches mentioned in Comment#30 upstream before. I sent upstream a remainer, see http://lists.gnu.org/archive/html/bug-tar/2014-07/msg00024.html . The patch from Comment#29 has been mentioned upstream ( can find http://lists.gnu.org/archive/html/bug-tar/2014-02/msg00019.html quite easy ), but without link - so they are aware there is one.
(In reply to M. Steinborn from comment #34) > I see. Notice that we have to get the patches mentioned in Comment#30 > upstream before. Any ETA for this?
Can you please escalate this? I don't see any response from anyone else from your email. We kind of need this ASAP.
Escalate where? This is Fedora, reminder was sent to upstream (as written in https://bugzilla.redhat.com/show_bug.cgi?id=771927#c34 ) . We need to wait for upstream inclusion at least to the git head.
(In reply to Ondrej Vasik from comment #37) > Escalate where? I meant escalate this with the upstream guys.
As written in http://lists.gnu.org/archive/html/bug-tar/2014-01/msg00032.html , Marcus didn't proposed capability patch upstream so far, as it will likely need some adjustments for upstream inclusion. First, there are other patches to be reviewed and included. We are not going to include this independently on upstream to Fedora, we did so in the case of extended attributes and it was very expensive to maintain (every rebase meant quite a lot of work and it took ~6 years to get xattrs included upstream). It is very likely capabilities will not take so long. You can try to escalate it yourself on bug-tar , if you believe this will make it happen faster. Marcus sent reminder for the previous patchset ~1 week ago - without response - but as it is PTO time, it might be expectable.
This message is a notice that Fedora 19 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 19. It is Fedora's policy to close all bug reports from releases that are no longer maintained. Approximately 4 (four) weeks from now this bug will be closed as EOL if it remains open with a Fedora 'version' of '19'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 19 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Still there, as far as I understand. Could anyone bump the affected version? No such a possibility for me.
(In reply to Andrew Travneff from comment #41) > Still there, as far as I understand. Could anyone bump the affected version? To 20 or 21?
I see nothing clearly related in tar-1.27.1-7.fc21 changelog and docs. So it's 21, I think.
And I think Rawhide might be a good choice. The problem is not solved upstream.
Created attachment 1164880 [details] Patch rebased and updated to tar 1.29
Hi Marcus, is upstream aware of this patch?
Sorry, comment #39 is still up to date. But I am quite sure that I mentioned the URL to this bugzilla bug report, so combined with the linked bug-tar mail (again comment #39) upstream should be aware (cf comment #34 here). Anyway, I am using my patches on top of the redhat patches. For upstream, I need to test them on top of tar 1.29 and/or tar git master first. Anyway, I'll perpare a new posting to bug-tar this week. Comment #28 here is a good reason why this is wanted.
> Anyway, I am using my patches on top of the redhat patches. For upstream, I > need to test them on top of tar 1.29 and/or tar git master first. Thanks for the update. > Anyway, I'll perpare a new posting to bug-tar this week. Comment #28 here is > a good reason why this is wanted. Makes sense, I'm aware of those reasons. Binary form of capabilities might be subject of change of course.
According to the low priority of this request and as it did not bother any user for years, I am closing this tracker. If you think this issue should be handled and investigated, feel free to reopen it.
Does it mean that tar will not support saving and restoring file capabilities? Or is it related just with specific implementation (libcap)? There are another similar tickets, and it bothers users, I think bug #771926, bug #771134, bug #832494
Thank for your reply, we will try to communicate this with upstream ans see what we can do. Fro tracking reasons leaving this tracker open.
> Does it mean that tar will not support saving and restoring file capabilities? GNU tar supports storing and extracting capabilities, but it is not binary-safe. `tar -x -f <file> --xattrs-include='security.*` should do that, you _have to_ check _yourself_ that the extracted binary format is compatible with the system where you are extracting.