Bug 723279
Summary: | mlocate works incorrect with bind-mounts | ||
---|---|---|---|
Product: | [Fedora] Fedora | Reporter: | Harald Reindl <h.reindl> |
Component: | mlocate | Assignee: | Michal Sekletar <msekleta> |
Status: | CLOSED EOL | QA Contact: | Fedora Extras Quality Assurance <extras-qa> |
Severity: | unspecified | Docs Contact: | |
Priority: | unspecified | ||
Version: | 34 | CC: | bharring, bugzilla, fedora, heri, info, jan, jan.kratochvil, L.Bonnaud, ralph-fedora453, roshan.shariff, till2.schaefer |
Target Milestone: | --- | Keywords: | Reopened |
Target Release: | --- | ||
Hardware: | Unspecified | ||
OS: | Linux | ||
URL: | https://fedorahosted.org/mlocate/ticket/23 | ||
Whiteboard: | |||
Fixed In Version: | Doc Type: | Bug Fix | |
Doc Text: | Story Points: | --- | |
Clone Of: | Environment: | ||
Last Closed: | 2022-06-08 00:08:44 UTC | Type: | --- |
Regression: | --- | Mount Type: | --- |
Documentation: | --- | CRM: | |
Verified Versions: | Category: | --- | |
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
Cloudforms Team: | --- | Target Upstream Version: | |
Embargoed: | |||
Attachments: |
Description
Harald Reindl
2011-07-19 15:15:07 UTC
Please add specific steps to reproduce - what is the locate command line, what is the file it is supposed to find, etc. what should i add? you have all mount-points above fact is that this worked as long there was a difference between bind-mounts and physical ones in the output of "mount" fact is i find NOTING on my machine inside "/mnt/data" which is not in the bind-mount "/Volumes/dune/www-servers" [harry@rh:~]$ stat /mnt/data/www/www.rhsoft.net/modules/events_edit.php Datei: „/mnt/data/www/www.rhsoft.net/modules/events_edit.php“ Größe: 24331 Blöcke: 48 EA Block: 4096 reguläre Datei Gerät: 902h/2306d Inode: 56366110 Verknüpfungen: 1 Zugriff: (0660/-rw-rw----) Uid: ( 48/ apache) Gid: ( 48/ apache) Zugriff : 2011-01-01 14:20:08.000000000 +0100 Modifiziert: 2011-01-01 14:20:08.000000000 +0100 Geändert : 2011-07-16 12:34:19.096088127 +0200 [harry@rh:~]$ locate events_edit.php /Volumes/dune/www-servers/contentlounge/updateservice/package/modules/events_edit.php /Volumes/dune/www-servers/digital.orf.at/modules/events_edit.php /Volumes/dune/www-servers/hvb/www.andersentag.at/modules/events_edit.php /Volumes/dune/www-servers/hvb/www.buchwoche.at/modules/events_edit.php /Volumes/dune/www-servers/hvb/www.fachkolleg.at/modules/events_edit.php /Volumes/dune/www-servers/hvb/www.hvb.at/modules/events_edit.php /Volumes/dune/www-servers/hvb/www.lesefestwoche.at/modules/events/interface_events_edit.php /Volumes/dune/www-servers/www.hofburg.at/modules/events_edit.php /Volumes/dune/www-servers/www.regionwagram.at/modules/events_edit.php /Volumes/dune/www-servers/www.regionwagram.at/modules/region/interface_events_edit.php /Volumes/dune/www-servers/www.vis.ac.at/modules/events_edit.php Thanks, those two commands were exactly what I was looking for. Thanks again for the information. Please check /etc/updatedb.conf: the default configuration in Fedora 15 includes /mnt (per #674635); is that the case in your setup? If not, please run, as root: > /usr/bin/updatedb -f "$(< /proc/filesystems awk '$1 == "nodev" && $2 != "rootfs" { print $2 }')" --debug-pruning >& log (i.e. take the command from /etc/cron.daily/mlocate.cron , add --debug-pruning and redirect output) and, again as root, > locate / | grep -F events_edit.php and attach both the log generated by updatedb and the output of the locate command. [root@rh:~]$ cat /etc/updatedb.conf PRUNE_BIND_MOUNTS = "yes" PRUNEFS = "9p afs anon_inodefs auto autofs bdev binfmt_misc cgroup cifs coda configfs cpuset debugfs devpts ecryptfs exofs fuse fuse.sshfs fusectl gfs gfs2 hugetlbfs inotifyfs iso9660 jffs2 lustre mqueue ncpfs nfs nfs4 nfsd pipefs proc ramfs rootfs rpc_pipefs securityfs selinuxfs sfs sockfs sysfs tmpfs ubifs udf usbfs" PRUNENAMES = ".git .hg .svn" PRUNEPATHS = "/afs /media /mnt /net /sfs /tmp /udev /var/cache/ccache /var/spool/cups /var/spool/squid /var/tmp" Created attachment 513986 [details]
logoutput of /usr/bin/updatedb --debug-pruning
Created attachment 513987 [details]
content of updatedb.conf
Created attachment 513988 [details]
locate-call which does not show a folder in /mnt/
Created attachment 513989 [details]
ls-output of the folder which is ignored by mlocate
prune = exclude and i am using since 10 years /mnt to mount all of my disks PRUNE_BIND_MOUNTS = "yes" does not work any longer because the new idiotic behavior that even "df" shows all sorts of BIND-Mounts including /var/named/chroot/etc/named i fix this in my "updatedb.conf" and hope this file is flagged as "config_noreplace", but anyways the dumb behavior that BIND-mounts are not flagged for any tool since F15 is one of the badest changes i have ever seen (In reply to comment #10) > prune = exclude and i am using since 10 years /mnt to mount all of my disks I realize the change can be disruptive - at the same time, the FHS does specify /mnt as a place for _temporary_ mounts. > PRUNE_BIND_MOUNTS = "yes" does not work any longer because the new idiotic > behavior that even "df" shows all sorts of BIND-Mounts including > /var/named/chroot/etc/named It does work, as far as I can tell. The observed exclusion of /mnt/data has nothing to do with bind mounts. > i fix this in my "updatedb.conf" and hope this file is flagged as > "config_noreplace", Yes, it is. As the cause of the behavior related to /mnt/data has been discovered, I'm closing this bug; if you have other cases of PRUNE_BIND_MOUNTS not working, please reopen it. > if you have other cases of PRUNE_BIND_MOUNTS not working, please reopen it
Jesus chrsit scroll upwards
[harry@rh:~]$ locate events_edit.php
/Volumes/dune/www-servers/contentlounge/updateservice/package/modules/events_edit.php
"/Volumes/dune/www-servers" IS A BIND MOUNT AND FOUND WITH LOCATE
the problem is that in F15 exists NO WAY to find out any longer
if it is a bind-mount because of braindead changes which are also
affecting "df" and thousands of other things
"mount"-command:
F14:
/data/www on /Volumes/dune/www-servers type none (rw,bind)
F15:
/dev/md2 on /Volumes/dune/www-servers type ext4 (rw,nosuid,noatime,nodiratime,commit=120,barrier=0,journal_async_commit,stripe=256,data=writeback,inode_readahead_blks=64)
Translation: In F15 you get bullshit exactly as the real device
I do not know what package exactly this triggers but whoever made this
change has too less knowledge of UNIX-Systems
(In reply to comment #12) > > if you have other cases of PRUNE_BIND_MOUNTS not working, please reopen it > > Jesus chrsit scroll upwards > > [harry@rh:~]$ locate events_edit.php > /Volumes/dune/www-servers/contentlounge/updateservice/package/modules/events_edit.php > > "/Volumes/dune/www-servers" IS A BIND MOUNT AND FOUND WITH LOCATE Fair enough. I'm afraid I wasn't checking every line in all of that output. > the problem is that in F15 exists NO WAY to find out any longer > if it is a bind-mount There is a way, I just made an error when implementing it (namely, only handling bind mounts within a single device). I'll fix it. > Translation: In F15 you get bullshit exactly as the real device df output doesn't matter for mlocate. This message is a notice that Fedora 15 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 15. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At this time, all open bugs with a Fedora 'version' of '15' have been closed as WONTFIX. (Please note: Our normal process is to give advanced warning of this occurring, but we forgot to do that. A thousand apologies.) Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, feel free to reopen this bug and simply change the 'version' to a later Fedora version. Bug Reporter: Thank you for reporting this issue and we are sorry that we were unable to fix it before Fedora 15 reached end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged to click on "Clone This Bug" (top right of this page) and open it against that version of Fedora. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping I'm afraid this problem still exists in F17 and rawhide. This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component. Another case to consider: mkdir /mnt/{1,2} mount /dev/sda1 /mnt/1 mount --bind /mnt/1/hiding/hidden /mnt/2 # At this point "/mnt/2" is a duplicate bind mount that should be hidden. mount /dev/sda2 /mnt/1/hiding # At this point "/mnt/2" is the unique path to the files and should not be hidden?! kzak suggests basically giving up and setting PRUNE_BIND_MOUNTS=no. This bug appears to have been reported against 'rawhide' during the Fedora 19 development cycle. Changing version to '19'. (As we did not run this process for some time, it could affect also pre-Fedora 19 development cycle bugs. We are very sorry. It will help us with cleanup during Fedora 19 End Of Life. Thank you.) More information and reason for this action is here: https://fedoraproject.org/wiki/BugZappers/HouseKeeping/Fedora19 This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component. This message is a notice that Fedora 19 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 19. It is Fedora's policy to close all bug reports from releases that are no longer maintained. Approximately 4 (four) weeks from now this bug will be closed as EOL if it remains open with a Fedora 'version' of '19'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 19 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed. This message is a reminder that Fedora 21 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 21. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '21'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 21 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. +1 to fix this ridiculous how many years it takes to get the bad impact of careless changes like this bug and https://bugzilla.redhat.com/show_bug.cgi?id=1001092 takes this makes *so much* more troubles than /etc/mtab made für a whole decade This message is a reminder that Fedora 23 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 23. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '23'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 23 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. mlocate-0.26-14.fc24.x86_64 /export/root/quad/home/lace/arch/video-copy/vp8/H264.mp4 /quad/home/lace/arch/video-copy/vp8/H264.mp4 fstab: /quad /export/root/quad none bind,noauto 0 0 rc.local: mount --target /export/root/quad i am impressed that changes with regressions left and right are introduced and *5 years* later the fallout is still not fixed This message is a reminder that Fedora 24 is nearing its end of life. Approximately 2 (two) weeks from now Fedora will stop maintaining and issuing updates for Fedora 24. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '24'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 24 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. Fedora 24 changed to end-of-life (EOL) status on 2017-08-08. Fedora 24 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed. mlocate-0.26-23.fc30.x86_64 This message is a reminder that Fedora 30 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '30'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 30 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. mlocate-0.26-24.fc31.x86_64 This message is a reminder that Fedora 31 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '31'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 31 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. mlocate-0.26-26.fc33.x86_64 I'm now experiencing this on Fedora 33 with the introduction of BTRFS as the installation filesystem. Simply put: I have the default updatedb.conf: [~]$ rpm -Vv mlocate | grep -F "/etc/updatedb.conf" ......... c /etc/updatedb.conf Both / (root) and /home are subvolumes of the same device (as executed in the install): [~]$ mount | grep -iF btrfs /dev/mapper/luks-73552c7c-b0e3-4ae8-9da1-6876b192af30 on / type btrfs (rw,relatime,seclabel,ssd,space_cache,subvolid=2336,subvol=/root00) /dev/mapper/luks-73552c7c-b0e3-4ae8-9da1-6876b192af30 on /home type btrfs (rw,relatime,seclabel,ssd,space_cache,subvolid=256,subvol=/home00) Unfortunately, running "updatedb" does not index anything in the /home volume by default, but will in the event that "--prune-bind-mounts=no" is set: [~]$ sudo updatedb -v | grep -i -F "/home/bharrington" [~]$ sudo updatedb -v --prune-bind-mounts=no | grep -i -F "/home/bharrington" | head -n 100 /home/bharrington /home/bharrington/.mozilla /home/bharrington/.cache /home/bharrington/.config /home/bharrington/.local ... As such, I'd say this now affects MANY more users as the default behavior touches anyone who has performed a default installation. @bharring: you are probably running into a different bug #906591. mlocate-0.26-28.fc34.x86_64 Created attachment 1779053 [details] Fix for the package. https://koji.fedoraproject.org/koji/taskinfo?taskID=67150871 Now it just considers mounts which have the same: * device major+minor * device name ("source") * filesystem type It sorts them by root (usually "/" except for binds and btrfs) and then by mount point - sorting by number of slashes first. And all except the first one in each group are considered bind mounts. It has turned my: 2.7G /var/lib/mlocate/mlocate.db -> 1.2G /var/lib/mlocate/mlocate.db ping, is the mlocate package maintained? If not Fedora should switch to some other locate package with maintained upstream. @Christian Stadelmann the underlying bullshit is the same root cause as https://bugzilla.redhat.com/show_bug.cgi?id=1001092#c12 and it's a shame how long it takes to a) revert such crap changes when the ecosystem can't cope with it or b) adopt the fucking ecosystem below a full decade normally common sense would dictate that the underlying change and adoption in userland is pushed *at the same time* but in open source development there is no common sense because it's normal breaking things left and right This seems like a dup of bug 906591. And in that bug it's pretty clear the bind mount handling is confusing things, both on Btrfs but also rpm-ostree based installations which make use of bind mounts. Silverblue solves this by using upstream's default which curiously Fedora is not using (I don't understand this). https://src.fedoraproject.org/rpms/mlocate/c/2fbdf7fbe91e3f37ed952724dae1bb0dbf6422ba?branch=rawhide On conventional installations, this appears to solve the problem on Btrfs. -PRUNE_BIND_MOUNTS = "yes" +#PRUNE_BIND_MOUNTS = "yes" (In reply to Chris Murphy from comment #43) > This seems like a dup of bug 906591. I have never tried btrfs nor Ostree myself. So I do not see how it could be related. It is true I see one should test this fix on btrfs and Ostree setups. Is enough the default Anaconda Btrfs installation an default Ostree spin? Personally I need bind mounts for NFSv4 setup in /etc/fstab: /root /export/root/root none bind,noauto 0 0 /home /export/root/home none bind,noauto 0 0 /usr /export/root/usr none bind,noauto 0 0 /etc /export/root/etc none bind,noauto 0 0 /tmp /export/root/tmp none bind,noauto 0 0 /var /export/root/var none bind,noauto 0 0 /quad /export/root/quad none bind,noauto 0 0 Maybe there exists some other NFSv4 solution but still bind mounts are a normal OS feature so why mlocate should not work with it? If it conflicts with Btrfs/Ostree setups maybe mlocate could have some adjustment of this fix for those. In fact, the issue with mlocate and bind mounts happens even without btrfs or NFS. The systemd unit that runs mlocate has ProtectSystem=true on by default, which causes it to run in a mount namespace with a bind-mounted read-only /usr directory, leading to a very incomplete db. See bug #1883734. @Roshan Shariff the issue here has *nothing* to do with ProtectSystem=true because it was an issue long before the switch to a systemd-unit nor are kernel-namespaces bind-mounts the underlying issue is "/etc/mtab -> ../proc/self/mounts" years ago and that you no longer can distinct bind-mounts and the real mountpoint from there but it's easy to distinct from the output of "findmnt --df" it just has to be done see my inital bugreport on top the problem is that it takes a random bind-mount instead the real mount-point *both* is nonse a) use a random bind-mount and b) show the same file dozens of times @Harald Reindl: I agree that mlocate's updatedb needs better logic to de-duplicate bind mounts, and I was hopeful that Jan Kratochvil's patch would fix some of these issues. And if not, perhaps it would be worth either copying the findmnt logic or just calling it. I was just trying to point out that with ProtectSystem=true, even /usr appears as a bind mount, so the problem is even more severe now (all files in /usr are missing). And it is a tricky edge case for the de-duplication logic, since /usr is a read-only bind mount shadowing itself. This message is a reminder that Fedora Linux 34 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 34 on 2022-06-07. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '34'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 34 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed. Fedora Linux 34 entered end-of-life (EOL) status on 2022-06-07. Fedora Linux 34 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. Thank you for reporting this bug and we are sorry it could not be fixed. |