Hide Forgot
Description of problem: Files that I know exist in my home directory were not displayed by the locate command. The problem was traced to updatedb not indexing /home, because it believes /home is a bind mount for some reason. Disabling the PRUNE_BIND_MOUNT option in /etc/updatedb.conf fixed the problem. However I believe the real bug is related to btrfs and bind mount detection. Version-Release number of selected component (if applicable): mlocate 0.26 for Fedora 18 How reproducible: Every time. Steps to Reproduce: 1. Make a btrfs partition and mount it as /home. 2. Create a home directory. 3. Place some files in it. 4. Run updatedb's cron script. 5. locate filename Actual results: filename not found Expected results: /home/user/filename Additional info:
Thanks for your report. Could you collect a debug log, please? 1. Edit the last line of /etc/cron.daily/mlocate.cron to read /usr/bin/updatedb --debug-pruning -f "$nodevs" 2. run bash /etc/cron.daily/mlocate.cron |& tee log 3. Restore /etc/cron.daily/mlocate.cron to the original (remove --debug-pruning) 4. Attach the generated "log" file to this bug.
Created attachment 694697 [details] This is with PRUNE_BIND_MOUNTS = "no" in /etc/updatedb.conf
Created attachment 694698 [details] This is with PRUNE_BIND_MOUNTS = "yes" in /etc/updatedb.conf
Hi Jonathan, Could you please provide output of findmnt?
Created attachment 695023 [details] Output of findmnt (root and home @ btrfs) I'm experiencing same/similar problem. I have both / and /home as different volumes on one btrfs filesystem and /home is pruned as bind mount. Attaching output of my findmnt.
Hi Petr, Thanks for the info. It looks like mlocate cannot differentiate btrfs subvolume from bind mount. What I noticed in /proc/self/mountinfo, while reproducing this issue, is that the only "thing" that could possibly show us the two volumes are not bind mounts, is the peer group number - bind mounts have the same peer group number, btrfs subvolumes have different numbers. Miroslav, would it be safe to use this to check whether we have bind mount or the subvolume?
(In reply to comment #6) > Thanks for the info. It looks like mlocate cannot differentiate btrfs > subvolume from bind mount. Yes, both are the same file system on the same device. > What I noticed in /proc/self/mountinfo, while reproducing this issue, is > that the only "thing" that could possibly show us the two volumes are not > bind mounts, is the peer group number - bind mounts have the same peer group > number, btrfs subvolumes have different numbers. > > Miroslav, would it be safe to use this to check whether we have bind mount > or the subvolume? I don't know, aren't peer groups related to mount namespaces? That sounds completely unrelated to bind mounts. Karel, could you help us answer the above question, or suggest an alternative way to distinguish btrfs subvolumes vs. bind mounts of the same subvolume?
(In reply to comment #7) > I don't know, aren't peer groups related to mount namespaces? That sounds > completely unrelated to bind mounts. Do you mean "shared:X" in or the mount ID (the first column)? The dominant peer group (e.g. shared:X) is really about propagation flags, try mount --make-private /mountpoint and the information disappear. > Karel, could you help us answer the above question, or suggest an > alternative way to distinguish btrfs subvolumes vs. bind mounts of the same > subvolume? Well, the subvolume could be also accessible from btrfs filesystem root (if mounted), you can also mount the same filesystem more than once, etc. IMHO it would be better to not care about subvolumes and bind mounts in applications like mlocate. All you need is to detect mountpoints (just compare directory devno and parental directory devno) and check if the mountpoint devno as returned by stat(2) is *unique*. That's all. The same subvolume has the same devno everywhere. mount /dev/sdb1 /mnt/test btrfs subvolume create /mnt/test/sub mount /dev/sdb1 /mnt/test2 -o subvol=sub mount --bind /mnt/test/sub /mnt/test3 mount --bind /mnt/test2 /mnt/test4 now the same subvolume is accessible by 4-ways: 1: from filesystem root /mnt/test/sub 2: from subvolume mountpoint /mnt/test2 3: from bind to subvolume in fs root 4: from bind to subvolume mountpoint and all the places share the same devno (stat(2) st.st_dev) and the devno is different to btrfs filesystem root (/mnt/test in my example). So it's pretty simple to detect duplicate directories. I don't see reason to use /proc/self/mountinfo at all. Just for the record, see mountinfo: $ grep mnt/test /proc/self/mountinfo 116 32 0:35 / /mnt/test rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache 119 32 0:35 /sub /mnt/test2 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache 122 32 0:35 /sub /mnt/test3 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache 125 32 0:35 /sub /mnt/test4 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache there is no information about bind mounts or subvolumes. All you can found is subdirectory (fs-root, 4th column) used for the target mountpoint.
(In reply to comment #8) > > Karel, could you help us answer the above question, or suggest an > > alternative way to distinguish btrfs subvolumes vs. bind mounts of the same > > subvolume? > > Well, the subvolume could be also accessible from btrfs filesystem root (if > mounted), you can also mount the same filesystem more than once, etc. > > IMHO it would be better to not care about subvolumes and bind mounts in > applications like mlocate. mlocate cares so that it doesn't present a single path N times in presence of bind mounts, which users find redundant. > mount /dev/sdb1 /mnt/test > btrfs subvolume create /mnt/test/sub > > mount /dev/sdb1 /mnt/test2 -o subvol=sub > mount --bind /mnt/test/sub /mnt/test3 > mount --bind /mnt/test2 /mnt/test4 > > now the same subvolume is accessible by 4-ways: > > 1: from filesystem root /mnt/test/sub > 2: from subvolume mountpoint /mnt/test2 > 3: from bind to subvolume in fs root > 4: from bind to subvolume mountpoint > > and all the places share the same devno (stat(2) st.st_dev) and the devno is > different to btrfs filesystem root (/mnt/test in my example). In the dump below, the device seems to be 0:35 both for the root volume at /mnt/test and the subvolumes; is it really supposed to be different? > $ grep mnt/test /proc/self/mountinfo > 116 32 0:35 / /mnt/test rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache > 119 32 0:35 /sub /mnt/test2 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache > 122 32 0:35 /sub /mnt/test3 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache > 125 32 0:35 /sub /mnt/test4 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache Anyway, looking closer, we might not actually need a detection of subvolumes. An easy way to detect btrfs subvolumes would definitely allow and easy and quick fix of this case, but a general fix for bug 723279 (if and when that happens) would probably fix this case as well.
(In reply to comment #9) > In the dump below, the device seems to be 0:35 both for the root volume at > /mnt/test and the subvolumes; is it really supposed to be different? Yes, btrfs uses the same devno in mountinfo for all stuff related to the filesystem. And the mountinfo devno is different to stat(2) st_dev :-)
I've just run into this too # updatedb --debug-pruning 2>&1 | grep home `/home' (42 on 34) is `/home' of `/dev/mapper/luks-5c0b8da3-45ed-4562-b8ae-7939af586c6d' (0:29), type `btrfs' => adding `/home' `/home', type `btrfs' `/home', type `btrfs' Skipping `/home': bind mount the bad thing about this is that it is the installer default to create /home as subvolume
This package has changed ownership in the Fedora Package Database. Reassigning to the new owner of this component.
*** Bug 1089743 has been marked as a duplicate of this bug. ***
Severity high, and last comment is from August 2013?
Found this tracking down the same issue affecting my F20 install. I can confirm the bug still exists in mlocate in F20 (should be update in the bug metadata), and that the PRUNE_BIND_MOUNT workaround (which is trivial, so maybe severity=high isn't correct either) fixes the issue.
Created attachment 962538 [details] Proposed patch
Also feel free to try the test build. https://msekleta.fedorapeople.org/mlocate-btrfs/
This workaround isn't: it changes the behaviour of updatedb so that a lot of files you don't want are in the index.
(In reply to James Patterson from comment #18) > This workaround isn't: it changes the behaviour of updatedb so that a lot of > files you don't want are in the index. Testing the proposed fix (of course with pruning enabled) would be very appreciated.
# head -1 /etc/updatedb.conf PRUNE_BIND_MOUNTS = "yes" # updatedb # locate Downloads (nothing found) # rpm -q mlocate mlocate-0.26-8.fc22.x86_64 It doesn't work :(
James thanks for the quick response. Can you please attach output of "cat /proc/self/mountinfo". Thanks.
I'd rather not - it's pretty big, what do you need in particular?
I'd like to have a look at your Downloads mount as well other related mounts, i.e. other subvolumes on the same btrfs filesystem.
Downloads was just an example of a normal directory in my homedir. # egrep "nfs|btrfs" /proc/self/mounts /dev/dm-0 / btrfs rw,seclabel,relatime,ssd,space_cache 0 0 /dev/dm-0 /home btrfs rw,seclabel,relatime,ssd,space_cache 0 0 nfs.homenet:/music /home/user/Music nfs rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,soft,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.1.1,mountvers=3,mountport=44916,mountproto=udp,local_lock=none,addr=192.168.1.1 0 0
James, can you please call stat on some dummy file in /home and on some file in / and paste the output here?
# touch /a ; sleep 1; touch /home/user/b # stat /a /home/user/b File: ‘/a’ Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 24h/36d Inode: 1370522 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Context: unconfined_u:object_r:etc_runtime_t:s0 Access: 2015-01-07 17:21:42.131854434 +0100 Modify: 2015-01-07 17:21:42.131854434 +0100 Change: 2015-01-07 17:21:42.131854434 +0100 Birth: - File: ‘/home/user/b’ Size: 0 Blocks: 0 IO Block: 4096 regular empty file Device: 29h/41d Inode: 2370307 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Context: unconfined_u:object_r:user_home_t:s0 Access: 2015-01-07 17:21:46.412860041 +0100 Modify: 2015-01-07 17:21:43.132855746 +0100 Change: 2015-01-07 17:21:43.132855746 +0100 Birth: -
This message is a notice that Fedora 19 is now at end of life. Fedora has stopped maintaining and issuing updates for Fedora 19. It is Fedora's policy to close all bug reports from releases that are no longer maintained. Approximately 4 (four) weeks from now this bug will be closed as EOL if it remains open with a Fedora 'version' of '19'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 19 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Can someone change to F21?
*** Bug 1196769 has been marked as a duplicate of this bug. ***
I am hands down out of ideas how to go about fixing this. It seems to me there is no sensible way how to distinguish between btrfs bind mount and btrfs subvolume after all. Any ideas, anyone?
Internally, mount option subvol= and subvolid= use bind mounts apparently. So I don't think they are distinguishable? Somewhat related is the issue of subvol vs subvolid mount options behaving differently, which is being worked on. https://lkml.org/lkml/2015/4/8/16
This message is a reminder that Fedora 21 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 21. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '21'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 21 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
->F23?
(In reply to James Patterson from comment #33) > ->F23? not sure what do you mean by the question mark ... did you test and it is still valid for F23? please, could you post some easy steps to reproduce? - I don't have the system that exhibited this any more, and the info in comments is a bit obsolete, for example "/proc/self/mounts" doesn't exist on F23 any more
I see this issue is still present on F23. btrfs is not in PRUNEFS. I don't know how to reproduce this issue. I have a /home on a btrfs partition and `locate [username]` et al doesn't find anything. Is it possible that this is permissions-related? My user is running at umask 077. Is it possible that this is related to disk encryption?
(In reply to Karel Volný from comment #34) > please, could you post some easy steps to reproduce? Still present on Fedora 23. The steps to reproduce is exactly the same as reported originally: : Steps to Reproduce: : 1. Make a btrfs partition and mount it as /home. : 2. Create a home directory. : 3. Place some files in it. : 4. Run updatedb's cron script. : 5. locate filename : : Actual results: : filename not found : : Expected results: : /home/user/filename (copy&pasted from #1)
FWIW, encountered on F23 where two btrfs subvols are mounted on /dev/sda3. PRUNE_BIND_MOUNTS="no" updates both / and /home.
(In reply to Takehiko Abe from comment #36) > Still present on Fedora 23. The steps to reproduce is exactly the same as > reported originally: I must be doing something wrong then ... anyways, more people confirmed (thanks) so updating the version
(In reply to Karel Volný from comment #38) > (In reply to Takehiko Abe from comment #36) > > Still present on Fedora 23. The steps to reproduce is exactly the same as > > reported originally: > > I must be doing something wrong then ... > > anyways, more people confirmed (thanks) so updating the version The original reproduce steps are wrong. I made a mistake while reporting them. To reproduce it you need to do what Fedora's installer does with BTRFS and create a BTRFS filesystem with two subvolumes. The installer creates subvolumes named "root" and "home" and "var" then the /etc/fstab looks like: UUID=44f6f562-1192-497a-a100-d37ec1435e2a / btrfs subvol=root,noatime,nodiratime,autodefrag 0 0 UUID=44f6f562-1192-497a-a100-d37ec1435e2a /home btrfs subvol=home,noatime,nodiratime,autodefrag 0 0 UUID=44f6f562-1192-497a-a100-d37ec1435e2a /var btrfs subvol=var,noatime,nodiratime,autodefrag 0 0 And then updatedb gets confused because these mounts are all on the same filesystem it thinks they are bind mounts even though the home and var subvolumes are not visible from root.
this is similar to my configuration, UUID=7a76222b-72a5-4c48-8a8e-047c8809bd11 / btrfs subvol=root 0 0 UUID=7a76222b-72a5-4c48-8a8e-047c8809bd11 /home btrfs subvol=home 0 0 with this setup, and PRUNE_BIND_MOUNTS="yes", updatedb only processes the root partition so I get no locate indexing for /home.
I'm experiencing the same thing on F23. updatedb sees my /root btrfs mount but not /home.
Fedora/Anaconda installations create subvolumes for mountpoints including root. So the top level of a Btrfs file system is never mounted, it's always effectively a bind mount, including root which in fstab is subvol=root mounted at /, just like subvol=home is mounted at /home. Nevertheless, locate finds things like "anaconda-ks.cfg" in root's home. But anything in /home seems to be excluded, so I'm not totally convinced it's a bind mount problem, unless the first bind mount is searched and each additional one is excluded. Anyway, same problem on Fedora 23 + kernel 4.5.0-0.rc1.git0.1.fc24.x86_64.
Please could we have a working locate command?
This message is a reminder that Fedora 23 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 23. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '23'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 23 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
This doesn't seem to be resolved, can someone move this to F24
I can reproduce in testing Fedora 25 instance using these steps: dnf install -y mlocate btrfs-progs modprobe scsi_debug dev_size_mb=40 mkfs.btrfs /dev/sda mkdir bt mount /dev/sda bt btrfs subvolume create bt/root btrfs subvolume create bt/home umount bt mount -o subvol=root /dev/sda bt mkdir bt/home mount -o subvol=home /dev/sda bt/home touch bt/rootfile touch bt/home/homefile updatedb locate rootfile locate homefile
I can reproduce this mlocate bug on Fedora 26 with mlocate-0.26-16.fc26.src.rpm
Portion of upstream Btrfs thread: https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg69617.html
This is also quite good explanation: https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg69620.html
I can reproduce this mlocate bug on Fedora 27 with mlocate-0.26-18.fc27.src.rpm Is there anything I can do to help with this bug?
This message is a reminder that Fedora 25 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 25. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '25'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 25 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
This is still a problem on Fedora26 (and I believe in Fedora27).
Fedora 25 changed to end-of-life (EOL) status on 2017-12-12. Fedora 25 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.
As comment @50 says, this is still a problem in F27.
just wanted to confirm it's in F27, mlocate-0.26-18.fc27.src.rpm does anyone know if it's going to be fixed anytime soon? I don't see it in the changelog for 0.26-20: https://koji.fedoraproject.org/koji/buildinfo?buildID=1038213 Is there anything that can be done to help?
(In reply to Michal Wasilewski from comment #55) > Is there anything that can be done to help? Status is unchanged from comments 48 and 49. It's not a problem with Btrfs, although it exposes the problem easier. Another possibility I haven't tested is with ostree based deployments, someone might want to test it out with Fedora Atomic Workstation making sure to explitly create a /home mount point - off hand I don't think that's in the default layout, but ostree deployments make prolifict use of bind mounts. Anyway, the problem might need to be asked on fs-devel.
I can confirm this bug on Fedora 28 with mlocate-0.26-20.fc28.src.rpm
This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle. Changing version to '29'.
This message is a reminder that Fedora 29 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a Fedora 'version' of '29'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version. Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora 29 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior this bug is closed as described in the policy above. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
Fedora 29 changed to end-of-life (EOL) status on 2019-11-26. Fedora 29 is no longer maintained, which means that it will not receive any further security or bug fix updates. As a result we are closing this bug. If you can reproduce this bug against a currently maintained version of Fedora please feel free to reopen this bug against that version. If you are unable to reopen this bug, please file a new report against the current release. If you experience problems, please add a comment to this bug. Thank you for reporting this bug and we are sorry it could not be fixed.
Yes, ostree is definitely affected: on Silverblue 31 I only see /etc and /boot files in my db.