Bug 906591 - updatedb does not index /home when /home is a bind mount
Summary: updatedb does not index /home when /home is a bind mount
Keywords:
Status: CLOSED RAWHIDE
Alias: None
Product: Fedora
Classification: Fedora
Component: mlocate
Version: 34
Hardware: Unspecified
OS: Linux
medium
high
Target Milestone: ---
Assignee: Michal Sekletar
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
: 1089743 1196769 (view as bug list)
Depends On:
Blocks: 1409406
TreeView+ depends on / blocked
 
Reported: 2013-01-31 23:42 UTC by Jonathan Briggs
Modified: 2022-01-05 15:00 UTC (History)
44 users (show)

Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Clone Of:
: 1409406 (view as bug list)
Environment:
Last Closed: 2022-01-05 15:00:03 UTC
Type: Bug
Embargoed:


Attachments (Terms of Use)
This is with PRUNE_BIND_MOUNTS = "no" in /etc/updatedb.conf (442.18 KB, text/x-log)
2013-02-07 18:22 UTC, Jonathan Briggs
no flags Details
This is with PRUNE_BIND_MOUNTS = "yes" in /etc/updatedb.conf (6.53 KB, text/x-log)
2013-02-07 18:23 UTC, Jonathan Briggs
no flags Details
Output of findmnt (root and home @ btrfs) (2.70 KB, text/plain)
2013-02-08 11:36 UTC, Petr Svoboda
no flags Details
Proposed patch (2.93 KB, patch)
2014-11-28 16:16 UTC, Michal Sekletar
no flags Details | Diff

Description Jonathan Briggs 2013-01-31 23:42:41 UTC
Description of problem:
Files that I know exist in my home directory were not displayed by the locate command.

The problem was traced to updatedb not indexing /home, because it believes /home is a bind mount for some reason. Disabling the PRUNE_BIND_MOUNT option in /etc/updatedb.conf fixed the problem.

However I believe the real bug is related to btrfs and bind mount detection.


Version-Release number of selected component (if applicable):
mlocate 0.26 for Fedora 18


How reproducible:
Every time.


Steps to Reproduce:
1. Make a btrfs partition and mount it as /home.
2. Create a home directory.
3. Place some files in it.
4. Run updatedb's cron script.
5. locate filename
  
Actual results:
filename not found

Expected results:
/home/user/filename


Additional info:

Comment 1 Miloslav Trmač 2013-02-01 20:19:13 UTC
Thanks for your report.  Could you collect a debug log, please?

1. Edit the last line of /etc/cron.daily/mlocate.cron to read
   /usr/bin/updatedb --debug-pruning -f "$nodevs"
2. run
   bash /etc/cron.daily/mlocate.cron |& tee log
3. Restore /etc/cron.daily/mlocate.cron to the original (remove --debug-pruning)
4. Attach the generated "log" file to this bug.

Comment 2 Jonathan Briggs 2013-02-07 18:22:50 UTC
Created attachment 694697 [details]
This is with PRUNE_BIND_MOUNTS = "no" in /etc/updatedb.conf

Comment 3 Jonathan Briggs 2013-02-07 18:23:30 UTC
Created attachment 694698 [details]
This is with PRUNE_BIND_MOUNTS = "yes" in /etc/updatedb.conf

Comment 4 Václav Pavlín 2013-02-08 08:20:55 UTC
Hi Jonathan,

Could you please provide output of findmnt?

Comment 5 Petr Svoboda 2013-02-08 11:36:49 UTC
Created attachment 695023 [details]
Output of findmnt (root and home @ btrfs)

I'm experiencing same/similar problem. I have both / and /home as different volumes on one btrfs filesystem and /home is pruned as bind mount. Attaching output of my findmnt.

Comment 6 Václav Pavlín 2013-02-08 12:42:16 UTC
Hi Petr,

Thanks for the info. It looks like mlocate cannot differentiate btrfs subvolume from bind mount.

What I noticed in /proc/self/mountinfo, while reproducing this issue, is that the only "thing" that could possibly show us the two volumes are not bind mounts, is the peer group number - bind mounts have the same peer group number, btrfs subvolumes have different numbers.

Miroslav, would it be safe to use this to check whether we have bind mount or the subvolume?

Comment 7 Miloslav Trmač 2013-02-08 23:45:15 UTC
(In reply to comment #6)
> Thanks for the info. It looks like mlocate cannot differentiate btrfs
> subvolume from bind mount.
Yes, both are the same file system on the same device.

> What I noticed in /proc/self/mountinfo, while reproducing this issue, is
> that the only "thing" that could possibly show us the two volumes are not
> bind mounts, is the peer group number - bind mounts have the same peer group
> number, btrfs subvolumes have different numbers.
> 
> Miroslav, would it be safe to use this to check whether we have bind mount
> or the subvolume?

I don't know, aren't peer groups related to mount namespaces?  That sounds completely unrelated to bind mounts.

Karel, could you help us answer the above question, or suggest an alternative way to distinguish btrfs subvolumes vs. bind mounts of the same subvolume?

Comment 8 Karel Zak 2013-02-20 13:52:05 UTC
(In reply to comment #7)
> I don't know, aren't peer groups related to mount namespaces?  That sounds
> completely unrelated to bind mounts.

Do you mean "shared:X" in or the mount ID (the first column)?

The dominant peer group (e.g. shared:X) is really about propagation flags, try

   mount --make-private /mountpoint

and the information disappear.

> Karel, could you help us answer the above question, or suggest an
> alternative way to distinguish btrfs subvolumes vs. bind mounts of the same
> subvolume?

Well, the subvolume could be also accessible from btrfs filesystem root (if mounted), you can also mount the same filesystem more than once, etc.

IMHO it would be better to not care about subvolumes and bind mounts in applications like mlocate.

All you need is to detect mountpoints (just compare directory devno and parental directory devno) and check if the mountpoint devno as returned by stat(2) is *unique*. That's all.

The same subvolume has the same devno everywhere.

 mount /dev/sdb1 /mnt/test
 btrfs subvolume create /mnt/test/sub

 mount /dev/sdb1 /mnt/test2 -o subvol=sub
 mount --bind /mnt/test/sub /mnt/test3
 mount --bind /mnt/test2 /mnt/test4

now the same subvolume is accessible by 4-ways:

 1: from filesystem root /mnt/test/sub
 2: from subvolume mountpoint /mnt/test2
 3: from bind to subvolume in fs root
 4: from bind to subvolume mountpoint

and all the places share the same devno (stat(2) st.st_dev) and the devno is different to btrfs filesystem root (/mnt/test in my example). So it's pretty simple to detect duplicate directories.

I don't see reason to use /proc/self/mountinfo at all.

Just for the record, see mountinfo:

$ grep mnt/test /proc/self/mountinfo
116 32 0:35 / /mnt/test rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache
119 32 0:35 /sub /mnt/test2 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache
122 32 0:35 /sub /mnt/test3 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache
125 32 0:35 /sub /mnt/test4 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache

there is no information about bind mounts or subvolumes. All you can found is subdirectory (fs-root, 4th column) used for the target mountpoint.

Comment 9 Miloslav Trmač 2013-02-20 14:21:59 UTC
(In reply to comment #8)
> > Karel, could you help us answer the above question, or suggest an
> > alternative way to distinguish btrfs subvolumes vs. bind mounts of the same
> > subvolume?
> 
> Well, the subvolume could be also accessible from btrfs filesystem root (if
> mounted), you can also mount the same filesystem more than once, etc.
> 
> IMHO it would be better to not care about subvolumes and bind mounts in
> applications like mlocate.

mlocate cares so that it doesn't present a single path N times in presence of bind mounts, which users find redundant.


>  mount /dev/sdb1 /mnt/test
>  btrfs subvolume create /mnt/test/sub
> 
>  mount /dev/sdb1 /mnt/test2 -o subvol=sub
>  mount --bind /mnt/test/sub /mnt/test3
>  mount --bind /mnt/test2 /mnt/test4
> 
> now the same subvolume is accessible by 4-ways:
> 
>  1: from filesystem root /mnt/test/sub
>  2: from subvolume mountpoint /mnt/test2
>  3: from bind to subvolume in fs root
>  4: from bind to subvolume mountpoint
> 
> and all the places share the same devno (stat(2) st.st_dev) and the devno is
> different to btrfs filesystem root (/mnt/test in my example).

In the dump below, the device seems to be 0:35 both for the root volume at /mnt/test and the subvolumes; is it really supposed to be different?

> $ grep mnt/test /proc/self/mountinfo
> 116 32 0:35 / /mnt/test rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache
> 119 32 0:35 /sub /mnt/test2 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache
> 122 32 0:35 /sub /mnt/test3 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache
> 125 32 0:35 /sub /mnt/test4 rw,relatime - btrfs /dev/sdb1 rw,ssd,space_cache


Anyway, looking closer, we might not actually need a detection of subvolumes.  An easy way to detect btrfs subvolumes would definitely allow and easy and quick fix of this case, but a general fix for bug 723279 (if and when that happens) would probably fix this case as well.

Comment 10 Karel Zak 2013-02-20 14:55:47 UTC
(In reply to comment #9)
> In the dump below, the device seems to be 0:35 both for the root volume at
> /mnt/test and the subvolumes; is it really supposed to be different?

Yes, btrfs uses the same devno in mountinfo for all stuff related to the filesystem. And the mountinfo devno is different to stat(2) st_dev :-)

Comment 11 Karel Volný 2013-08-27 14:22:57 UTC
I've just run into this too

# updatedb --debug-pruning 2>&1 | grep home
 `/home' (42 on 34) is `/home' of `/dev/mapper/luks-5c0b8da3-45ed-4562-b8ae-7939af586c6d' (0:29), type `btrfs'
 => adding `/home'
 `/home', type `btrfs'
 `/home', type `btrfs'
Skipping `/home': bind mount


the bad thing about this is that it is the installer default to create /home as subvolume

Comment 12 Fedora Admin XMLRPC Client 2014-01-17 15:54:56 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 13 Miloslav Trmač 2014-04-22 19:22:36 UTC
*** Bug 1089743 has been marked as a duplicate of this bug. ***

Comment 14 Need Real Name 2014-04-23 18:47:58 UTC
Severity high, and last comment is from August 2013?

Comment 15 Andy Ross 2014-08-02 04:56:57 UTC
Found this tracking down the same issue affecting my F20 install.

I can confirm the bug still exists in mlocate in F20 (should be update in the bug metadata), and that the PRUNE_BIND_MOUNT workaround (which is trivial, so maybe severity=high isn't correct either) fixes the issue.

Comment 16 Michal Sekletar 2014-11-28 16:16:31 UTC
Created attachment 962538 [details]
Proposed patch

Comment 17 Michal Sekletar 2014-11-28 16:19:36 UTC
Also feel free to try the test build.

https://msekleta.fedorapeople.org/mlocate-btrfs/

Comment 18 James Patterson 2014-12-05 21:31:14 UTC
This workaround isn't: it changes the behaviour of updatedb so that a lot of files you don't want are in the index.

Comment 19 Michal Sekletar 2015-01-07 09:03:53 UTC
(In reply to James Patterson from comment #18)

> This workaround isn't: it changes the behaviour of updatedb so that a lot of
> files you don't want are in the index.

Testing the proposed fix (of course with pruning enabled) would be very appreciated.

Comment 20 James Patterson 2015-01-07 09:20:46 UTC
# head -1 /etc/updatedb.conf
PRUNE_BIND_MOUNTS = "yes"

# updatedb

# locate Downloads
(nothing found)

# rpm -q mlocate
mlocate-0.26-8.fc22.x86_64

It doesn't work :(

Comment 21 Michal Sekletar 2015-01-07 09:55:00 UTC
James thanks for the quick response. Can you please attach output of "cat /proc/self/mountinfo". Thanks.

Comment 22 James Patterson 2015-01-07 10:16:41 UTC
I'd rather not - it's pretty big, what do you need in particular?

Comment 23 Michal Sekletar 2015-01-07 11:21:55 UTC
I'd like to have a look at your Downloads mount as well other related mounts, i.e. other subvolumes on the same btrfs filesystem.

Comment 24 James Patterson 2015-01-07 12:06:56 UTC
Downloads was just an example of a normal directory in my homedir.

# egrep "nfs|btrfs" /proc/self/mounts
/dev/dm-0 / btrfs rw,seclabel,relatime,ssd,space_cache 0 0
/dev/dm-0 /home btrfs rw,seclabel,relatime,ssd,space_cache 0 0
nfs.homenet:/music /home/user/Music nfs rw,relatime,vers=3,rsize=32768,wsize=32768,namlen=255,soft,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=192.168.1.1,mountvers=3,mountport=44916,mountproto=udp,local_lock=none,addr=192.168.1.1 0 0

Comment 25 Michal Sekletar 2015-01-07 14:54:06 UTC
James, can you please call stat on some dummy file in /home and on some file in / and paste the output here?

Comment 26 James Patterson 2015-01-07 16:22:36 UTC
# touch /a ; sleep 1; touch /home/user/b
# stat /a /home/user/b
  File: ‘/a’
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 24h/36d	Inode: 1370522     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: unconfined_u:object_r:etc_runtime_t:s0
Access: 2015-01-07 17:21:42.131854434 +0100
Modify: 2015-01-07 17:21:42.131854434 +0100
Change: 2015-01-07 17:21:42.131854434 +0100
 Birth: -
  File: ‘/home/user/b’
  Size: 0         	Blocks: 0          IO Block: 4096   regular empty file
Device: 29h/41d	Inode: 2370307     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: unconfined_u:object_r:user_home_t:s0
Access: 2015-01-07 17:21:46.412860041 +0100
Modify: 2015-01-07 17:21:43.132855746 +0100
Change: 2015-01-07 17:21:43.132855746 +0100
 Birth: -

Comment 27 Fedora End Of Life 2015-01-09 22:03:39 UTC
This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 28 James Patterson 2015-01-10 06:45:47 UTC
Can someone change to F21?

Comment 29 Michal Sekletar 2015-03-27 11:48:01 UTC
*** Bug 1196769 has been marked as a duplicate of this bug. ***

Comment 30 Michal Sekletar 2015-03-27 12:26:11 UTC
I am hands down out of ideas how to go about fixing this. It seems to me there is no sensible way how to distinguish between btrfs bind mount and btrfs subvolume after all. Any ideas, anyone?

Comment 31 Chris Murphy 2015-04-09 19:56:40 UTC
Internally, mount option subvol= and subvolid= use bind mounts apparently. So I don't think they are distinguishable?

Somewhat related is the issue of subvol vs subvolid mount options behaving differently, which is being worked on.
https://lkml.org/lkml/2015/4/8/16

Comment 32 Fedora End Of Life 2015-11-04 15:36:53 UTC
This message is a reminder that Fedora 21 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 21. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '21'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 21 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 33 James Patterson 2015-11-09 18:59:35 UTC
->F23?

Comment 34 Karel Volný 2015-11-10 11:20:05 UTC
(In reply to James Patterson from comment #33)
> ->F23?

not sure what do you mean by the question mark ... did you test and it is still valid for F23?

please, could you post some easy steps to reproduce?

- I don't have the system that exhibited this any more, and the info in comments is a bit obsolete, for example "/proc/self/mounts" doesn't exist on F23 any more

Comment 35 Christian Stadelmann 2015-11-13 12:56:32 UTC
I see this issue is still present on F23.
btrfs is not in PRUNEFS. 

I don't know how to reproduce this issue. I have a /home on a btrfs partition and `locate [username]` et al doesn't find anything.
Is it possible that this is permissions-related? My user is running at umask 077.
Is it possible that this is related to disk encryption?

Comment 36 Takehiko Abe 2015-11-14 14:49:06 UTC
(In reply to Karel Volný from comment #34)
> please, could you post some easy steps to reproduce?

Still present on Fedora 23. The steps to reproduce is exactly the same as reported originally:

: Steps to Reproduce:
: 1. Make a btrfs partition and mount it as /home.
: 2. Create a home directory.
: 3. Place some files in it.
: 4. Run updatedb's cron script.
: 5. locate filename
: 
: Actual results:
: filename not found
: 
: Expected results:
: /home/user/filename

(copy&pasted from #1)

Comment 37 Steeve McCauley 2015-11-25 10:37:19 UTC
FWIW, encountered on F23 where two btrfs subvols are mounted on /dev/sda3.  PRUNE_BIND_MOUNTS="no" updates both / and /home.

Comment 38 Karel Volný 2015-11-25 12:50:48 UTC
(In reply to Takehiko Abe from comment #36)
> Still present on Fedora 23. The steps to reproduce is exactly the same as
> reported originally:

I must be doing something wrong then ...

anyways, more people confirmed (thanks) so updating the version

Comment 39 Jonathan Briggs 2015-11-25 18:53:26 UTC
(In reply to Karel Volný from comment #38)
> (In reply to Takehiko Abe from comment #36)
> > Still present on Fedora 23. The steps to reproduce is exactly the same as
> > reported originally:
> 
> I must be doing something wrong then ...
> 
> anyways, more people confirmed (thanks) so updating the version

The original reproduce steps are wrong. I made a mistake while reporting them.

To reproduce it you need to do what Fedora's installer does with BTRFS and create a BTRFS filesystem with two subvolumes.

The installer creates subvolumes named "root" and "home" and "var" then the /etc/fstab looks like:

UUID=44f6f562-1192-497a-a100-d37ec1435e2a /                       btrfs   subvol=root,noatime,nodiratime,autodefrag 0 0
UUID=44f6f562-1192-497a-a100-d37ec1435e2a /home                   btrfs   subvol=home,noatime,nodiratime,autodefrag 0 0
UUID=44f6f562-1192-497a-a100-d37ec1435e2a /var                    btrfs   subvol=var,noatime,nodiratime,autodefrag  0 0

And then updatedb gets confused because these mounts are all on the same filesystem it thinks they are bind mounts even though the home and var subvolumes are not visible from root.

Comment 40 Steeve McCauley 2015-11-25 19:09:13 UTC
this is similar to my configuration,

UUID=7a76222b-72a5-4c48-8a8e-047c8809bd11 /                       btrfs   subvol=root     0 0
UUID=7a76222b-72a5-4c48-8a8e-047c8809bd11 /home                   btrfs   subvol=home     0 0

with this setup, and PRUNE_BIND_MOUNTS="yes", updatedb only processes the root partition so I get no locate indexing for /home.

Comment 41 Devin Henderson 2016-01-28 04:06:27 UTC
I'm experiencing the same thing on F23. updatedb sees my /root btrfs mount but not /home.

Comment 42 Chris Murphy 2016-01-28 04:27:54 UTC
Fedora/Anaconda installations create subvolumes for mountpoints including root. So the top level of a Btrfs file system is never mounted, it's always effectively a bind mount, including root which in fstab is subvol=root mounted at /, just like subvol=home is mounted at /home.

Nevertheless, locate finds things like "anaconda-ks.cfg" in root's home. But anything in /home seems to be excluded, so I'm not totally convinced it's a bind mount problem, unless the first bind mount is searched and each additional one is excluded.

Anyway, same problem on Fedora 23 + kernel 4.5.0-0.rc1.git0.1.fc24.x86_64.

Comment 43 James Patterson 2016-05-20 20:57:50 UTC
Please could we have a working locate command?

Comment 44 Fedora End Of Life 2016-11-24 10:55:00 UTC
This message is a reminder that Fedora 23 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 23. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '23'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 23 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 45 Steeve McCauley 2016-11-24 12:02:21 UTC
This doesn't seem to be resolved, can someone move this to F24

Comment 46 Karel Volný 2017-01-01 20:27:36 UTC
I can reproduce in testing Fedora 25 instance using these steps:

dnf install -y mlocate btrfs-progs
modprobe scsi_debug dev_size_mb=40
mkfs.btrfs /dev/sda
mkdir bt
mount /dev/sda bt
btrfs subvolume create bt/root
btrfs subvolume create bt/home
umount bt
mount -o subvol=root /dev/sda bt
mkdir bt/home
mount -o subvol=home /dev/sda bt/home
touch bt/rootfile
touch bt/home/homefile
updatedb
locate rootfile
locate homefile

Comment 47 Michal Halenka 2017-11-02 11:55:15 UTC
I can reproduce this mlocate bug on Fedora 26 with mlocate-0.26-16.fc26.src.rpm

Comment 48 Chris Murphy 2017-11-05 12:04:37 UTC
Portion of upstream Btrfs thread:
https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg69617.html

Comment 49 Chris Murphy 2017-11-06 18:48:59 UTC
This is also quite good explanation:
https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg69620.html

Comment 50 Michal Halenka 2017-11-15 11:05:26 UTC
I can reproduce this mlocate bug on Fedora 27 with mlocate-0.26-18.fc27.src.rpm

Is there anything I can do to help with this bug?

Comment 51 Fedora End Of Life 2017-11-16 19:27:23 UTC
This message is a reminder that Fedora 25 is nearing its end of life.
Approximately 4 (four) weeks from now Fedora will stop maintaining
and issuing updates for Fedora 25. It is Fedora's policy to close all
bug reports from releases that are no longer maintained. At that time
this bug will be closed as EOL if it remains open with a Fedora  'version'
of '25'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version'
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not
able to fix it before Fedora 25 is end of life. If you would still like
to see this bug fixed and are able to reproduce it against a later version
of Fedora, you are encouraged  change the 'version' to a later Fedora
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's
lifetime, sometimes those efforts are overtaken by events. Often a
more recent Fedora release includes newer upstream software that fixes
bugs or makes them obsolete.

Comment 52 john.haxby@oracle.com 2017-11-16 19:40:38 UTC
This is still a problem on Fedora26 (and I believe in Fedora27).

Comment 53 Fedora End Of Life 2017-12-12 10:27:05 UTC
Fedora 25 changed to end-of-life (EOL) status on 2017-12-12. Fedora 25 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 54 Jonathan Briggs 2017-12-12 17:17:36 UTC
As comment @50 says, this is still a problem in F27.

Comment 55 Michal Wasilewski 2018-03-24 13:44:59 UTC
just wanted to confirm it's in F27, mlocate-0.26-18.fc27.src.rpm

does anyone know if it's going to be fixed anytime soon? I don't see it in the changelog for 0.26-20:  https://koji.fedoraproject.org/koji/buildinfo?buildID=1038213

Is there anything that can be done to help?

Comment 56 Chris Murphy 2018-03-25 02:00:30 UTC
(In reply to Michal Wasilewski from comment #55)
> Is there anything that can be done to help?

Status is unchanged from comments 48 and 49. It's not a problem with Btrfs, although it exposes the problem easier. Another possibility I haven't tested is with ostree based deployments, someone might want to test it out with Fedora Atomic Workstation making sure to explitly create a /home mount point - off hand I don't think that's in the default layout, but ostree deployments make prolifict use of bind mounts. Anyway, the problem might need to be asked on fs-devel.

Comment 57 Michal Halenka 2018-07-31 10:34:55 UTC
I can confirm this bug on Fedora 28 with mlocate-0.26-20.fc28.src.rpm

Comment 58 Jan Kurik 2018-08-14 10:32:01 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 29 development cycle.
Changing version to '29'.

Comment 59 Ben Cotton 2019-10-31 20:50:42 UTC
This message is a reminder that Fedora 29 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 29 on 2019-11-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '29'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 29 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 60 Ben Cotton 2019-11-27 19:06:34 UTC
Fedora 29 changed to end-of-life (EOL) status on 2019-11-26. Fedora 29 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.

Comment 61 Jens Petersen 2020-03-29 13:28:17 UTC
Yes, ostree is definitely affected: on Silverblue 31 I only see /etc and /boot files in my db.

Comment 62 Juanje Ojeda 2020-08-25 11:53:02 UTC
I confirm that 'Fedora Silverblue 32' has the same problem.
The command 'locate' doesn't show anything, but if I comment out the line `PRUNE_BIND_MOUNTS = "yes"`, then it works as expected.

OS version: Fedora 32.20200816.0 (Silverblue)
Package 'mlocate' version: mlocate-0.26-25.fc32.x86_64

Comment 63 Chris Murphy 2020-09-29 17:45:47 UTC
Can anyone thing of a negative consequence of changing PRUNE_BIND_MOUNTS = "yes" in /etc/updatedb.conf, to "no" by default? That would fix this problem but I don't know if there are any negative consequences of doing so.

Another way to unambiguously detect subvolumes is they always have an inode number of 256.

Comment 64 Garry T. Williams 2020-09-30 05:02:01 UTC
Bug #1883734 may be related and has a different work-around.

Comment 65 Juanje Ojeda 2020-10-27 16:59:15 UTC
Here are some discussions about this bug: https://github.com/fedora-silverblue/issue-tracker/issues/76

Comment 66 Garry T. Williams 2020-10-27 22:27:01 UTC
(In reply to Juanje Ojeda from comment #65)
> Here are some discussions about this bug:
> https://github.com/fedora-silverblue/issue-tracker/issues/76

Yes, but I do not think that completely explains what is going on here.  I have this bug.  I have *no* bind mounts on a simple Fedora workstation configured with btrfs.  Now I know that btrfs mounts its subvolumes using something like bind mounts, but it is not obvious.

Furthermore, F33 workstation defaults to this configuration.  So I suspect that *all* new workstation installs will have this bug.

garry@gtw$ df -h
Filesystem      Size  Used Avail Use% Mounted on
devtmpfs         16G     0   16G   0% /dev
tmpfs            16G  197M   16G   2% /dev/shm
tmpfs           6.3G  2.2M  6.3G   1% /run
/dev/nvme0n1p2  477G   57G  419G  12% /
tmpfs            16G   24M   16G   1% /tmp
/dev/nvme0n1p2  477G   57G  419G  12% /var/log
/dev/nvme0n1p2  477G   57G  419G  12% /home
/dev/nvme0n1p1  150M   21M  130M  14% /boot/efi
tmpfs           3.2G  928K  3.2G   1% /run/user/1000
garry@gtw$ mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,noexec,seclabel,size=16254356k,nr_inodes=4063589,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,size=6511640k,nr_inodes=819200,mode=755)
cgroup2 on /sys/fs/cgroup type cgroup2 (rw,nosuid,nodev,noexec,relatime,seclabel,nsdelegate)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)
none on /sys/fs/bpf type bpf (rw,nosuid,nodev,noexec,relatime,mode=700)
configfs on /sys/kernel/config type configfs (rw,nosuid,nodev,noexec,relatime)
/dev/nvme0n1p2 on / type btrfs (rw,noatime,seclabel,ssd,space_cache,subvolid=258,subvol=/root)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,nosuid,noexec,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=15163)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel,pagesize=2M)
mqueue on /dev/mqueue type mqueue (rw,nosuid,nodev,noexec,relatime,seclabel)
debugfs on /sys/kernel/debug type debugfs (rw,nosuid,nodev,noexec,relatime,seclabel)
tracefs on /sys/kernel/tracing type tracefs (rw,nosuid,nodev,noexec,relatime,seclabel)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,seclabel,size=16279096k,nr_inodes=409600)
/dev/nvme0n1p2 on /var/log type btrfs (rw,noatime,seclabel,ssd,space_cache,subvolid=256,subvol=/var_log)
/dev/nvme0n1p2 on /home type btrfs (rw,noatime,seclabel,ssd,space_cache,subvolid=259,subvol=/home)
/dev/nvme0n1p1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro)
sunrpc on /var/lib/nfs/rpc_pipefs type rpc_pipefs (rw,relatime)
tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=3255816k,nr_inodes=813954,mode=700,uid=1000,gid=1000)
garry@gtw$ sudo btrfs fi show
Label: 'fedora_localhost-live'  uuid: 625e5794-4d04-4e1e-a15e-df9b3661e41d
        Total devices 2 FS bytes used 56.86GiB
        devid    1 size 476.79GiB used 80.01GiB path /dev/nvme0n1p2
        devid    2 size 476.94GiB used 80.01GiB path /dev/nvme1n1p1

garry@gtw$ btrfs fi df /    
Data, RAID1: total=78.00GiB, used=55.84GiB
System, RAID1: total=8.00MiB, used=16.00KiB
Metadata, RAID1: total=2.00GiB, used=1.02GiB
GlobalReserve, single: total=119.75MiB, used=0.00B
garry@gtw$

Comment 67 Chris Murphy 2020-10-29 18:46:21 UTC
Fix for Silverblue on the way, but it is Silverblue specific. I'm not sure this will fix it on Btrfs, including Silverblue on Btrfs.
https://github.com/fedora-silverblue/issue-tracker/issues/76#issuecomment-718499153

The assumption that bind mounts are duplicates is obviously not correct. That assumption needs to be set aside, and figure out a different way of eliminating duplicates. A non-trivial number of users will also have a "var" subvolume mounted at /var.

For what it's worth:

The effect of 'mount -o subvol/subvolid' on Btrfs is pretty much a bind mount. A true bind mount needs to be able to see the directory being bind mounted, whereas on Btrfs it can do the path resolution without it being immediately available. i.e. if the file system is not mounted at all, I can still 'mount -o subvol=home /dev/X /home'. This is the "flat" layout style. Whereas the "nested" style doesn't show up as a mountpoint, it superficially manifests as a directory but has its own fd and inside is a separate namespace where the inode numbering starts over.

'stat /' and 'stat /home' show they are not the same device. Btrfs subvolumes are always inode 256. And they have unique fd, so they show up as different devices.

Device: 24h/36d	Inode: 256         Links: 1
Device: 29h/41d	Inode: 256         Links: 1

'stat -f' they show up with different ID's

ID: 8fda9dd9590ce4b0 Namelen: 255     Type: btrfs
ID: 8fda9dd9590ce41f Namelen: 255     Type: btrfs

And from statfs() directly, they look related but not identical.
f_fsid={val=[2413469145, 1494017200]}
f_fsid={val=[2413469145, 1494017055]}

Why? Imagine /home has dozens or even hundreds of snapshots. You want rsync to have a way of avoiding them, so you aren't traversing hundreds of almost identical file trees, so you use the -x option:

       --one-file-system, -x    don't cross filesystem boundaries

And that's how rsync, borgbackup, and others, know how to avoid the problem of crossing into snapshots. The follow-up quandary is, well what if a user creates a subvolume inside their ~/ ? Should we include that location by default or not? What do they expect? Is there a way to differentiate between subvolume and subvolume snapshots, so that we can traverse into the former but not the latter? Yes, probably via 'btrfs subvolume list' or libbtrfsutil. But is that interface adequate? I'm not sure. But this comment is getting long, and maybe out of scope, so I'll stop here.

Comment 68 Chris Murphy 2020-10-29 18:47:34 UTC
Oops. Forgot to add this detail on the subvolume layout types.
https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Layout

Comment 69 Ben Cotton 2020-11-03 17:21:45 UTC
This message is a reminder that Fedora 31 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 31 on 2020-11-24.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '31'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 31 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

Comment 70 Yang Wentao 2020-11-12 13:22:16 UTC
As in comment 66:

> Furthermore, F33 workstation defaults to this configuration.  So I suspect that *all* new workstation installs will have this bug.

New fedora 33 installs still have this bug/ (for workstation and all desktop spins), and is very easy to confirm. `locate <filename>` simply show nothing beginning with `/home`.

Comment 71 Yang Wentao 2020-11-12 13:31:52 UTC
BTW, snapshots for /home directory (for example /.snapshots/home-$(date +%Y%m%d) does show up in `locate <filename>` search results.

Surprised that this bug persisted for 7 years with no signs of potential fixes. However, I guess this bug has not much UX implications as everybody using fedora desktops, especially workstation version, will search their files using desktop utility than `locate` command. As a relatively new user (using fedora for a couple of months) I didn't even notice this bug.

Comment 72 Chris Murphy 2020-12-15 03:00:16 UTC
Possible work around is to modify updatedb.conf as follows:

a) PRUNE_BIND_MOUNTS = "no" ;  man page says the default is no, but Fedora's /etc/updatedb.conf sets it to "yes" which is the central problem in this bug.
b) PRUNEPATHS ; man page says the default is no paths are skipped, but Fedora's /etc/updatedb.conf has quite a long list, but you can add more, e.g. /.snapshots /home/chris/.snapshots
c) sudo updatedb

(Ideally, I'd like to see the .directory convention for btrfs snapshots go away. It causes problems for find, and du results too. And it's a potential security risk. A better convention is for programs to create them in the hidden top-level of the file system, which is normally not mounted. libbtrfsutil provides an _fd variant for subvolume/snapshot creation/removal for this purpose; both C API and Python bindings are available.)

Comment 73 Ben Cotton 2021-02-09 16:23:36 UTC
This bug appears to have been reported against 'rawhide' during the Fedora 34 development cycle.
Changing version to 34.

Comment 74 Chris Murphy 2021-02-28 23:18:10 UTC
This is fixed on Silverblue by 
https://src.fedoraproject.org/rpms/mlocate/c/2fbdf7fbe91e3f37ed952724dae1bb0dbf6422ba?branch=rawhide

Which changes /etc/updatedb.conf

-#PRUNE_BIND_MOUNTS = "yes"
+PRUNE_BIND_MOUNTS = "yes"

That's the upstream default per the man updatedb.conf page: "By default, bind mounts are not skipped."

@msekleta, can we just stop deviating from the upstream default? Or do we really need a patch to extend a test for $VARIANT_ID across all desktop editions and spins?

Comment 75 Chris Murphy 2021-02-28 23:22:29 UTC
Sorry that should be:

-PRUNE_BIND_MOUNTS = "yes"
+#PRUNE_BIND_MOUNTS = "yes"


i.e. Silverblue comments this out, in effect making it "no", same as upstream.

Comment 76 Allen Winter 2021-05-11 17:17:20 UTC
I'm not sure what Silverblue is vs. rawhide.. but I'm using Fedora 33.
I commented out PRUNE_BIND_MOUNTS = "yes" then reran updatedb.  much happier now.

Comment 77 Chris Murphy 2021-12-01 19:19:08 UTC
This is expected to be fixed with this Fedora 36 change. Anyone can follow the how to test section and apply it to Fedora 34/35.
https://fedoraproject.org/wiki/Changes/Plocate_as_the_default_locate_implementation
https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/message/WYS5LANEOA2W7MYCXDQDJTOMFOUZCYYK/

Once the change is approved, this bug can be set to Closed->Rawhide

Comment 78 Nikita Bige 2021-12-24 16:33:00 UTC
Fedora 35
same issue with /home 
followed 
https://fedoraproject.org/wiki/Changes/Plocate_as_the_default_locate_implementation
It helps. 
I've switched to plocate

Comment 79 Ben Cotton 2022-01-05 15:00:03 UTC
Per #2030170, the switch to plocate is now in Rawhide. Closing this bug.


Note You need to log in before you can comment on or make changes to this bug.