Description of problem: In fstab have a root filesystem, anda separate data disk/filesystem, bind mounts, and bindfs mounts listed in that order, with all the bind mounts and bindfs mounts relocating directories from the data disk. This fstab worked fine in fedora 14. Now the bindfs mounts break startup. There are no systemd options and all fstab entries can be mounted with mount -a (works). No custom systemd .mount files were added. In fact it's a fresh upgrade from fedora 14 so nothing in systemd has been altered. When bootup fails, in the emergency terminal I see that the data disk is not mounted yet. Systemd documentation claims that mount fstab dependencies are calculated automatically. The bind mounts (as opposed to bindfs) work fine. I suspect that the generally different "device" field syntax of bindfs mounts, and especially fuse in general, breaks the dependency analysis. If I comment out these mounts the system starts fine and I can un-comment and mount them after booting. I'm sure I can engineer some dependency hack or .mount files to fix this for me. Version-Release number of selected component (if applicable): Fedora 16 fully updated about 2 days ago. How reproducible: always Steps to Reproduce: 1. Use fstab described above which worked in Fedora 14 2. Boot up. 3. Actual results: Boot fails when the bind-fs fails because the underlying file system is not mounted yet. Expected results: Boot succeeds with bindfs mounts delayed until after the pre-requisite storage mount. If systemd cannot determine the dependencies of an fstab entry it should fall back to the fstab standard and assume that anything listed above it should be mounted first. Additional info:
Would you attach said fstab?
It's a simple enough point but sure I'll make a simple test case for you.. here it is: /dev/dm-1 / ext4 defaults 0 1 /dev/sda1 /boot ext4 defaults 0 2 /dev/sdb1 /data ext3 defaults 0 2 /data /test0 ext3 bind 0 0 /data/test1 /test1 ext3 bind 0 0 bindfs#/data/test2 /test2 fuse rw 0 0 /dev/mapper/VolGroup-lv_swap swap swap defaults 0 0 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 I just booted this just now.. It boots fine with ONLY the bindfs line commented and it fails on /test with it un-commented. Either way.. after it's un-commented (manually or at at the emergency prompt), mount -a works just fine (which I believe is what init.d/localmount used). This test case is running in a toy vm so I can play with it easily. Really this is 2 bugs IMO: 1) mount order is wrong 2) failure behavior is wrong IMO, but I guess I don't remember how that worked before, because bug 1 didn't exist before but I thought mount failures didn't break bootup. You remove a non-root esata drive and boot fails? Sure there are about 4 or 5 ways around that probably.. but still, is that expected behavior?
Simple workaround: add nofail to the bindfs mount. It's hard to consider that hack a bug fix though, but for me now it's certainly a fine solution. It turns out that /test2 does actually get mounted on boot in that case, so it's a better fix than it might appear. It must be that systemd makes multiple passes at mounting things. Adding comment=system.d to the /data drive did not help. If only every bug could be avoided by typing "nofail". :)
quoting myself 2 posts up: "It boots fine with ONLY the bindfs line commented and it fails on /test with it un-commented. " ... that should of course say "... fails on /test2 ..."
(In reply to comment #2) > bindfs#/data/test2 /test2 fuse rw 0 0 It is true that systemd does not understand the bindfs syntax and won't infer the ordering dependency. Parsing the bindfs syntax would not be general enough. The only practicable fix that I can see is to add After dependencies to fuse mounts on all mounts that are listed above them in fstab.
That is exactly the response I expected. Of course.. if systemd can't understand an fstab entry it should honor the traditional meaning of fstab which is exactly the fix described. You can't try to outsmart the user when you don't understand the syntax and it's not very nice breaking what used to work just because you wish there was a better way. I get not wanting to attempt to support every major fusefs syntax too. Can we call it confirmed then?
We can't really automatically detect dependencies of these kinds, and simply adding deps between all mounts simply by the order they are listed in would effectively serialize the boot, which is certainly not desirable. I am thinking of simply adding x-systemd-after=/foobar/waldo as a new mount option people can use to explciitly tell systemd about dependencies. That would work pretty nicely I guess.
*** Bug 881481 has been marked as a duplicate of this bug. ***
Note: the suggestion that ended up happening in Bug 881481 was to have a general systemd dependency syntax, e.g. '_dep=foobar.mount'; IOW, taking systemd unit names rather than mount-points, which would allow the same system to handle this, supercede _netdev, and probably would be able to cover cases we haven't even thought of yet. (This plus _netdev would in theory suffice for Bug 881481, but I'd be in favor of the single general system covering both.)
Well, the "_" namespace is mount's own. Systemd should extend the fstab syntax only in its own "x-systemd-" namespace. But yeah, there are three options: x-systemd-after=<PATH> -- the explicit path to wait for x-systemd-after=<UNIT> -- the systemd unit name to wait for x-systemd-requires-mounts-for=<PATH> -- referring to all mount unit that are necessary for this file to be around, possibly multiple. THis is a superset of the first option The latter would work more or less like RequiresMountsFor= in unit files. I think the third syntax is probably the most useful one here, and should solve the usecases at hand. Adding the second syntax later on, should the need arise should be OK, too.
This message is a reminder that Fedora 16 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 16. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '16'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 16's end of life. Bug Reporter: Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 16 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged to click on "Clone This Bug" and open it against that version of Fedora. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete. The process we are following is described here: http://fedoraproject.org/wiki/BugZappers/HouseKeeping
Does it need a x- prefix? Take a look at https://tools.ietf.org/html/rfc6648 (Deprecating the "X-" Prefix and Similar Constructs in Application Protocols).
Lennart Poettering"We can't really automatically detect dependencies of these kinds, and simply adding deps between all mounts simply by the order they are listed in would effectively serialize the boot, which is certainly not desirable." With respect, this isn't true. Well the first part is: You cannot automatically detect dependencies of fuse mounts (or potentially something else like them). The second part though is not true. You do not need to serialize the whole boot. You do not need to make every mount depend on the mount before it (unless every mount is a fuse mount). You only need to make every fuse mount depend on all the mounts before it (in no particular order). That is VERY different from serializing the whole mount process. If you have mounts 1 through 5 that are normal and 6 is a fuse mount and 7 through 20 are normal. Then 1 through five can be parallelized as much as normal dependency rules allow, but they must all finish before 6 is performed. Then 7 through 20 carry on. That might not be ideal always but it's far from fully serial and it's the right thing to do if you don't know. If there are no special options specified, this is exactly what should happen because that is the standard defined by fstab and will cause the least breakage as more people(distributions) adopt systemd. Special options to allow the user to provide hints to improve on that situation from there are great, but should not be required to prevent breakage.
To keep this in perspective, most people probably aren't using bindfs mounts in a customized fstab. The small amount of serialization constraint I describe is a loss of a few seconds at most for a small but not tiny group of people. By implementing some ability to add hints, it's a fixable loss of a few seconds for a small group of people. However, requiring the hints to make it work is instead going to result in boot failure for probably a high fraction of that small but not tiny group. Some fraction of those won't ever figure out why before moving on. Some fraction just won't have time to figure out why this morning and will break their toe cursing about it and then limp through the day without their computer.
This message is a reminder that Fedora 18 is nearing its end of life. Approximately 4 (four) weeks from now Fedora will stop maintaining and issuing updates for Fedora 18. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as WONTFIX if it remains open with a Fedora 'version' of '18'. Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, simply change the 'version' to a later Fedora version prior to Fedora 18's end of life. Thank you for reporting this issue and we are sorry that we may not be able to fix it before Fedora 18 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora, you are encouraged change the 'version' to a later Fedora version prior to Fedora 18's end of life. Although we aim to fix as many bugs as possible during every release's lifetime, sometimes those efforts are overtaken by events. Often a more recent Fedora release includes newer upstream software that fixes bugs or makes them obsolete.
I'm trying to use a fuse mount (funionfs) that "stacks" several NFS-mounted directories. The fuse mounts are broken upon a fresh boot since systemd mounts them first, and then mounts the NFS ones. I've added noauto and comment=systemd.automount to both the NFS and fuse fstab entries, attempting to delay the fuse mounts until after the NFS mounts are already up. It's no good -- the fuse mounts always come up before the NFS mounts. I'm running Fedora 20. Is there any hope for getting this fixed in that distro, or do I have to wait (for F21, perhaps)??
Until a fstab option is implemented I can suggest a workaround - add the missing ordering dependencies using drop-in unit files. For example, if "/test" needs to be mounted after "/data", you can express that using a two-line file like this: # cat /etc/systemd/system/test.mount.d/ordering.conf [Unit] After=data.mount
I have sent a patch with this to systemd-devel. We will see... [PATCH] fstab-generator: add x-systemd.{after,requires-mounts-for}=
This is implemented now upstream. (though under the name x-systemd.requires=).