Description of problem: cloud-init fails when calling `xfs_growfs /dev/mapper/atomicos-root`: ``` Sep 11 18:15:50 localhost.localdomain cloud-init[729]: 2017-09-11 18:15:50,299 - util.py[WARNING]: Failed to resize filesystem (cmd=('xfs_growfs', '/dev/mapper/atomicos-root')) Sep 11 18:15:50 localhost.localdomain cloud-init[729]: 2017-09-11 18:15:50,301 - util.py[WARNING]: Running module resizefs (<module 'cloudinit.config.cc_resizefs' from '/usr/lib/python3.6/site-packages/cloudinit/config/cc_resizefs.py'>) failed ``` Running it manually shows: ``` [root@cloudhost ~]# xfs_info /dev/mapper/atomicos-root xfs_info: /dev/mapper/atomicos-root is not a mounted XFS filesystem ``` It looks like this change was desired: https://bugzilla.redhat.com/show_bug.cgi?id=1477192 We need to either get cloud-init updated to handle this or get xfs team to officially support this feature. Version-Release number of selected component (if applicable): [root@cloudhost ~]# rpm -q cloud-init xfsprogs cloud-init-0.7.9-8.fc27.noarch xfsprogs-4.12.0-4.fc27.x86_64 How reproducible: Always Steps to Reproduce: 1. Boot https://kojipkgs.fedoraproject.org/compose/branched/Fedora-27-20170910.n.0/compose/CloudImages/x86_64/images/Fedora-Atomic-27-20170910.n.0.x86_64.qcow2
Sigh, yes, xfs_growfs has always been documented as acting on a /mount point/ not a device. We had reports of people doing "xfs_growfs /mount/point" where they forgot that they had unmounted /mount/point and then accidentally ended up growing the filesystem above it, i.e. the root fs. And without shrink, they were stuck. Hence the change. Rejecting directories that aren't mountpoints but still allowing device paths would be a reasonable option, I guess. too bad that an undocumented behavior ended up coded into cloud-init. :(
(In reply to Eric Sandeen from comment #1) > Sigh, yes, xfs_growfs has always been documented as acting on a /mount > point/ not a device. That is interesting. I guess I would have thought device would have been supported even before mount point. i.e. usually the progression for resizing filesystems was offline resize support and then online resize support. Hence block device as an argument is not an unreasonable thing to expect to work. I'm not sure how things progressed in XFS but I guess online grow was always supported since the documentation only says specifying a mount point works. > > We had reports of people doing "xfs_growfs /mount/point" where they forgot > that they had unmounted /mount/point and then accidentally ended up growing > the filesystem above it, i.e. the root fs. And without shrink, they were > stuck. Hence the change. Yeah I understand that, tricky problem. /me would love shrink-ability on XFS. I definitely miss it from ext4. Combining XFS + lvm-thin is just not as straightforward for a normal user to comprehend. > > Rejecting directories that aren't mountpoints but still allowing device > paths would be a reasonable option, I guess. too bad that an undocumented > behavior ended up coded into cloud-init. :( +1, I think accepting a block device (with a filesystem on top) as an argument is something people would expect to work. Rejecting directories that aren't mountpoints but still allowing device paths would prevent the problem you originally described, but allow people to still pass in block devices. How do you feel about accepting that as an RFE?
xfs_growfs has always been exclusively an online operation, so implying that a device works in general isn't desired; it /must/ be mounted. There is no offline resize. If we accept (when we accepted ...) a device, it simply found the mount point from it, and used that under the covers. What information are you gathering from xfs_info?
(In reply to Eric Sandeen from comment #3) > xfs_growfs has always been exclusively an online operation, so implying that > a device works in general isn't desired; it /must/ be mounted. There is no > offline resize. good to know > > If we accept (when we accepted ...) a device, it simply found the mount > point from it, and used that under the covers. > > What information are you gathering from xfs_info? i'm not sure what cloud-init is doing under the covers, but I think it simply just tries to run `xfs_growfs /dev/mapper/atomicos-root` and it fails. I don't think it uses `xfs_info` at all. I assume the same root cause is what is behind xfs_info (BZ1477192). Is that a valid assumption?
Oh, so it really does want to grow. .. this bug refers to both actions, so wasn't sure... Ok, let me huddle with my xfs dev compatriots and figure out where to go from here...
(In reply to Eric Sandeen from comment #5) > Oh, so it really does want to grow. .. this bug refers to both actions, so > wasn't sure... oops - I meant to copy the output from the xfs_growfs command in the original bug description. Either way they both "fail" the same way: ``` [root@cloudhost ~]# xfs_info /dev/mapper/atomicos-root xfs_info: /dev/mapper/atomicos-root is not a mounted XFS filesystem [root@cloudhost ~]# xfs_growfs /dev/mapper/atomicos-root xfs_growfs: /dev/mapper/atomicos-root is not a mounted XFS filesystem ``` > > Ok, let me huddle with my xfs dev compatriots and figure out where to go > from here... Thanks! I'm hoping to solve this problem for Fedora 27 so I appreciate you getting back to me so soon.
(I see that yes, cloud-init really does want to grow the filesystem not just get info w/ xfs_info). And looking further, I think that this is all that's needed for cloud-init to invoke xfs_growfs properly: diff --git a/cloudinit/config/cc_resizefs.py b/cloudinit/config/cc_resizefs.py index ceee952..d78e17f 100644 --- a/cloudinit/config/cc_resizefs.py +++ b/cloudinit/config/cc_resizefs.py @@ -54,7 +54,7 @@ def _resize_ext(mount_point, devpth): def _resize_xfs(mount_point, devpth): - return ('xfs_growfs', devpth) + return ('xfs_growfs', mount_point) def _resize_ufs(mount_point, devpth):
Proposed as a Freeze Exception for 27-beta by Fedora user dustymabe using the blocker tracking app because: This is mostly a cosmetic bug for Atomic Host. For the cloud base image we use ext4 so we aren't affected there. It would be nice to have cloud-init working properly for xfs filesystems; proposing as freeze exception.
(In reply to Eric Sandeen from comment #7) > > def _resize_xfs(mount_point, devpth): > - return ('xfs_growfs', devpth) > + return ('xfs_growfs', mount_point) > seems to work. we'll try to propose to upstream cloud-init
feel free to cc: me if you like, and/or simply quote the man page ;)
upstream pull request: https://code.launchpad.net/~dustymabe/cloud-init/+git/cloud-init/+merge/330701
Discussed at blocker review meeting [1]: AcceptedFreezeException - This seems to be small change which will fix cloud-init for xfs filesystems [1] https://meetbot-raw.fedoraproject.org/fedora-blocker-review/2017-09-18
cloud-init-0.7.9-9.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2017-62a8a2b453
cloud-init-0.7.9-9.fc25 has been submitted as an update to Fedora 25. https://bodhi.fedoraproject.org/updates/FEDORA-2017-5507234819
cloud-init-0.7.9-9.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2017-d1613e2c23
(In reply to Fedora Update System from comment #15) > cloud-init-0.7.9-9.fc27 has been submitted as an update to Fedora 27. > https://bodhi.fedoraproject.org/updates/FEDORA-2017-d1613e2c23 I you want to see this pushed to Beta, please make sure it's tested (either here or in bodhi). Thanks.
(In reply to Kamil Páral from comment #16) > (In reply to Fedora Update System from comment #15) > > cloud-init-0.7.9-9.fc27 has been submitted as an update to Fedora 27. > > https://bodhi.fedoraproject.org/updates/FEDORA-2017-d1613e2c23 > > I you want to see this pushed to Beta, please make sure it's tested (either > here or in bodhi). Thanks. I added karma in bodhi (the update is locked right now though). Please test using the following images and also add karma in bodhi: https://dustymabe.fedorapeople.org/cloud-init-0.7.9-9-atomic.qcow2 https://dustymabe.fedorapeople.org/cloud-init-0.7.9-9.qcow2 9272e54c6303112f03be4dfa5dc06bf2 cloud-init-0.7.9-9-atomic.qcow2 3f850a752b59434c4fed6ebbd99bc3d5 cloud-init-0.7.9-9.qcow2
cloud-init-0.7.9-9.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-d1613e2c23
cloud-init-0.7.9-9.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-62a8a2b453
cloud-init-0.7.9-9.fc25 has been pushed to the Fedora 25 testing repository. If problems still persist, please make note of it in this bug report. See https://fedoraproject.org/wiki/QA:Updates_Testing for instructions on how to install test updates. You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2017-5507234819
cloud-init-0.7.9-9.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.