Bug 981909 - grub2-probe partmap fails for LVM on RAID
grub2-probe partmap fails for LVM on RAID
Product: Fedora
Classification: Fedora
Component: grub2 (Show other bugs)
x86_64 Linux
unspecified Severity urgent
: ---
: ---
Assigned To: Peter Jones
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2013-07-06 17:50 EDT by Larkin Lowrey
Modified: 2015-02-17 10:54 EST (History)
8 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2015-02-17 10:54:54 EST
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Patch to fix grub2-probe problem with vgs output (677 bytes, patch)
2013-07-12 12:57 EDT, Larkin Lowrey
no flags Details | Diff
replacement for /sbin/vgs to strip whitespace (178 bytes, application/x-sh)
2014-01-17 13:29 EST, Alex Headley
no flags Details

  None (edit)
Description Larkin Lowrey 2013-07-06 17:50:07 EDT
Description of problem:

grub2-mkconfig fails because grub2-probe fails to properly detect raid volumes as LVM PVs.

Running grub2-probe directly reveals the issue:

# grub2-probe --device /dev/mapper/Flash-Root --target=partmap
mdadm: cannot open /dev/md0  : No such file or directory
/usr/sbin/grub2-probe: error: cannot open ‘/dev/md0  ’: No such file or directory.

It is trying to access "/dev/md0  " and not "/dev/md0". The code parses the output of 'pvs' and strips off the leading spaces but not the trailing spaces.

Version-Release number of selected component (if applicable):

Fedora release 19 (Schrödinger’s Cat)

How reproducible:
Every time.

Steps to Reproduce:
1. Configure a LVM logical volume using one or more physical volume(s)
2. Run grub2-probe --device /dev/mapper/<lv name> --target=partmap
3. Observe errors

Actual results:
mdadm: cannot open /dev/md0  : No such file or directory
/usr/sbin/grub2-probe: error: cannot open ‘/dev/md0  ’: No such file or directory.

Expected results:
No error. I'm not sure what the correct behavior is supposed to be.

Additional info:
This all worked fine prior to using fedup to upgrade from F17 to F19.
Comment 1 Larkin Lowrey 2013-07-06 17:55:38 EDT
Correction: grub2-probe calls out to 'vgs' not 'pvs'. Here's the syscall per strace:

execve("/usr/sbin/vgs", ["vgs", "--options", "pv_name", "--noheadings", "Flash"], [/* 25 vars */]) = 0

The vgs command outputs:

[pid 16852] write(1, "  /dev/md0  \n", 13) = 13

... which grub2-probe reads then issues:

[pid 16851] stat("/dev/md0  ",  <unfinished ...>

So, it clearly stripped the leading spaces but not the trailing spaces.
Comment 2 Larkin Lowrey 2013-07-12 12:57:06 EDT
Created attachment 772826 [details]
Patch to fix grub2-probe problem with vgs output

I was in a hurry so I fixed the bug myself so I could get a working grub config. I'm attaching a patch for my changes.
Comment 3 Sam Kingston 2013-07-30 08:06:41 EDT
Why should grub2-probe have to hack around clearly broken lvm2 behaviour?

Another hack that avoids having to patch grub2-probe is to drop to a shell at failure and symlink '/dev/md0' to '/dev/md0  ' (substitute for correct md device node).
Comment 4 Larkin Lowrey 2013-07-30 12:19:23 EDT
The code is already stripping the leading whitespace. Neglecting to strip trailing whitespace leaves the code only half complete. Doing a half job that leaves the code brittle is a bug. Symlinking around the problem is a hack.
Comment 5 Jeff Buhrt 2013-12-24 09:17:39 EST
This still fails on F19 and F20. The 'bigger' problem is not just having to drop to a shell during Fedup (which BTW still wouldn't complete on a F18 -> F19 or F19 -> F20 upgrade), is that grub2-mkconfig errors and doesn't detect the config... next is grub2-install also will fail.

My configuration on all but my laptop is mdraid mirroring two drives, then LVM.
Indirectly related:
1. On some of the 'older' machines that have been upgraded and hardware replaced over time I have moved /boot into LVM as the seize requirement grew and it was supported, and in the last days had to move a /usr from it's own LV into the root LV).
2. I did a couple days of testing a F18 install ~1.25yr ago to compare MDRAID+LVM vs LVM+DMRAID per LV... Having used LVM for 20+years and Unixes for ~30yr, I was wondering about mirroring the LV's and thinking it might be helpful (a few nice hopes)... but LVM+DMRAID was slower than dirt in comparison to MDRAID+LVM.

Example on F20 after a yum upgrade:
#  grub2-mkconfig -o /tmp/grub.cfg
Generating grub.cfg ...
Found linux image: /boot/vmlinuz-3.12.5-302.fc20.i686
Found initrd image: /boot/initramfs-3.12.5-302.fc20.i686.img
mdadm: cannot open /dev/md1  : No such file or directory
/usr/sbin/grub2-probe: error: cannot open ‘/dev/md1  ’: No such file or directory.
mdadm: cannot open /dev/md1  : No such file or directory
/usr/sbin/grub2-probe: error: cannot open ‘/dev/md1  ’: No such file or directory.

the problem reported above is confirmed by adding -x to make 'set -e -x' at the top of /etc/grub.d/10_linux
+ '[' x/boot = x/ ']'
+ '[' -z '' ']'
++ prepare_grub_to_access_device /dev/mapper/BackupVG-BootLV
++ grub_add_tab
++ sed -e 's/^/	/'
+++ /usr/sbin/grub2-probe --device /dev/mapper/BackupVG-BootLV --target=partmap
mdadm: cannot open /dev/md1  : No such file or directory
/usr/sbin/grub2-probe: error: cannot open ‘/dev/md1  ’: No such file or directory.

This will break anyone who also tries to run:
# /usr/sbin/grub2-install /dev/sda
Path ‘/boot/grub2’ is not readable by GRUB on boot. Installation is impossible. Aborting.
Comment 6 Jeff Buhrt 2013-12-24 20:49:31 EST
I should have noted that the upstream 2.02~beta1 Grub2 seems fine.
The broken production F19/F20 files for LVM on mdraid:

For anyone that stumbles on this issue before a fix is released:
1) Download Grub using https://www.gnu.org/software/grub/grub-download.html
-No need to change/patch anything. Upstream Grub2 is fine.
2) yum install flex bison
3) compile following the INSTALL
4) Manually copy the two files into place:
mv /usr/sbin/grub2-probe /usr/sbin/grub2-probe.real
cp -p grub-probe /usr/sbin/grub2-probe
mv /usr/sbin/grub2-bios-setup /usr/sbin/grub2-bios-setup.real
cp -p grub-bios-setup /usr/sbin/grub2-bios-setup

If someone needs a default i686 prebuilt from the 2.02-beta1 (no DM, just MD), I put a copy at: http://www.aftinc.net/misc/grub2-tools-partial2.02~beta1.tgz
Comment 7 Alex Headley 2014-01-17 13:29:07 EST
Created attachment 851751 [details]
replacement for /sbin/vgs to strip whitespace

Just wanted to add that I ran into this bug and based on the comments in the bug report, I decided that it would be easier to just replace {/usr,}/sbin/vgs with a small script to run the original command and strip the extra whitespace from the output (though only when run with the options that grub2-probe uses). Obviously this isn't a permanent solution since it would have to be done everytime the lvm2 package is updated (or worked around with $PATH wrangling) but it was quicker than building a new grub2-tools rpm.
Comment 8 Fedora End Of Life 2015-01-09 13:43:19 EST
This message is a notice that Fedora 19 is now at end of life. Fedora 
has stopped maintaining and issuing updates for Fedora 19. It is 
Fedora's policy to close all bug reports from releases that are no 
longer maintained. Approximately 4 (four) weeks from now this bug will
be closed as EOL if it remains open with a Fedora 'version' of '19'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 19 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.
Comment 9 Fedora End Of Life 2015-02-17 10:54:54 EST
Fedora 19 changed to end-of-life (EOL) status on 2015-01-06. Fedora 19 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this

Thank you for reporting this bug and we are sorry it could not be fixed.

Note You need to log in before you can comment on or make changes to this bug.