Bug 2173780

+++ This bug was initially created as a clone of Bug #1744693 +++

Description of problem:
grub2-mkconfig with os-prober does not work correctly. Error messages are displayed and not all existing OS are added to the grub.cfg.

Version-Release number of selected component (if applicable):
grub2-2.02-81.fc30.src.rpm

How reproducible:
Run grub2-mkconfig -o /boot/grub2/grub.cfg with elevated privileges

Steps to Reproduce:
1. Assure, that the output of "dmsetup ls" with elevated privileges is "No devices found"
2. Run "grub2-mkconfig -o /boot/grub2/grub.cfg" with elevated privileges

Actual results:
Output of grub2-mkconfig -o /boot/grub2/grub.cfg:
---
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.2.9-200.fc30.x86_64
Found initrd image: /boot/initramfs-5.2.9-200.fc30.x86_64.img
Found linux image: /boot/vmlinuz-5.2.8-200.fc30.x86_64
Found initrd image: /boot/initramfs-5.2.8-200.fc30.x86_64.img
Found linux image: /boot/vmlinuz-5.2.7-200.fc30.x86_64
Found initrd image: /boot/initramfs-5.2.7-200.fc30.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-23e754d628dd42d399b1bb9d632bde1b
Found initrd image: /boot/initramfs-0-rescue-23e754d628dd42d399b1bb9d632bde1b.img
device-mapper: remove ioctl on osprober-linux-sda2  failed: Device or resource busy
Command failed.
device-mapper: remove ioctl on osprober-linux-sda3  failed: Device or resource busy
Command failed.
device-mapper: remove ioctl on osprober-linux-sdb11  failed: Device or resource busy
Command failed.
device-mapper: remove ioctl on osprober-linux-sdb3  failed: Device or resource busy
Command failed.
device-mapper: remove ioctl on osprober-linux-sdb8  failed: Device or resource busy
Command failed.
device-mapper: remove ioctl on osprober-linux-sdb9  failed: Device or resource busy
Command failed.
Found Ubuntu 19.04 (19.04) on /dev/sda2
Found Arch Linux on /dev/sda3
Found Kali GNU/Linux Rolling (kali-rolling) on /dev/sdb11
done
---
  
grep "menuentry " /boot/grub2/grub.cfg randomly does not show all expected boot entries. Sometimes Kali is missing, sometimes there is no Ubuntu, sometimes both Kali and Ubuntu are missing. I guess, also Arch Linux is missing sometimes.

Expected results:
No error messages during the generation of the grub.cfg. And all found OS are present in the Grub menu. And I expect the output of "sudo dmsetup ls" to "No devices found" afterwards.


Additional info:
---
cat /etc/default/grub
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"
GRUB_DEFAULT=saved
GRUB_DISABLE_SUBMENU=true
GRUB_TERMINAL_OUTPUT="console"
GRUB_CMDLINE_LINUX="resume=UUID=1bda8b9c-aa68-453a-bf46-f077d0b0e472 rhgb quiet"
GRUB_DISABLE_RECOVERY="true"
GRUB_SAVEDEFAULT="true"
GRUB_ENABLE_BLSCFG="false"
#GRUB_ENABLE_BLSCFG=true
---

---
parted -l
Modell: ATA Samsung SSD 840 (scsi)
Festplatte  /dev/sda:  120GB
Sektorgröße (logisch/physisch): 512B/512B
Partitionstabelle: gpt
Disk-Flags: pmbr_boot

Nummer  Anfang  Ende    Größe   Dateisystem  Name  Flags
 1      1049kB  538MB   537MB                      bios_grub
 2      538MB   43.5GB  42.9GB  ext4
 3      43.5GB  86.4GB  42.9GB  ext4
 4      86.4GB  120GB   33.6GB  ext4


Modell: ATA ST1000DM003-1CH1 (scsi)
Festplatte  /dev/sdb:  1000GB
Sektorgröße (logisch/physisch): 512B/4096B
Partitionstabelle: msdos
Disk-Flags: 

Nummer  Anfang  Ende    Größe   Typ       Dateisystem     Flags
 1      1049kB  322GB   322GB   primary   ext4
 2      322GB   333GB   10.7GB  primary   ext4
 3      333GB   344GB   10.7GB  primary   ext4
 4      344GB   1000GB  657GB   extended
 5      344GB   362GB   18.3GB  logical   linux-swap(v1)
 6      362GB   469GB   107GB   logical   ext4
11      469GB   491GB   21.5GB  logical   ext4
 7      491GB   598GB   107GB   logical   ext4
 8      598GB   609GB   10.7GB  logical   ext4
 9      609GB   620GB   10.7GB  logical   ext4
10      620GB   1000GB  381GB   logical   ntfs
---

The system is installed in BIOS mode. Fedora is on /dev/sda4, Xubuntu 19.04 on /dev/sda2, Arch Linux on /dev/sda3, and Kali Linux on /dev/sdb11

Workaround:
Adding "dmsetup remove_all" after setting the variable OSPROBED in /etc/grub.d/30_os-prober "solves" this issue:
---
OSPROBED="`os-prober | tr ' ' '^' | paste -s -d ' '`"
if [ -z "${OSPROBED}" ] ; then
  # empty os-prober output, nothing doing
  exit 0
fi

dmsetup remove_all
---

There is an older bug report with additional informations: https://bugzilla.redhat.com/show_bug.cgi?id=1600778

--- Additional comment from Kenneth Florek on 2019-09-04 23:54:56 UTC ---

Same bug here, for a long time. I just figured it would disappear one day. One other thing I noticed is that there are items dm0, dm1,.. in /dev/. one for each "device-mapper" error. (They appear in nautilus.) For me, it about a GPT partitioned nvme drive with EFI boot. Any partition that gets this error, such as

  device-mapper: remove ioctl on osprober-linux-nvme0n1p2  failed: Device or resource busy
  Command failed.

will not get a repeat error if

 sudo grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

is run again, but the partition will still not get a line in the grub boot menu. If you keep repeating the command, eventually there will be 0 items in the grub menu (other than the Fedora one you are running.)

 "dmsetup remove_all" will reset the process to the beginning, but putting that in "/etc/grub.d/30_os-prober" does not fix the leaving out of menu items at all for me. (Even though the osprober-linux-nvme0n1p* items are gone afterwards.) Out of the 3 other partitions to boot, I get just one, nine times out of ten tries, and I have never gotten all 3.

 Using EFI, I can get a menu from the BIOS (with F11) which boots the 4 different partitions directly, which is my workaround. The reason 99.9% of Fedora users do not notice this problem is probably they do not have other linux versions on their computer. They would not even run "grub2-mkconfig." It is still a pity that this cool feature of grub2 has been undone in Fedora since at least Fedora 28.

--- Additional comment from  on 2019-11-10 19:48:43 UTC ---

With Fedora 31, the bug is gone.  Alas, all is not well: os-prober is causing grub2-mkconfig to take up to 19 minutes to execute on my systems.

--- Additional comment from Dr J Austin on 2019-11-14 12:43:25 UTC ---

Fully updated F31

os-prober is very slow when a second partition is present with a previous
version of Fedora.

The run time increases from 2 seconds to nearly 2.78 minutes
when os-prober is called - see below.

This delay (of several minutes) also occurred when initially installing F31
from a fast XFCE Live USB3 stick.
It took longer running os-prober than the whole of the rest of the installation!


geany /etc/default/grub
and add
GRUB_DISABLE_OS_PROBER="true"

[root@harting:~]$ time grub2-mkconfig --no-grubenv-update -o fred_no_os-prober
Generating grub configuration file ...
Adding boot menu entry for EFI firmware configuration
done

real    0m2.191s
user    0m1.092s
sys     0m1.008s

[root@harting:~]$ geany /etc/default/grub
and remove
GRUB_DISABLE_OS_PROBER="true"

[root@harting:~]$ time grub2-mkconfig --no-grubenv-update -o fred_os-prober
Generating grub configuration file ...
Found Fedora 30 (Thirty) on /dev/nvme0n1p5
Adding boot menu entry for EFI firmware configuration
done

real    2m47.766s
user    0m2.150s
sys     0m1.924s
[root@harting:~]$

--- Additional comment from  on 2019-11-14 17:14:31 UTC ---

Yes, indeed, the 'Device or resource busy' problem seems to be gone.

But I can confirm, that os-prober takes very long to finish: 18:38.97 total

Maybe it is better to file a new bug report and close this one?

--- Additional comment from Terry Barnaby on 2019-12-12 13:47:25 UTC ---

I have just adding bug https://bugzilla.redhat.com/show_bug.cgi?id=1782773

This is on the slowness of os-prober due to the performance of grub2-mount I believe.

--- Additional comment from ed leaver on 2020-02-09 04:47:05 UTC ---

The problem also exists in CentOS-8-Stream. But there appears to be a simple fix that works both there and in Fedora 30:

The do_unmount() function in /usr/libexec/osprobes/50mounted-tests and in /usr/libexec/linux-boot-probes/50mounted-tests calls

        dmsetup remove $dm_device

In both files change this call to

        dmsetup remove --force --deferred --retry $dm_device

Then make sure 'sudo dmsetup ls' returns "No devices found" before kicking off "grub2-mkconfig -o /boot/grub2/grub.cfg.new"

Please give it a try, let us know how it goes. Thanks!

--- Additional comment from Daniel Mach on 2020-02-25 18:23:42 UTC ---

I probably have the same problem:

Created a sparse file.
Partitioned it using sfdisk, an UEFI layout (EFI, /boot, /, swap, /home)
Ran losetup -fp <image> to make the partitions available
Mounted image's partitions in the correct order + bind-mounted dev, proc, sys
Installed packages via dnf install --installroot=... etc.
Chrooted to the installed system
Ran grub2-mkconfig

grub2-mkconfig is extremly slow and it looks it's due to os-prober.
What's even worse, it creates a config file based on the *host* environment and not the content of the image.
I'm not sure if it's because I'm running f33 host (newer) and f32 chroot
or if the host settings win because the probe comes second...
I'd expect records from image's /etc/fstab to take precedence.

I know I'm doing crazy things, but it still should be working as expected :)
If someone is interested, I can provide my intallation script and instructions as an reproducer.

--- Additional comment from Ben Cotton on 2020-04-30 20:24:25 UTC ---

This message is a reminder that Fedora 30 is nearing its end of life.
Fedora will stop maintaining and issuing updates for Fedora 30 on 2020-05-26.
It is Fedora's policy to close all bug reports from releases that are no longer
maintained. At that time this bug will be closed as EOL if it remains open with a
Fedora 'version' of '30'.

Package Maintainer: If you wish for this bug to remain open because you
plan to fix it in a currently maintained version, simply change the 'version' 
to a later Fedora version.

Thank you for reporting this issue and we are sorry that we were not 
able to fix it before Fedora 30 is end of life. If you would still like 
to see this bug fixed and are able to reproduce it against a later version 
of Fedora, you are encouraged  change the 'version' to a later Fedora 
version prior this bug is closed as described in the policy above.

Although we aim to fix as many bugs as possible during every release's 
lifetime, sometimes those efforts are overtaken by events. Often a 
more recent Fedora release includes newer upstream software that fixes 
bugs or makes them obsolete.

--- Additional comment from Ben Cotton on 2020-05-26 18:22:02 UTC ---

Fedora 30 changed to end-of-life (EOL) status on 2020-05-26. Fedora 30 is
no longer maintained, which means that it will not receive any further
security or bug fix updates. As a result we are closing this bug.

If you can reproduce this bug against a currently maintained version of
Fedora please feel free to reopen this bug against that version. If you
are unable to reopen this bug, please file a new report against the
current release. If you experience problems, please add a comment to this
bug.

Thank you for reporting this bug and we are sorry it could not be fixed.