Bug 1789155 - non-existent devices appeared after 5.4.7 kernel update
Summary: non-existent devices appeared after 5.4.7 kernel update
Keywords:
Status: CLOSED CURRENTRELEASE
Alias: None
Product: Fedora
Classification: Fedora
Component: kernel
Version: 31
Hardware: x86_64
OS: Linux
unspecified
unspecified
Target Milestone: ---
Assignee: Hans de Goede
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2020-01-08 21:57 UTC by Paul Grabbe
Modified: 2021-04-04 11:09 UTC (History)
21 users (show)

Fixed In Version: kernel-5.5.9-200.fc31 kernel-5.5.9-100.fc30
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2021-04-04 11:09:30 UTC
Type: Bug


Attachments (Terms of Use)
output of journalctl --no-hostname -k -b > dmesg.txt (108.26 KB, text/plain)
2020-01-08 21:57 UTC, Paul Grabbe
no flags Details
screenshot of Gnome Disks and lsusb output (111.87 KB, image/png)
2020-01-08 22:00 UTC, Paul Grabbe
no flags Details
lshw for system showing phantom floppy (28.67 KB, text/plain)
2020-01-13 16:41 UTC, Paul Grabbe
no flags Details
lshw of system that does not show phantom floppy (28.65 KB, text/plain)
2020-01-13 16:43 UTC, Paul Grabbe
no flags Details

Description Paul Grabbe 2020-01-08 21:57:58 UTC
Created attachment 1650846 [details]
output of journalctl --no-hostname -k -b > dmesg.txt

1. Please describe the problem:

non-existent devices appeared after kernel update. Three devices have appeared: a floppy drive (no such hardware installed), a USB drive (none installed), and a block device (/dev/pktcdvd/pktcdvd0). The USB drive appears to be related to the presence of a Belkin 6-port USB hub. Disconnect the hub and USB drive disappears. All ports are empty. All three drives visible in Gnome Disks tool. Noticed this problem upon reboot after update when Device Notifier shows a floppy disk.


2. What is the Version-Release number of the kernel:

5.4.7-200.fc31.x86_64


3. Did it work previously in Fedora? If so, what kernel version did the issue
   *first* appear?  Old kernels are available for download at
   https://koji.fedoraproject.org/koji/packageinfo?packageID=8 :

This only happened after 5.4.7 kernel update. Never seen this before.


4. Can you reproduce this issue? If so, please provide the steps to reproduce
   the issue below:

Always present with this kernel version


5. Does this problem occur with the latest Rawhide kernel? To install the
   Rawhide kernel, run ``sudo dnf install fedora-repos-rawhide`` followed by
   ``sudo dnf update --enablerepo=rawhide kernel``:

Have not tried that yet


6. Are you running any modules that not shipped with directly Fedora's kernel?:

Stock fedora kernel. No other modules installed


7. Please attach the kernel logs. You can get the complete kernel log
   for a boot with ``journalctl --no-hostname -k > dmesg.txt``. If the
   issue occurred on a previous boot, use the journalctl ``-b`` flag.

log attached.

Comment 1 Paul Grabbe 2020-01-08 22:00:20 UTC
Created attachment 1650847 [details]
screenshot of Gnome Disks and lsusb output

Comment 2 Constantine Apostol 2020-01-11 16:55:12 UTC
Hello, i would like to add, that i have the same problem. A floppy appears out of nowhere, since kernel 5.4.7-200.fc31.x86_64 and also after the latest kernel 5.4.8-200.fc31.x86_64.
I see the floppy as fd0 in dmesg, lsblk, lsdev, vmstat -d.
Of course it can't be mounted. It also shows as detected device but not mounted, at Gnome's Nautilus.

Comment 3 Paul Grabbe 2020-01-13 16:41:20 UTC
Created attachment 1651929 [details]
lshw for system showing phantom floppy

lshw output of system that shows phantom floppy

Comment 4 Paul Grabbe 2020-01-13 16:43:17 UTC
Created attachment 1651930 [details]
lshw of system that does not show phantom floppy

lshw output of system that does not show phantom floppy. Has same updates as a different system that does show phantom.

Comment 5 Hans de Goede 2020-01-13 17:02:14 UTC
I've been looking into this a bit. The floppy / fd0 device showing up is because we have dropped the die-floppy-die.patch on the rebase to 5.4 .

I'm probably one of the few people around who still remembers why we need that patch ...

So the story is like this:

1. BIOS-es are supposed to let the OS know if there is a floppy drive by listening a "PNP0700" in the list of PNP devices which cannot be auto-detected / probed properly by the OS.

2.1 Most BIOS-es simply always list the "PNP0700" device, e.g. older laptops with a custom docking station connector will always list it because the dock may have a floppy drive

2.2 In many other BIOS-es the user can configure a floppy-drive type; and unless this gets explicitly set to "None" the will add a PNP0700" device and this almost always defaults to 1.44M leading the a (ghost) fd0 device under Linux

3. The 2 BIOS problems lead to Linux loading the floppy driver and creating a fd0 device on many many machines which do not have a floppy drive at all. The UI issues caused by this are a new twist, the main reason the die-floppy-die patch was written is because probing a drive which is not there can cause long long delays during boot, or during installation time (anaconda used to look for a driver disk and each time it checked the timeouts started again).

The die-floppy-die patch does not actually kill the floppy driver, it just removes the modaliases so that it is not auto-loaded by udev. Users who actually still have a floppy driver and want to use it can do so by loading the module by e.g. dropping a file in /etc/modules-load.d .

So for the fd0 problem the solution at least is to restore the die-floppy-die patch, the other 2 new block devices are weird. Perhaps these are somehow a side-effect ?  It is probably best to fix the floppy each first and then see from there.

Jeremy (jcline) are you ok with restoring the die-floppy-die patch? And if so can you take care of restoring it before the next 5.4.x kernel build?

Comment 6 Laura Abbott 2020-01-13 17:55:11 UTC
Is there no better option than just carrying the patch forever? Part of the reason I did the clean up was because these patches had been unloved for so long.

Comment 7 Hans de Goede 2020-01-13 19:11:37 UTC
(In reply to Laura Abbott from comment #6)
> Is there no better option than just carrying the patch forever? Part of the
> reason I did the clean up was because these patches had been unloved for so
> long.

I see 2 other options:

1) Disable the floppy driver altogether, since floppies are very rare now anyways
2) Put the floppy driver in its own kernel-modules-floppy package

The problem is that there are simply many many many systems reporting the "PNP0700" pnp-id while they do not actually have a floppy drive.

Comment 8 Laura Abbott 2020-01-13 19:17:16 UTC
I guess I'd counter with what's the issue with showing these extra devices? Is it just confusion?

If there are so many systems out there with non-existant floppy drives, it would seem better to try and get this fixed upstream than just carrying a patch. If we really don't want to do that, I'd vote for turning it off.

Comment 9 Hans de Goede 2020-01-13 19:31:08 UTC
(In reply to Laura Abbott from comment #8)
> I guess I'd counter with what's the issue with showing these extra devices?
> Is it just confusion?

There are a number of issues:

1) Blocking on kernel timeouts while the kernel tries to access the non existing drive, at least in the past this used to cause significant (boot) delays

2) timeout errors in dmesg, my experience is that any kinda errors in dmesg is likely to results in bug being reported, which we then get to deal with.

3) The non existing floppy drive showing up in the UI, see the attached screenshot, which is just plain wrong.

> If there are so many systems out there with non-existant floppy drives, it
> would seem better to try and get this fixed upstream than just carrying a
> patch.

The problem is that the old floppy drive interface does not really allow detecting if there is a driver there or not; and relying on the BIOS gets it wrong more often then not. There really isn't a proper fix (I looked into this in detail around the time the die-floppy-die patch was written).

> If we really don't want to do that, I'd vote for turning it off.

Ack, a third option would be to move the /etc/modprobe.d/*.conf we already have in kernel-module-extras to /lib/modprobe.d/ where they really belong since they are packaged files and add a floppy.conf file which blacklists the floppy module with a comment to put an empty floppy.conf in /etc/modprobe.d if the user really wants to use the floppy drive. This assume the /lib vs /etc dirs work as e.g. udev does, the manpage is not clear not this, so this has to be tested first.
Note the floppy driver already is in -extras. But I guess going with just disabling at and then wait to see if anyone complains is a good first step.

Comment 10 Hans de Goede 2020-01-13 19:35:32 UTC
Ah I see now that all the /etc/modprobe.d/*.conf are there deliberately and are meant for direct editing and are marked %config(noreplace) I think that following that pattern to also blacklist the floppy module by default would be a good solution.

Comment 11 Justin M. Forbes 2020-01-13 20:17:15 UTC
I would it might be more desirable to move the floppy module to kernel-modules-extra. We aren't going to fix ancient bios issues, and I don't recall there being a more suitable workaround in kernel. I think killing the floppy module is a bit premature. They do still exist, and occasionally people like to access them. I would also be okay with the blacklist.

Comment 12 Hans de Goede 2020-01-13 21:04:23 UTC
(In reply to Justin M. Forbes from comment #11)
> I would it might be more desirable to move the floppy module to
> kernel-modules-extra.

It already is in kernel-modules-extra, yet we have 2 different users reporting the floppy unexpectedly showing up for them. IIRC kernel-modules-extra actually sometimes gets dragged in by some other packages, so it is probably installed more often then one would expect / like.

> We aren't going to fix ancient bios issues, and I
> don't recall there being a more suitable workaround in kernel. I think
> killing the floppy module is a bit premature. They do still exist, and
> occasionally people like to access them. I would also be okay with the
> blacklist.

I personally believe that the blacklist is the way to go, then we don't have to carry a patch for this anymore, while preserving the behavior of not loading it by default.

Comment 13 Justin M. Forbes 2020-03-03 16:18:21 UTC
*********** MASS BUG UPDATE **************

We apologize for the inconvenience.  There are a large number of bugs to go through and several of them have gone stale.  Due to this, we are doing a mass bug update across all of the Fedora 31 kernel bugs.

Fedora 31 has now been rebased to 5.5.7-200.fc31.  Please test this kernel update (or newer) and let us know if you issue has been resolved or if it is still present with the newer kernel.

If you have moved on to Fedora 32, and are still experiencing this issue, please change the version to Fedora 32.

If you experience different issues, please open a new bug report for those.

Comment 14 Hans de Goede 2020-03-03 16:54:56 UTC
Since we have not made any changes to reduce the effects of the dropping of the die_floppy_die.patch patch this problem still exists in current Fedora kernels.

As mentioned before I believe that the proper fix here is to drop a blacklist file for the floppy module like we are already doing for e.g. appletalk, ax25, batman-adv, etc.

E.g. do something like this:

--- a/mod-extra-blacklist.sh
+++ b/mod-extra-blacklist.sh
@@ -46,3 +46,11 @@ foreachp()
 [ -d "$buildroot/etc/modprobe.d/" ] || mkdir -p "$buildroot/etc/modprobe.d/"
 find "$buildroot/$kernel_base/extra" -name "*.ko*" | \
        foreachp check_blacklist
+
+# Many BIOS-es export a PNP-id which causes the floppy driver to autoload
+# even though most modern systems don't have a 3.5" floppy driver anymore
+# this replaces the old die_floppy_die.patch which removed the PNP-id from
+# the module
+if [ -f $buildroot/$kernel_base/extra/drivers/block/floppy.ko* ]; then
+       blacklist "floppy"
+fi


Justin, would you be ok with this change to fix this ?

Do you want me to give this a test-build and then assuming it works as expected, add it to all the current Fedora distgit kernel branches ?

Comment 15 Paul Grabbe 2020-03-03 21:34:54 UTC
Confirming comment 14 from Hans: problem still exists in kernels 5.5.5-200.fc31.x86_64 and 5.5.7-200.fc31.x86_64.

Comment 16 Constantine Apostol 2020-03-04 22:10:52 UTC
Feedback
--------
The floppy is present in all kernel versions since it was reported, including the latest one 5.5.7-200.fc31.x86_64
Also is still present in Gnome's Nautilus, Gnome's Disks, etc.

Comment 17 Justin M. Forbes 2020-03-06 15:35:14 UTC
> 
> Justin, would you be ok with this change to fix this ?
> 
> Do you want me to give this a test-build and then assuming it works as
> expected, add it to all the current Fedora distgit kernel branches ?

It is a good idea. You can do that, or I can pick it up for 5.5.9. Whichever you prefer.

Thanks

Comment 18 Hans de Goede 2020-03-06 20:10:09 UTC
(In reply to Justin M. Forbes from comment #17)
> > 
> > Justin, would you be ok with this change to fix this ?
> > 
> > Do you want me to give this a test-build and then assuming it works as
> > expected, add it to all the current Fedora distgit kernel branches ?
> 
> It is a good idea. You can do that, or I can pick it up for 5.5.9. Whichever
> you prefer.

If you can pick it up that would be great. Note that:

1. This is untested
2. We will also need this in rawhide/f32 / we also need this for the 5.6-rc# (and other future) kernels

Comment 19 Hans de Goede 2020-03-09 15:41:15 UTC
Justin, since I'm preparing some Fedora kernel changes now anyways to fix bug 1809681, bug 1564895 and bug 1808874, I will also add the suggested changes to mod-extra-blacklist.sh myself (doing a test build now).

Comment 20 Hans de Goede 2020-03-09 16:48:00 UTC
Ok, I've added the suggest mod-extra-blacklist.sh fix to distgit for F31+, so that it will be picked up by the next official Fedora kernel builds.

I've not added it to F30 since there we do not seem to have any code generating modprobe.conf.d blacklist files?

Comment 21 Justin M. Forbes 2020-03-10 14:19:36 UTC
(In reply to Hans de Goede from comment #19)
> Justin, since I'm preparing some Fedora kernel changes now anyways to fix
> bug 1809681, bug 1564895 and bug 1808874, I will also add the suggested
> changes to mod-extra-blacklist.sh myself (doing a test build now).

1809681 is not a 5.5 kernel issue and shouldn't impact f31. The dmar issues I have a potential fix for that should be in the 5.5.9 release.

Comment 22 Fedora Update System 2020-03-12 18:38:23 UTC
FEDORA-2020-57216f2e13 has been submitted as an update to Fedora 30. https://bodhi.fedoraproject.org/updates/FEDORA-2020-57216f2e13

Comment 23 Fedora Update System 2020-03-12 18:38:27 UTC
FEDORA-2020-90a64eda89 has been submitted as an update to Fedora 31. https://bodhi.fedoraproject.org/updates/FEDORA-2020-90a64eda89

Comment 24 Fedora Update System 2020-03-14 01:01:59 UTC
kernel-5.5.9-200.fc31, kernel-headers-5.5.9-200.fc31, kernel-tools-5.5.9-200.fc31 has been pushed to the Fedora 31 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-90a64eda89

Comment 25 Fedora Update System 2020-03-14 02:20:57 UTC
kernel-5.5.9-100.fc30, kernel-headers-5.5.9-100.fc30, kernel-tools-5.5.9-100.fc30 has been pushed to the Fedora 30 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2020-57216f2e13

Comment 26 Fedora Update System 2020-03-18 02:22:20 UTC
kernel-5.5.9-200.fc31, kernel-headers-5.5.9-200.fc31, kernel-tools-5.5.9-200.fc31 has been pushed to the Fedora 31 stable repository. If problems still persist, please make note of it in this bug report.

Comment 27 Fedora Update System 2020-03-18 04:05:13 UTC
kernel-5.5.9-100.fc30, kernel-headers-5.5.9-100.fc30, kernel-tools-5.5.9-100.fc30 has been pushed to the Fedora 30 stable repository. If problems still persist, please make note of it in this bug report.

Comment 28 Paul Grabbe 2020-03-18 12:29:21 UTC
kernel 5.5.9-200.fc31.x86_64 fixed the non-existent device problem I was seeing. Thanks for the great work.

Comment 29 Paul Grabbe 2021-03-27 12:31:43 UTC
Same problem has resurfaced in 5.11.10, and I believe I also saw it in 5.11.9.

Comment 30 Hans de Goede 2021-03-27 14:00:56 UTC
Thank you for reporting the return of this issue. I've submitted a merge-req for the Fedora kernels to get this fixed:

https://gitlab.com/cki-project/kernel-ark/-/merge_requests/984

Once this has been merged; and a new 5.11 kernel with it included has been build the problem should go away again.

Comment 31 Paul Grabbe 2021-04-02 13:01:59 UTC
Thank you Hans. At least on my system the problem has gone away with the 5.11.11-200.fc33.x86_64 kernel update.

Comment 32 Hans de Goede 2021-04-04 11:09:30 UTC
(In reply to Paul Grabbe from comment #31)
> Thank you Hans. At least on my system the problem has gone away with the
> 5.11.11-200.fc33.x86_64 kernel update.

Thank you for reporting back that this is re-fixed now, lets close this again.


Note You need to log in before you can comment on or make changes to this bug.