Bug 1572349

Created attachment 1427381 [details]
journalctl output from a 4.17 boot

I do not have a BCM4312, but I have a BCM4306 and a BCM4318. Both work on Ubuntu with 4.17.0-rc1. I don't think there were any changes between rc1 and rc2, but I'm building the latter and will report back if -rc2 does break b43.

Have you checked to see if rfkill is blocking wifi? See if 'rfkill list' shows anything blocked.

When I built 4.17-rc2, it did not work here either. I have started a bisection to locate the bad commit. Unfortunately, the machine in questi0on is a PowerBook G4, thus it will take a while even though there are only 8 builds needed. I will report the bad commit, and I hope a fix.

I'm pretty sure it didn't work on at least one other 4.17 kernel before rc2. I don't have it with me and can't test 4.17 kernels until I get home. I can start testing Linus' kernels over the weekend, but it will go slowly. I wasn't sure if a driver got dropped or something that might have been Fedora related.

My kernels are all from Linus's source. I was not looking forward to that bisection, and retested 4.17-rc2. This time it worked. My only conclusion is that I did not wait long enough for NetworkManager to start the interface.

In short, there does not appear to be anything wrong with kernels from mainline. The driver has certainly not been dropped. I cannot say anything about Fedora's setup.

I tried a Linus 4.17-rc2+ kernel and am seeing the same problem. rfkill list shows no blocks. ip link shows wlan0 as existing. But no APs show up when using the GUI wireless connection interface. So it looks like I'll need to do a bisect.

I'm only going to get two or three builds a day. I'm going to include the bisect log below, but there are probably too many steps left to eyeball the commits. But if someone else starts a bisect of their own it will give them a starting point.
git bisect start
# bad: [0644f186fc9d77bb5bd198369e59fb28927a3692] Merge tag 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost
git bisect bad 0644f186fc9d77bb5bd198369e59fb28927a3692
# good: [0adb32858b0bddf4ada5f364a84ed60b196dbcda] Linux 4.16
git bisect good 0adb32858b0bddf4ada5f364a84ed60b196dbcda
# bad: [9abf8acea297b4c65f5fa3206e2b8e468e730e84] Merge tag 'tty-4.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty
git bisect bad 9abf8acea297b4c65f5fa3206e2b8e468e730e84
# good: [bb2407a7219760926760f0448fddf00d625e5aec] Merge tag 'docs-4.17' of git://git.lwn.net/linux
git bisect good bb2407a7219760926760f0448fddf00d625e5aec
# bad: [093ba72914b696521e4885756a68a3332782c8de] inet: frags: add a pointer to struct netns_frags
git bisect bad 093ba72914b696521e4885756a68a3332782c8de
# good: [7083a45ad0df78546b3a735f0090b075efdf44e3] ibmvnic: Fix recent errata commit
git bisect good 7083a45ad0df78546b3a735f0090b075efdf44e3

I'm including an updated bisect log. I'm down to about 36 possible commits. Tonight I'm going to see if I can get away without make clean to finish things off.
git bisect start
# bad: [0644f186fc9d77bb5bd198369e59fb28927a3692] Merge tag 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost
git bisect bad 0644f186fc9d77bb5bd198369e59fb28927a3692
# good: [0adb32858b0bddf4ada5f364a84ed60b196dbcda] Linux 4.16
git bisect good 0adb32858b0bddf4ada5f364a84ed60b196dbcda
# bad: [9abf8acea297b4c65f5fa3206e2b8e468e730e84] Merge tag 'tty-4.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty
git bisect bad 9abf8acea297b4c65f5fa3206e2b8e468e730e84
# good: [bb2407a7219760926760f0448fddf00d625e5aec] Merge tag 'docs-4.17' of git://git.lwn.net/linux
git bisect good bb2407a7219760926760f0448fddf00d625e5aec
# bad: [093ba72914b696521e4885756a68a3332782c8de] inet: frags: add a pointer to struct netns_frags
git bisect bad 093ba72914b696521e4885756a68a3332782c8de
# good: [7083a45ad0df78546b3a735f0090b075efdf44e3] ibmvnic: Fix recent errata commit
git bisect good 7083a45ad0df78546b3a735f0090b075efdf44e3
# bad: [bc67a0daf8f3bc6fa8fcb68090f3c444de7f951c] ipmr: Make vif fib notifiers common
git bisect bad bc67a0daf8f3bc6fa8fcb68090f3c444de7f951c
# good: [872619d8cf810c17279335ef531a2a34f3b4e589] tipc: step sk->sk_drops when rcv buffer is full
git bisect good 872619d8cf810c17279335ef531a2a34f3b4e589
# good: [da18ab32d78b6414267d3e5c8c9b5ab34a6d3321] tipc: tipc_disc_addr_trial_msg() can be static
git bisect good da18ab32d78b6414267d3e5c8c9b5ab34a6d3321
# skip: [716b840c76417e608af3a8d354028604045ec46f] rsi: handle BT traffic in driver
git bisect skip 716b840c76417e608af3a8d354028604045ec46f
# bad: [070f2d7e264acd6316fc24092b7f51a18c75ac9c] net: Drop NETDEV_UNREGISTER_FINAL
git bisect bad 070f2d7e264acd6316fc24092b7f51a18c75ac9c

I've been slowed down by a lot of commits not building. I get errors like:
ERROR: "__udivdi3" [drivers/net/ethernet/mellanox/mlxsw/mlxsw_spectrum.ko] undefined!
I finally got another build completed, but can't test it remotely. Most likely the problem I'm looking for will be on the broken build side of this commit. 
Is there a config setting that I might use to avoid the building problem while still letting me test for the problem I'm looking for?

My current git bisect log is:
git bisect start
# bad: [0644f186fc9d77bb5bd198369e59fb28927a3692] Merge tag 'for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mst/vhost
git bisect bad 0644f186fc9d77bb5bd198369e59fb28927a3692
# good: [0adb32858b0bddf4ada5f364a84ed60b196dbcda] Linux 4.16
git bisect good 0adb32858b0bddf4ada5f364a84ed60b196dbcda
# bad: [9abf8acea297b4c65f5fa3206e2b8e468e730e84] Merge tag 'tty-4.17-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty
git bisect bad 9abf8acea297b4c65f5fa3206e2b8e468e730e84
# good: [bb2407a7219760926760f0448fddf00d625e5aec] Merge tag 'docs-4.17' of git://git.lwn.net/linux
git bisect good bb2407a7219760926760f0448fddf00d625e5aec
# bad: [093ba72914b696521e4885756a68a3332782c8de] inet: frags: add a pointer to struct netns_frags
git bisect bad 093ba72914b696521e4885756a68a3332782c8de
# good: [7083a45ad0df78546b3a735f0090b075efdf44e3] ibmvnic: Fix recent errata commit
git bisect good 7083a45ad0df78546b3a735f0090b075efdf44e3
# bad: [bc67a0daf8f3bc6fa8fcb68090f3c444de7f951c] ipmr: Make vif fib notifiers common
git bisect bad bc67a0daf8f3bc6fa8fcb68090f3c444de7f951c
# good: [872619d8cf810c17279335ef531a2a34f3b4e589] tipc: step sk->sk_drops when rcv buffer is full
git bisect good 872619d8cf810c17279335ef531a2a34f3b4e589
# good: [da18ab32d78b6414267d3e5c8c9b5ab34a6d3321] tipc: tipc_disc_addr_trial_msg() can be static
git bisect good da18ab32d78b6414267d3e5c8c9b5ab34a6d3321
# skip: [716b840c76417e608af3a8d354028604045ec46f] rsi: handle BT traffic in driver
git bisect skip 716b840c76417e608af3a8d354028604045ec46f
# bad: [070f2d7e264acd6316fc24092b7f51a18c75ac9c] net: Drop NETDEV_UNREGISTER_FINAL
git bisect bad 070f2d7e264acd6316fc24092b7f51a18c75ac9c
# skip: [09e93f28aa8d37a07d0c2ad742d085a21d845cda] mt76x2: remove warnings in mt76x2_mac_write_txwi()
git bisect skip 09e93f28aa8d37a07d0c2ad742d085a21d845cda
# skip: [f63b4c971f5fb1310f145785c3b2b77651ef129e] wl1251: Update wl->nvs_len after wl->nvs is valid
git bisect skip f63b4c971f5fb1310f145785c3b2b77651ef129e
# skip: [4b5adc736828dc25ca33e263ad8c0b9dcd3bf325] brcmfmac: move allocation of control rx buffer to brcmf_sdio_bus_preinit()
git bisect skip 4b5adc736828dc25ca33e263ad8c0b9dcd3bf325
# bad: [c2b72e8e10abfb54099828f8fdc68b8b0174e4e4] fsl/fman: remove unnecessary set_dma_ops() call and HAS_DMA dependency
git bisect bad c2b72e8e10abfb54099828f8fdc68b8b0174e4e4
# skip: [882164a4a928bcaa53280940436ca476e6b1db8e] ssb: Prevent build of PCI host features in module
git bisect skip 882164a4a928bcaa53280940436ca476e6b1db8e
# skip: [8100091d02487ff267af0d410ceb9eefebc8ea03] bcma: Replace mdelay with usleep_range in bcma_pmu_resources_init
git bisect skip 8100091d02487ff267af0d410ceb9eefebc8ea03
# skip: [262f2b53f67936b59cc8dfc6f3899ab8905bf1ed] brcmfmac: call brcmf_attach() just before calling brcmf_bus_started()
git bisect skip 262f2b53f67936b59cc8dfc6f3899ab8905bf1ed
# skip: [6942bdc4bf577e6fedff03111d10763a07622734] rtlwifi: enable mac80211 fast-tx support
git bisect skip 6942bdc4bf577e6fedff03111d10763a07622734
# skip: [6942bdc4bf577e6fedff03111d10763a07622734] rtlwifi: enable mac80211 fast-tx support
git bisect skip 6942bdc4bf577e6fedff03111d10763a07622734
# skip: [551e1ef4d29134ff1bb01869d522dbae0438032e] mt76: add mt76_init_stream_cap routine
git bisect skip 551e1ef4d29134ff1bb01869d522dbae0438032e
# skip: [1ca72c3047aa1146fe22b1e79be713ac90572827] rtlwifi: Add Support VHT to spec_ver
git bisect skip 1ca72c3047aa1146fe22b1e79be713ac90572827
# skip: [0542503c4c164c65cd1567b0f2b3f887af6c81eb] brcmfmac: move brcmf_attach() function in core.c
git bisect skip 0542503c4c164c65cd1567b0f2b3f887af6c81eb
# skip: [a24853aab59184ebd19c5e078c7b29e1c316e3a1] ssb: use put_device() if device_register fail
git bisect skip a24853aab59184ebd19c5e078c7b29e1c316e3a1
# skip: [24114a5f9470f75b47bdd6d40f4fc784326a9b58] mt76: initialize available_antennas_{tx,rx} info
git bisect skip 24114a5f9470f75b47bdd6d40f4fc784326a9b58
# skip: [81b813ed2fde96fd28cee477281b724ea4a28dcc] rtlwifi: Add rate section and its related definition and comment
git bisect skip 81b813ed2fde96fd28cee477281b724ea4a28dcc

The built, but untested commit is:
996bfed118748c128ad4b6c05c09fd2f5fdfa1b4 Merge tag 'wireless-drivers-next-for-davem-2018-03-24' of git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/wireless-drivers-next

I turned off the mlxsw module in .config and I can build commits in the range I was having trouble in before. I have one good test since doing so. Though it is possible that not building that module could suppress the bug I'm looking for. Hopefully I'll be able to finish the bisect this weekend.

The bad commit appears to be:
882164a4a928bcaa53280940436ca476e6b1db8e ssb: Prevent build of PCI host features in module

I'm going to check and see if the problem has been fixed in Linus' tree since I started the bisect. If it hasn't, I'll test reverting the bad commit to see if that will fix things.

The problem appears to still be present in: ee946c36be21dcfe044f1c432cd6c6a33682e244 Merge tag 'platform-drivers-x86-v4.17-2' of git://git.infradead.org/linux-platform-drivers-x86

Reverting 882164a4a928bcaa53280940436ca476e6b1db8e from ee946c36be21dcfe044f1c432cd6c6a33682e244 fixes wireless, so we can be very sure something in 882164a4a928bcaa53280940436ca476e6b1db8e is triggering the problem I am seeing.

This is not a bug, but rather a configuration problem. That commit really did fix a bug, and it needs to be kept. It seems, however, that it did introduce another error.

To help figure out what is wrong, please post the configuration used for a "good" and "bad" kernel.

Created attachment 1432414 [details]
Configure file used for both builds

I used this when I built from ee946c36be21dcfe044f1c432cd6c6a33682e244 with and without 882164a4a928bcaa53280940436ca476e6b1db8e reverted. So it covers both a good and a bad build. The file was touched by the second build, but I wasn't asked any questions, so in theory it shouldn't have been changed.

Your assumption that the configuration file did not change is incorrect. Kmake can silently add or subtract variables. When I took your posted configuration file, copied it ti .config on my system (with commit 882164a4a928 included) and did a 'make oldconfig', the following differences resulted:
--- fedora_config_before      2018-05-06 19:06:25.532129517 -0500
+++ fedora_config_after       2018-05-06 19:13:27.270729563 -0500
@@ -2900,7 +2898,6 @@ CONFIG_B43_BUSES_BCMA_AND_SSB=y
 # CONFIG_B43_BUSES_BCMA is not set
 # CONFIG_B43_BUSES_SSB is not set
 CONFIG_B43_PCI_AUTOSELECT=y
-CONFIG_B43_PCICORE_AUTOSELECT=y
 CONFIG_B43_SDIO=y
 CONFIG_B43_BCMA_PIO=y
 CONFIG_B43_PIO=y
@@ -4279,8 +4275,6 @@ CONFIG_SSB_PCMCIAHOST=y
 CONFIG_SSB_SDIOHOST_POSSIBLE=y
 CONFIG_SSB_SDIOHOST=y
 # CONFIG_SSB_DEBUG is not set
-CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
-CONFIG_SSB_DRIVER_PCICORE=y
 CONFIG_SSB_DRIVER_GPIO=y
 CONFIG_BCMA_POSSIBLE=y
 CONFIG_BCMA=m

The change also affected configuration parameters for B44 and B43LEGACY, but they are not important here. In fact, the removal of SSB_DRIVER_PCICORE is the most important change as it causes driver ssb to ignore the PCI cores of the device, which completely destroys the possibility for b43 to communicate.

I will try to find a proper solution for the faulty commit. In the meantime, change CONFIG_SSB from "m" to "y". That will also restore your system.

I'll test this (CONFIG_SSB=y) with rc4. It should be done by the time I get home. Worst case Fedora can change their config to avoid the problem. I'm still keeping a 4.16 Fedora kernel on the machine, so I don't need something quickly. I just don't want to be locked out of future kernels as that will eventually be a problem.

I tested CONFIG_SSB=y with rc4 and wireless worked normally.

Excellent, thanks for tracking this down, I know it took a while. I will get the change in todays rawhide build.

I think the "final" fix is likely to be

diff --git a/drivers/ssb/Kconfig b/drivers/ssb/Kconfig
index 9371651d8017..8e1579fa5fbc 100644
--- a/drivers/ssb/Kconfig
+++ b/drivers/ssb/Kconfig
@@ -117,7 +117,7 @@ config SSB_SERIAL
 
 config SSB_DRIVER_PCICORE_POSSIBLE
        bool
-       depends on SSB_PCIHOST && SSB = y
+       depends on SSB && (PCI = y || PCI = SSB)
        default y
 
 config SSB_DRIVER_PCICORE

In any case, setting CONFIG_SSB=y will work.

My BCM43XX cards are all PCMCIA-based PCI cards and they do not show this problem.

Thanks for your help.

I'll test Larry's fix, building a new kernel during the day. I'll also make sure to test normal Fedora kernels as soon as there is one available.
Your welcome for my testing effort. I really should be doing more work for Fedora.
Thank you for getting fixes upstream and deployed.

I tested Larry's fix with CONFIG_SSB=m and it worked fine.

I wasn't sure about the timing of when the other fix was applied to Fedora's kernel config and was sure kernel-4.17.0-0.rc4.git1.1.fc29 had it and I didn't test it last night. But looking at the Josh's git tree, it looks like it was there. So I'll get the latest rawhide kernel tested tonight.

I ended up testing this with 4.17.0-0.rc4.git1.2.fc29.i686 from the rawhide nodebug repo and it worked fine.
I don't know if you want to wait for Larry's fix to get upstream to close this (in case you want to go back to CONFIg_SSB=m) so I'm going to leave it open.

It looks like the fix is going to be these two commits:
https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/wireless-drivers.git/commit/?id=36910d82a80c1c0c61e505c6d3ecaa901ee13a26
https://git.kernel.org/pub/scm/linux/kernel/git/kvalo/wireless-drivers.git/commit/?id=ebd27d3317c6521a9511f779ea96dc943c4e8003
They haven't made it into Linus' tree yet.

The commits are in Dave Miller's net tree now. Once I seem them in Linus' tree I'll test building it with CONFIG_SSB=m with an otherwise Fedora config file.

The fix is now in Linus' tree. I'll get a build done overnight to test them.
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=ebd27d3317c6521a9511f779ea96dc943c4e8003

Created attachment 1441952 [details]
Config file used to build from bc2dbc5420e82560e650f8531ceca597441ca171

I built the latest Linus kernel (bc2dbc5420e82560e650f8531ceca597441ca171) last night using a config file based off of config-4.17.0-0.rc6.git2.1.fc29.i686 with CONFIG_SSB=m and it worked normally.
So you should be able to go back to CONFIG_SSB=m now if you want.

I've just checked that the bug is still present in last available fc28 kernel, kernel-4.17.3-200.fc28.x86_64

Last available 4.16 kernel instead works great (kernel-4.16.15-300.fc28.x86_64) and lspci provides:
09:00.0 Ethernet controller: Broadcom Limited NetLink BCM5784M Gigabit Ethernet PCIe (rev 10)
0c:00.0 Network controller: Broadcom Limited BCM4312 802.11b/g LP-PHY (rev 01)

I hope this info can be useful.

4.17.0-0.rc7.git1.2.fc29.i686 works for me. After that kernel i386 builds were dropped for a while and recent 4.18 builds have some other problem that I haven't looked at in detail. So I can't tell if the problem was reintroduced after being fixed. Still that would be unusual. In Fedora 29 CONFIG_SSB=Y which hides the problem. I don't know if that got done in Fedora 28. Possibly you are seeing a different problem.

Kernel 4.17.3-100.fc27.x86_64 breaks Broadcom network 4356 and bluetooth 43142.  

Both Kconfig patches are configured per comment #25

Installed kernel 4.17.4 no change in wifi, still does not work.  Wifi module does not load due to module verification failure.

The severity of this bug needs to be changed to high.



Jul  7 09:09:27 BRSINC-01Fed kernel: wl: module verification failed: signature and/or required key missing - tainting kernel
Jul  7 09:09:27 BRSINC-01Fed systemd[1]: Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.
Jul  7 09:09:27 BRSINC-01Fed kernel: acer_wmi: Acer Laptop ACPI-WMI Extras
Jul  7 09:09:27 BRSINC-01Fed kernel: acer_wmi: Unsupported machine has AMW0_GUID1, unable to load
Jul  7 09:09:27 BRSINC-01Fed kernel: uvcvideo: Found UVC 1.00 device HP TrueVision HD (04f2:b56c)
Jul  7 09:09:27 BRSINC-01Fed kernel: uvcvideo 2-2:1.0: Entity type for entity Extension 4 was not initialized!
Jul  7 09:09:27 BRSINC-01Fed kernel: uvcvideo 2-2:1.0: Entity type for entity Processing 2 was not initialized!
Jul  7 09:09:27 BRSINC-01Fed kernel: uvcvideo 2-2:1.0: Entity type for entity Camera 1 was not initialized!
Jul  7 09:09:27 BRSINC-01Fed kernel: input: HP TrueVision HD: HP TrueVision as /devices/pci0000:00/0000:00:10.0/usb2/2-2/2-2:1.0/input/input12
Jul  7 09:09:27 BRSINC-01Fed kernel: usbcore: registered new interface driver uvcvideo
Jul  7 09:09:27 BRSINC-01Fed kernel: USB Video Class driver (1.1.1)
Jul  7 09:09:27 BRSINC-01Fed systemd[1]: Starting Load/Save RF Kill Switch Status...
Jul  7 09:09:27 BRSINC-01Fed kernel: eth0: Broadcom BCM4365 802.11 Hybrid Wireless Controller 6.30.223.271 (r587334)

I do not see any BCM4312 mention in the posted excerpt of the log. The 4365 does  not use b43, which is the basis of this thread. The bug reported and fixed here cannot be causing your problem. Without having the PCI ID for the device, I cannot even tell you what driver is appropriate.

You need to create a new bug. When you do, I suggest that you post the entire output of dmesg, and the output of 'lspci -nn'.

The bug reported in comment 31 and 32 has been moved to bug 1599138

Broadcom WiFi still broken in 4.7.11, on Fedora 27, for a BCM4322 module in my Dell laptop. Currently using 4.16.16, the last version where WiFi worked.

bug 1599138 is very much related to this one

I have been having this same issue on my old Dell laptop for quite a while now.  All Fedora kernels 4.17.x and 4.18.x have had problems with the Broadcom BCM4322, and I have had to run a 4.16 kernel to keep my wifi working.  I am running Fedora 28 with the broadcom-wl rpmfusion-nonfree package version 6.30.223.271-6.fc28 installed.  Over the weekend, I spent some time trying to figure out what was going on, and I tried the following to arrive at a solution.

When I run lspci, I see the BCM4322 as follows:

  $ lspci
  [...]
  04:00.0 Network controller: Broadcom Inc. and subsidiaries BCM4322 802.11a/b/g/n Wireless LAN Controller (rev 01)
  [...]

  So I then looked at the system logs for both a working 4.16.18 and a non-working 4.18.9 kernel, pulling references to the PCI adapter 04.00.0:

  $ grep '04:00.0' log-4.16.18.txt (journalctl -b 0)
  Sep 22 06:30:31 bear-republic kernel: pci 0000:04:00.0: [14e4:432b] type 00 class 0x028000
  Sep 22 06:30:31 bear-republic kernel: pci 0000:04:00.0: reg 0x10: [mem 0xf8000000-0xf8003fff 64bit]
  Sep 22 06:30:31 bear-republic kernel: pci 0000:04:00.0: enabling Extended Tags
  Sep 22 06:30:31 bear-republic kernel: pci 0000:04:00.0: supports D1 D2
  Sep 22 06:30:31 bear-republic kernel: pci 0000:04:00.0: PME# supported from D0 D3hot D3cold
  Sep 22 06:30:50 bear-republic kernel: wl 0000:04:00.0 wlp4s0: renamed from eth0
  Sep 22 06:30:54 bear-republic NetworkManager[954]: <info>  [1537615854.3421] rfkill0: found WiFi radio killswitch (at /sys/devices/pci0000:00/0000:00:1c.1/0000:04:00.0/ieee80211/phy0/rfkill0) (driver wl)
  Sep 22 06:30:54 bear-republic ModemManager[767]: <info>  Couldn't check support for device at '/sys/devices/pci0000:00/0000:00:1c.1/0000:04:00.0': not supported by any plugin

  $ grep '04:00.0' log-4.18.9.txt (journalctl -b 0)
  Sep 23 08:10:17 bear-republic kernel: pci 0000:04:00.0: [14e4:432b] type 00 class 0x028000
  Sep 23 08:10:17 bear-republic kernel: pci 0000:04:00.0: reg 0x10: [mem 0xf8000000-0xf8003fff 64bit]
  Sep 23 08:10:17 bear-republic kernel: pci 0000:04:00.0: enabling Extended Tags
  Sep 23 08:10:17 bear-republic kernel: pci 0000:04:00.0: supports D1 D2
  Sep 23 08:10:17 bear-republic kernel: pci 0000:04:00.0: PME# supported from D0 D3hot D3cold
  Sep 23 08:10:17 bear-republic kernel: ssb: Sonics Silicon Backplane found on PCI device 0000:04:00.0

The first 5 entries are the same in both logs, but line 6 shows output from the wl module in 4.16.18, and line 6 of the 4.18.9 kernel shows output from ssb.  It turns out the the 4.16 kernels were configuring ssb as a module, while the 4.17 and 4.18 kernels are including it directly in the kernel:

  $ grep CONFIG_SSB /boot/config-4.16.18-200.fc28.x86_64
  CONFIG_SSB_POSSIBLE=y
  CONFIG_SSB=m
  CONFIG_SSB_SPROM=y
  CONFIG_SSB_BLOCKIO=y
  CONFIG_SSB_PCIHOST_POSSIBLE=y
  CONFIG_SSB_PCIHOST=y
  CONFIG_SSB_B43_PCI_BRIDGE=y
  CONFIG_SSB_PCMCIAHOST_POSSIBLE=y
  CONFIG_SSB_PCMCIAHOST=y
  CONFIG_SSB_SDIOHOST_POSSIBLE=y
  CONFIG_SSB_SDIOHOST=y
  # CONFIG_SSB_DEBUG is not set
  CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
  CONFIG_SSB_DRIVER_PCICORE=y
  CONFIG_SSB_DRIVER_GPIO=y
  # CONFIG_USB_HCD_SSB is not set

  $ grep CONFIG_SSB /boot/config-4.18.9-200.fc28.x86_64
  CONFIG_SSB_POSSIBLE=y
  CONFIG_SSB=y
  CONFIG_SSB_SPROM=y
  CONFIG_SSB_BLOCKIO=y
  CONFIG_SSB_PCIHOST_POSSIBLE=y
  CONFIG_SSB_PCIHOST=y
  CONFIG_SSB_B43_PCI_BRIDGE=y
  CONFIG_SSB_PCMCIAHOST_POSSIBLE=y
  CONFIG_SSB_PCMCIAHOST=y
  CONFIG_SSB_SDIOHOST_POSSIBLE=y
  CONFIG_SSB_SDIOHOST=y
  # CONFIG_SSB_DEBUG is not set
  CONFIG_SSB_DRIVER_PCICORE_POSSIBLE=y
  CONFIG_SSB_DRIVER_PCICORE=y
  CONFIG_SSB_DRIVER_GPIO=y
  # CONFIG_USB_HCD_SSB is not set

Line 2 shows the difference:  CONFIG_SSB=m for the 4.16.18 kernel, while CONFIG_SSB=y for 4.18.9 kernel.  So I thought I would build the 4.18.9 kernel with CONFIG_SSB=m and give it a try.

I downloaded the source RPM from koji (kernel-4.18.9-200.fc28.src.rpm) and updated the SOURCE/kernel-[arch].config files with CONFIG_SSB=m, rebuilt the source archive, and used mock to build a new kernel.  After reinstalling the 4.18.9 RPMs and rebooting, I now have wifi working with the latest kernel.

I am not sure if there is a problem with the SSB component that is causing this problem, but when built as a module, it is not loaded when I boot my system, and the wl module works correctly for the BCM4322.  But when SSB is included directly in the kernel, things go bad for the BCM4322 wifi and wl does not work.

The problem is not that ssb is configured with "y" or "m", but that you have both wl and ssb/b43 installed on your computer. Both are configured to respond to the PCI IDs of that BCM4322. The results are not deterministic. Whether it works depends on which device gets control first.

When you generated the new kernel, I suspect that you did not reinstall wl. If you had, you would have seen that having SSB as a module did not make any difference.

I have just tested 4.19.0-rc5 with SSB being configured with y or m. Both worked.

I am not sure why SSB is configured into the kernel with Fedora builds. My thinking is that it should be a module. After all, only a minor fraction of Linux users have this hardware.

This bug should be closed as INVALID, the problem is caused by installing both wl and ssb/b43 without blacklisting one of them.

Here is the the lsmod output for kerner 4.12.16.fc27.  You can see that both B43 and wl drivers are available.  My system has booted the correct driver "wl" for over 3 years.  There are no blacklist entries in conf file /etc/yum/pluginconf.d/blacklist.conf.  

[root@BRSINC-01Fed Sources]# lsmod | grep b43
b43                   454656  0
bcma                   61440  1 b43
mac80211              892928  1 b43
ssb                    81920  1 b43
cfg80211              737280  3 wl,b43,mac80211
mmc_core              172032  5 b43,sdhci,ssb,sdhci_pci,cqhci

[root@BRSINC-01Fed Sources]# lsmod | grep wl
wl                   6459392  0
cfg80211              737280  3 wl,b43,mac80211


Are you suggesting that I have just been "lucky" for over 3 years that the wl driver was loaded before the b43 driver.  I think not but rather the kernel is bypassing the wl driver due to the error below detailed in dmesg

[  +0.009925] wl: loading out-of-tree module taints kernel.
[  +0.000006] wl: module license 'MIXED/Proprietary' taints kernel.
[  +0.000001] Disabling lock debugging due to kernel taint
[  +0.007829] wl: module verification failed: signature and/or required key missing - tainting kernel

This was reported on 7/7/2018 in comment #32 yet there has been no response regarding this error.  The kernel is loading wl driver but does not use it due to tainting. 

Can someone following this bug confirm this same error?

(In reply to Paul Lambert from comment #39)
> Are you suggesting that I have just been "lucky" for over 3 years that the
> wl driver was loaded before the b43 driver.  I think not but rather the
> kernel is bypassing the wl driver due to the error below detailed in dmesg

With wl, a single driver is loaded upon matching the PCI ID. For ssb/b43, matching the PCI ID caused ssb to load. Then after some decoding of the PCI registers, then b43 was triggered. As long as both wl and ssb are modules, then wl would win. As soon as ssb became part of the kernel, then it got a significant head start and wl lost. It worked based on the exact nature of the specifics of the drivers. I would call that luck.

It is never safe to have two drivers that trigger on the same PCI ID without blacklisting the one you do not want.

> It is never safe to have two drivers that trigger on the same PCI ID without
> blacklisting the one you do not want.

The problem is that as long as ssb is compiled in, there is no way to blacklist it, or to unload it and it will always take over.
That solved by rebuilding the kernel with ssb as a module (as it should be, anyway).

I've been actively looking for ways to disable/blacklist the built in ssb, but I couldn't find one.

(In reply to Larry Finger from comment #40)
> (In reply to Paul Lambert from comment #39)
> > Are you suggesting that I have just been "lucky" for over 3 years that the
> > wl driver was loaded before the b43 driver.  I think not but rather the
> > kernel is bypassing the wl driver due to the error below detailed in dmesg
> 
> With wl, a single driver is loaded upon matching the PCI ID. For ssb/b43,
> matching the PCI ID caused ssb to load. Then after some decoding of the PCI
> registers, then b43 was triggered. As long as both wl and ssb are modules,
> then wl would win. As soon as ssb became part of the kernel, then it got a
> significant head start and wl lost. It worked based on the exact nature of
> the specifics of the drivers. I would call that luck.
> 
> It is never safe to have two drivers that trigger on the same PCI ID without
> blacklisting the one you do not want.

Well, since `ssb` and `b43` can't be blacklisted, I decided to leave `wl` behind and switch to `b43`. It worked and here's how I did it.

When removing `wl` and enabling `b43`, there's output in `dmesg` which complains about missing firmware. As a solution, I installed `b43-fwcutter` from the repository, then installed the Broadcom firmware as described [here](http://linuxwireless.sipsolutions.net/en/users/Drivers/b43/#other_distros). Then I made sure that `b43` was NOT blacklisted anywhere and I removed the package `broadcom-wl` completely (`kmods-wl` too).

With the firmware files in place and `wl` out of the way, the `b43` driver loaded properly with `modprobe b43` and the WiFi worked. Because `wl` doesn't exist anymore on my system, there is no more conflict at boot time and `b43` loads everytime.

To me, that is the "correct" solution. As I see it, wl should only be used for those devices that are not handled by either b43, bcmsmac, or bcmfmac.

The firmware is not included in Linux because Broadcom has refused to allow redistribution of said firmware. The only way for a user to get it is to extract it from one of the drivers that Broadcom was forced to publish after they were caught in a GPL violation. Program fwcutter is our code that extracts the individual firmware pieces from their binary blob.

(In reply to Larry Finger from comment #43)
> To me, that is the "correct" solution. As I see it, wl should only be used
> for those devices that are not handled by either b43, bcmsmac, or bcmfmac.

None of the b43, bcmsmac, or bcmfmac work for macbooks 4.1. I've tried them all.
So, on those machines, disabling the ssb at boot is paramount to a working wifi.
It's either a custom kernel or one that somehow allows to disable the ssb.

Imho, the current situation with ssb (compiled in and always on) is sub optimal because it breaks a years long, perfectly legitimate use case.

The kernel has a mantra of never-ever breaking existing use-cases. I believe our builds should strive in that direction too. Unless, of course, there are compelling reasons not to.

I agree, but it is not the kernel that is breaking things here. It is the configuration that Fedora is using. There are no reasons that I know about that force "CONFIG_SSB = y". There was a bug introduced by the ARM people that forced them to briefly force that condition, but it was fixed. Perhaps the Fedora configuration got changed at that time, but never changed back.

I am not surprised that your macbook needed to use wl. Apple has frequently used hardware that was unlike anything used by other vendors. Without you posting a PCI ID for that device, I could not comment further. As I said, wl should be the last resort, but that is the beauty of Linux. You can do what you want. Only the bug in the Fedora configuration got in the way.

(In reply to Larry Finger from comment #45)
> I agree, but it is not the kernel that is breaking things here. It is the
> configuration that Fedora is using. There are no reasons that I know about
> that force "CONFIG_SSB = y". There was a bug introduced by the ARM people
> that forced them to briefly force that condition, but it was fixed. Perhaps
> the Fedora configuration got changed at that time, but never changed back.
> 
> I am not surprised that your macbook needed to use wl. Apple has frequently
> used hardware that was unlike anything used by other vendors. Without you
> posting a PCI ID for that device, I could not comment further. As I said, wl
> should be the last resort, but that is the beauty of Linux. You can do what
> you want. Only the bug in the Fedora configuration got in the way.

Umm, no the change was a direct result of the comments in this bug, you might also want to look back at your comment #20 on May 8 (the day that this was changed):

> I think the "final" fix is likely to be
> 
> diff --git a/drivers/ssb/Kconfig b/drivers/ssb/Kconfig
> index 9371651d8017..8e1579fa5fbc 100644
> --- a/drivers/ssb/Kconfig
> +++ b/drivers/ssb/Kconfig
> @@ -117,7 +117,7 @@ config SSB_SERIAL
>  
>  config SSB_DRIVER_PCICORE_POSSIBLE
>         bool
> -       depends on SSB_PCIHOST && SSB = y
> +       depends on SSB && (PCI = y || PCI = SSB)
>         default y
>  
>  config SSB_DRIVER_PCICORE
> 
> In any case, setting CONFIG_SSB=y will work.


Anyway, your patch made it unnecessary, but it had already been changed, and I didn't see a need to change it back at the time.  If it is causing a problem now, it can certainly be changed back.

*** Bug 1599138 has been marked as a duplicate of this bug. ***

(In reply to Larry Finger from comment #45)
> I am not surprised that your macbook needed to use wl. Apple has frequently
> used hardware that was unlike anything used by other vendors. Without you
> posting a PCI ID for that device, I could not comment further. As I said, wl
> should be the last resort, but that is the beauty of Linux. You can do what
> you want. Only the bug in the Fedora configuration got in the way.

You can find that info (and a bit more) going on from comment 14 on Bug 1599138, which Jeremy Cline just marked as a duplicate of this one.

Here's a direct link to that comment:
https://bugzilla.redhat.com/show_bug.cgi?id=1599138#c15



(In reply to Justin M. Forbes from comment #46)
> If it is causing a problem now, it can certainly be changed back.

Yeah, that would be useful for a couple mac models and possibly for some other strange broadcom configs.

Created attachment 1487829 [details]
Wifi output comparing kernel 4.16.12.fc27 to 4.18.19.fc27

My Broadcom wifi device is 4365 and uses on the WL driver.  Any Fedora kernel starting with 4.17.xx does not recognize my wifi device.  When I bring up settings and select wifi there is none.  

I have extracted the pertinent setting data for both kernel 4.16.12 where wifi works and kernel 4.18.19 where is does not.

The main difference I see is that in kernel 4.18.19 the bcma driver is loaded along with bcm43xx-fwcutter.  This is does not work with WL.  Additionally, the output from dmesg shows

[  +0.047087] bcma: bus0: Found chip with id 43142, rev 0x01 and package 0x08
[  +0.000049] bcma: bus0: Core 0 found: ChipCommon (manuf 0x4BF, id 0x800, rev 0x28, class 0x0)
[  +0.000043] bcma: bus0: Core 1 found: IEEE 802.11 (manuf 0x4BF, id 0x812, rev 0x21, class 0x0)
[  +0.000058] bcma: bus0: Core 2 found: PCIe (manuf 0x4BF, id 0x820, rev 0x16, class 0x0)
[  +0.000068] bcma: bus0: Core 3 found: UNKNOWN (manuf 0x43B, id 0x368, rev 0x00, class 0x0)
[  +0.011198] bcma: bus0: Bus registered

The bcma driver should not be loading for wl Broadcom wifi.  Kernel 4.17 and later are not allowing the wl driver to load because as soon as the loader sees 43142 it assumes that it is a b43 device.  I believe this is the root problem as the "Core 3" device is showing an unknown type when in fact is most likely my wl wifi device.  

from kernel 4.12.16.fc27
02:00.0 Network controller [0280]: Broadcom Inc. and subsidiaries BCM43142 802.11b/g/n [14e4:4365] (rev 01)
	Subsystem: Hewlett-Packard Company Device [103c:804a]
	Flags: bus master, fast devsel, latency 0, IRQ 39
	Memory at f0b00000 (64-bit, non-prefetchable) [size=32K]
	Capabilities: <access denied>
	Kernel driver in use: wl
	Kernel modules: wl

The kernel loading logic for Broadcom needs to essentially duplicate the established complex table for Broadcom wifi devices.  Or, is there a simpler way by editing a .conf driver boot file?

Is bcma a module or is it built into the kernel? You could find that with "lsmod" or by checking CONFIG_BCMA in the kernel configuration file. That may be in /proc/config.gz os as a conf* file in /boot. I do not run Fedora, thus I do not know which method you need to use.

If bcma is a module, the correct way is to blacklist it in some file in /etc/modprobe.d/. If it is built into the kernel, then the Fedora kernel configuration needs to be changed. As we have no idea what the wl driver is doing, then we cannot duplicate the detection mechanism.

SOLVED: 

BCMA is a module.  The Fedora kernel configuration module file is config-4.18.9-100.fc27.x86_64
located in /boot. 

The solution is to blacklist the bcma module by adding a file to the directory /etc/modprobe.d/ that contains "blacklist bcma." 

I first identified the CONFIG_SSB entry in the 4.12.16.fc27.  This value was CONFIG_SSB=m
I then set this entry from y to m in the kernel configuration file for 4.18.9.fc27.  This made no difference as the wl wifi was still not recognized.  

Next I blacklisted the bcma driver and the wl wifi device was recognized and successfully connected.
Finally, I re-edited the kernel configuration file for 4.18.9.fc27 and changed CONFIG_SSB=m back to CONFIG_SSB=y and rebooted.  The wl wifi device was recognized and successfully connected.

The short story is to know your wifi hardware and blacklist the drivers that will NOT work.  
Use lspci -nn -d: | grep Broadcom and find the PCI ID [14e4:4365] (rev 01).  Then find one of the many Broadcom PCI.ID lookup tables to find which driver are used for this PIC ID (4365).  Run lsmod | grep -e b43 -e bcm.  Then blacklist the others such as bcma, b43, bcmfmac and bcmsmac that are returned by the lsmod command.

Congratulations. Be aware that you always need to blacklist the in-kernel driver whenever an external one uses the same PCI ID.

Note that changing the conf file in /boot does not change anything other than that file. It is a record of the configuration that was used to compile that kernel. To change what is actually in a kernel, you would need to install the kernel source, copy that configuration file to the kernel source directory as .config, edit .config to make your changes (or run "make xconfig"), then run "make" and "sudo make modules_install install", and finally reboot into the new kernel that you just built. Part of what your distro does is insulate you from that whole process.

As others have suggested, it seems logical go back to CONFIG_SSB=m, as compiling SSB into the kernel breaks some configurations that need wl.  Unless, of course, there is another usage case that *requires* CONFIG_SSB=y.  Barring that, it seems that maximum flexibility is achieved when SSB is built as a module.

CONFIG_SSB is set to be a module in the next kernel builds (4.19.0-rc6 in Rawhide, 4.18.11 on stable releases)

Running pending testing 4.18.11 on MacBook Pro 5,3 with Wifi working again.

This bug is related to and most likely simlar to Bug 1622367.  In fact bugs have been filed for all Linux distros regarding the failure of the kernel 4.17 and later breaking Broadcom wifi and bluetooth wireless devices.

If interested see my Comments #2 & #3 in bug 1622367

I think this can be closed now. The original problem got resolved and the bug report got hijacked for another problem and it looks like it got resolved as well.

I agree.