Bug 491090 - Review Request: linux-firmware - firmware files for use with Linux kernel
Review Request: linux-firmware - firmware files for use with Linux kernel
Status: CLOSED ERRATA
Product: Fedora
Classification: Fedora
Component: Package Review (Show other bugs)
rawhide
All Linux
medium Severity medium
: ---
: ---
Assigned To: Nobody's working on this, feel free to take it
Fedora Extras Quality Assurance
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2009-03-19 09:02 EDT by David Woodhouse
Modified: 2010-05-29 09:29 EDT (History)
9 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2010-05-29 09:29:49 EDT
Type: ---
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---
kmcmartin: fedora‑review+
kevin: fedora‑cvs+


Attachments (Terms of Use)

  None (edit)
Description David Woodhouse 2009-03-19 09:02:37 EDT
Spec URL: http://david.woodhou.se/kernel-firmware.spec
SRPM URL: http://david.woodhou.se/kernel-firmware-20090319-1.src.rpm
Description: Kernel-firmware includes firmware files required for some devices to
operate.

We're being asked to ship the linux-firmware.git repository instead of the limited set of firmware files which were extracted from legacy drivers in the kernel itself. The git repository has collected a number of new firmware files, and will continue to do so.
Comment 1 David Woodhouse 2009-03-19 09:03:30 EDT
We'll want to drop the kernel-firmware subpackage from the kernel while we're at it, and tweak the Requires:
Comment 2 Itamar Reis Peixoto 2009-03-19 10:03:56 EDT
(In reply to comment #1)
> We'll want to drop the kernel-firmware subpackage from the kernel while we're
> at it, and tweak the Requires:  

seems to be a good idea for me, because kernel-firmware doesn't change every time and this will save some bandwidth.
Comment 3 Nicolas Chauvet (kwizart) 2009-04-06 09:14:42 EDT
I can review the package itself , but I can expect few problems.

While it is definitely better to find firmwares in one place over the web, it is also enhancement to remove various firmwares from the source kernel and then the kernel.src.rpm . I expect having a monolithic kernel-firmware package to lead to various problems. I will try to sort the issues I expect here, but I won't prevent this package to be introduced. So I'm starting the review.

1 - I cannot identify which driver(and license) is related to this file:
* atmsar11.fw ?

2 - Some firmware that was previously removed (seen in alsa-firmware or other packages) still appear within WHENCE - minor.

3 -  Potential Conflicting files:
* Found in alsa-firmware-1.19
File: ess/maestro3_assp_kernel.fw
File: ess/maestro3_assp_minisrc.fw
* Found in libertas-usb8388-firmware
/lib/firmware/usb8388.bin (not exactly the same namespace than kernel-firmware libertas sub-directory)

4 - kernel-firmware size.
The kernel-firmware size will grow from a ratio of 10 with new kernel-firmware introduction (320ko -> 3.2Mo) . This lead to two questions.
- Are some firmwares bundled in kernel-firmware supposed to be updated?
This might not be relevant as soon as we are using presto. But it seems safer to only bundle firmwares files that doesn't move often.
- Is it possible for big firmwares image to be splited to another package so they can be un-installed easily.
This is very important for LiveCD folk to have a image <700Mo if some old/rare firmwares can be removed.
- Do every drivers related to each firmware files are relevant on every CPU ? (x86, x86_64, ppc, ppc64). I expect some are only x86 and x86_64.
(minor).

5 - kernel-firmware version. 
Using a date will allow kernel-firmware to update the one built from the kernel.src.rpm. But it isn't clear what this date means. For example we don't know if the last collection to date is also suitable for 2.6.27 kernel or 2.6.30. 
One could say it is the last archive that have a firmware update. or the archive that collected others firmwares (even if those firmwares was older than the written date).
So keeping the kernel version may eventually suggest that a minimal kernel version is needed with this firmware collection. (like 2.6.27.$date-)

6 - Multiple ABI for firmware. 
To assure smooth transition, we used to eventually bundle multiple ABI of a firmware (seen in iwl4965-firmware) The request from rel-eng was that the same package should allow to work on kernel from F-n to F-n+2. Hence, this kind of Upgrade can allow a fallback if something went wrong.
If a firmware can use different ABI. It would be more relevant to split it in another package.

7 - kernel-firmware and kernel interaction.
Since kernel-firmware can be optional. It will be better not to have a Requires: kernel-firmware >= kf-version but a conflict: kernel-firmware < kf-version from the kernel itself. Once that said, each driver could requires a different kernel-firmware version if multiple ABI can be bundled for a given driver.

8 - Building subpackage vs splitting tarball for kernel-firmware.
It might be easier at the first sight to build sub-packages if more splitting is needed. (for big firmware). One can say it will not matter since we have presto enabled by F-11. The problem is with sorting the required version for earch firmware and the kernel cannot be solved by splitting sub-package.

9 - kernel-firmware to be a default/optional package ?
Since this package bundles various kind of firmwares, it would depends on the case to know if this would be better to have it installed by default or optional.
I think it is better to have raid disk, nic adapter and wifi, installed in the default install media while v4l devices can be installed in post install. Other adaptation could be done if the device remains rare or for the LiveMedia case.


Actually, there is only one thing that could prevent this package to be introduced in Fedora, it is the unknown redistribution status of some firmwares.
We may implicitly consider that if firmwares was contributed to the kernel, it is allowed to redistribute. Do we have a FE-legal advice about that?


So, to sum up the package review itself. I would have a general answear on the point I have raised. 
It will also be better to fix conflicting files along with thinking of a better version scheme that may prevent epoch uses before to introduce the package.
* There is a typo in the URL : htp://
* rpmint warning on kernel-firmware.src: 
 W: mixed-use-of-spaces-and-tabs (spaces: line 7, tab: line 1)
* Some licenses are bundled in the sub-directory whereas only License.* and WHENCE are bundled as %doc.

Thoses items was checked and are OK:
- Timestamps have been keept while installing files.
- Every docs/Licenses are either ASCII or UTF-8
Comment 4 David Woodhouse 2009-04-06 22:42:43 EDT
Thanks for the detailed analysis.

1. Well spotted; thanks. Will fix that upstream.

2. We're thinking about changing to a more machine-readable form for the WHENCE file, in which case perhaps we could automatically strip those out. It's harmless enough though.

3. I'll fix the ess one; the usb8388 isn't actually a duplicate AFAIK (it's for the OLPC device supporting mesh, vs. the mass market ones).

4. Spitting into subpackages might make some sense, especially if we can automatically install those packages on demand.

5. The latest one should _always_ be applicable. If firmware ABI changes, the filename must also change (like an soname changes), and the linux-firmware package would then carry _both_ versions. Which takes us to...

6. The upstream repository will continue to carry the old-ABI versions of firmware, and our 'latest' firmware package can continue to include those for as long as we desire.

7. To avoid unpleasant surprises, I've started off with the firmware package not being optional -- and was imagining that those who want _only_ Free Software on their machines would have an alternative package which Provides: kernel-firmware. I'm certainly happy to entertain ideas on how this should turn out in the end, but we do need to make sure that it 'just works' for most people.

8. I think subpackages are probably a good idea, in the end. To start with perhaps we should keep it simple and just provide everything though. Or maybe have just two: 'kernel-firmware-core' and 'kernel-firmware-extra', with the former containing just the firmwares which _used_ to be in the kernel itself (again, to reduce surprises).

9. I agree that we should probably install firmware for core devices (scsi/net) by default, and let non-essential firmware get installed on demand. Which means subpackages.

How well does presto actually work? Can we start doing this today?

With regard to redistribution status -- everything we've _added_ to the linux-firmware repository has clear permission from an authoritative source. The only ones we can be dubious about are the ones which were previously in the kernel in binary form, which was already a licensing problem... but we were shipping them anyway. I can't see how shipping them in a form such that the GPL _doesn't_ apply to them would make us any _less_ happy to ship them.

Will update the package and fix the details you mentioned.
Comment 5 Peter Lemenkov 2009-04-17 05:22:16 EDT
Just FYI - debian firmware package contains some additional blobs:

http://packages.debian.org/sid/firmware-linux

 * 3Com Typhoon firmware, version 03.001.008
...
 * kaweth/new_code.bin, version unknown
 * kaweth/new_code_fix.bin, version unknown
 * kaweth/trigger_code.bin, version unknown
 * kaweth/trigger_code_fix.bin, version unknown
 * Matrox G200 WARP engine microcode, version unknown
 * Matrox G400/G550 WARP engine microcode, version unknown
 * Rage 128 CCE microcode, version unknown
 * Radeon R100-family CP microcode, version unknown
 * Radeon R200-family CP microcode, version unknown
 * Radeon R300-family CP microcode, version unknown
 * Radeon R400-family CP microcode, version unknown
 * Radeon R500-family CP microcode, version unknown
 * Radeon RS690 CP microcode, version unknown
 * Tehuti network card firmware, version unknown
Comment 6 David Woodhouse 2009-04-17 08:29:24 EDT
Kaweth and tehuti are in the upstream repo; the 3D cards are from patches which have yet to go upstream; I'll be chasing them up for 2.6.31.
Comment 7 Jason Tibbitts 2009-06-27 03:57:15 EDT
Is anything happening with this package?
Comment 8 David Woodhouse 2009-08-21 04:49:24 EDT
Oh, I forgot to upload the package after making the fixes I was asked for. Sorry.

Spec URL: http://david.woodhou.se/kernel-firmware.spec
SRPM URL: http://david.woodhou.se/kernel-firmware-20090319-1.src.rpm
Comment 9 Nicolas Chauvet (kwizart) 2009-09-02 15:27:18 EDT
there is a 404 on the src.rpm
Comment 10 Nicolas Chauvet (kwizart) 2009-09-14 17:34:48 EDT
Ok I've missed that the archive can be downloaded from another place.

NeedChange - pick the tar.bz2 seems more appropriate instead of the tar.gz as Source0:

* Juste a note:
/lib/firmware/rt2860.bin
/lib/firmware/rt2870.bin
Thoses firmware comes from Ralink, but the kernel support for them might be missing. (they are missing from 2.6.30 at least)

* From the initial package (~300ko) to the current (~9000 ko), the package grown by a ratio of 30. I'm really in favour of more splitting. On the other side, we will probably need to update the guideline about how to package firmware, because we used to have one package per module (which module name used to match the firmware package name).

https://fedoraproject.org/wiki/Firmware and
https://fedoraproject.org/wiki/Packaging:LicensingGuidelines#BinaryFirmware. 
Quoting: "Firmware packages must be named <foo>-firmware, where <foo> is the driver or other hardware component that the firmware is for. "
This will probably need to update the current packaging guideline WRT firmwares.

* I used to try to make a dvb-firmware package using the get-dvb-firmware perl script bundled in the kernel-doc package, but thoses files where conflicting:
/lib/firmware/v4l-cx231xx-avcore-01.fw
/lib/firmware/v4l-cx23885-avcore-01.fw
/lib/firmware/v4l-cx23885-enc.fw
/lib/firmware/v4l-cx25840.fw

But some firmwares from http://www.linuxtv.org/downloads/firmware/ are still missing, I expect it would be better to have another archive/package for those v4l/dvb firmware to avoid conflict and to have them sorted (at least) by category.

Do we have an updated package (as suggested in c#6)?
Comment 11 Till Maas 2009-09-16 17:17:45 EDT
Nicolas, if you want to review this package, please set fedora-review to ? or reassing to the default owner of the component, if you do not want to.
Comment 12 David Woodhouse 2010-01-06 11:57:03 EST
Spec URL: http://david.woodhou.se/kernel-firmware.spec
SRPM URL: http://david.woodhou.se/kernel-firmware-20100106-1.src.rpm

The .bz2 files are created automatically by the kernel.org mirroring system; the .gz is the original.

We are slowly working on getting the dvb firmware into the upstream git repository; there's not a lot we can do about that in the scope of the package review. I think this is ready to be shipped now.
Comment 13 David Woodhouse 2010-01-06 12:18:50 EST
Renamed to linux-firmware for consistency with upstream. And the mirror script finally made the .bz2 file, so I included that too.

Spec URL: http://david.woodhou.se/linux-firmware.spec
SRPM URL: http://david.woodhou.se/linux-firmware-20100106-1.src.rpm
Comment 14 Peter Lemenkov 2010-01-06 13:41:41 EST
I don't see the review, Kyle. Anyway, the package is in a good shape, except missing "Require: udev" (owner of the /lib/firmware directory).
Comment 15 David Woodhouse 2010-01-06 13:56:20 EST
New Package CVS Request
=======================
Package Name: linux-firmware
Short Description: Firmware files used by the Linux kernel
Owners: dwmw2
Branches: F-12
InitialCC: dwmw2 kernel-maint
Comment 16 Kevin Fenzi 2010-01-06 16:36:05 EST
cvs done.
Comment 17 Jason Tibbitts 2010-01-26 18:17:31 EST
How did this supposedly reviewed and accepted package come to be assigned to nobody?  Who actually did the package review?
Comment 18 Nicolas Chauvet (kwizart) 2010-01-30 06:09:00 EST
@Jason
Despite I was assigned to the bug, someone else approved the package. Hence I don't feel responsible for the current shape of it, given that I wasn't one who approved.
I also lack time to sort that out.

I think we should have a better guideline for when a firmware can goes in linux-firmware and when it goes in a separate package. (as new firmwares are still  reviewed).

For this review, I think the c14 from Peter deserve an answear.
Comment 19 David Woodhouse 2010-01-30 07:15:38 EST
> For this review, I think the c14 from Peter deserve an answear.    

[dwmw2@macbook devel]$ grep udev linux-firmware.spec
Requires:	udev
Comment 20 Mr. Meval 2010-04-12 02:17:51 EDT
Is this related to the newest kernel requiring this?  xorg-x11-drv-ati-firmware

If so it breaks things.
Comment 21 Peter Lemenkov 2010-05-29 09:29:49 EDT
I think we should close this ticket now.

Note You need to log in before you can comment on or make changes to this bug.