Bug 130754
Summary: | Needed: Fedora Core kernel compilation tutorial | ||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Product: | [Fedora] Fedora Documentation | Reporter: | Paul W. Frields <stickster> | ||||||||||
Component: | docs-requests | Assignee: | Kevin Hobbs <kevin.hobbs.1> | ||||||||||
Status: | CLOSED WONTFIX | QA Contact: | Paul W. Frields <stickster> | ||||||||||
Severity: | medium | Docs Contact: | |||||||||||
Priority: | medium | ||||||||||||
Version: | devel | CC: | feliciano.matias, herrold, kevin.hobbs.1, k.georgiou, shishz | ||||||||||
Target Milestone: | --- | Keywords: | FutureFeature | ||||||||||
Target Release: | --- | ||||||||||||
Hardware: | All | ||||||||||||
OS: | Linux | ||||||||||||
URL: | http://www.redhat.com/archives/fedora-docs-list/2004-August/msg00229.html | ||||||||||||
Whiteboard: | |||||||||||||
Fixed In Version: | Doc Type: | Enhancement | |||||||||||
Doc Text: | Story Points: | --- | |||||||||||
Clone Of: | Environment: | ||||||||||||
Last Closed: | 2005-11-27 14:46:17 UTC | Type: | --- | ||||||||||
Regression: | --- | Mount Type: | --- | ||||||||||
Documentation: | --- | CRM: | |||||||||||
Verified Versions: | Category: | --- | |||||||||||
oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |||||||||||
Cloudforms Team: | --- | Target Upstream Version: | |||||||||||
Embargoed: | |||||||||||||
Bug Depends On: | |||||||||||||
Bug Blocks: | 129807 | ||||||||||||
Attachments: |
|
Description
Paul W. Frields
2004-08-24 13:00:49 UTC
I would like to participate in any way I can. I have a custom build I want to try and I figure it dosn't take any extra effort to keep a journal. Since Kevin has started this and posted a draft to the list, I'm moving it to the docs in progress tracker bug. ;-) Thanks for working on this! Created attachment 104791 [details]
A new kernel compilation tutorial
Created attachment 105353 [details]
A newer cleaner kernel compilation tutorial
This version has a much simpler rpmmacros and thus simpler commands through the
rest of document.
Kevin, What you've got is a good start. A couple things to keep in mind as you move through this process. We've exchanged a few e-mails already, so I'll just post this in Bugzilla for easy reference. Also, these things may be helpful for other new FDP contributors. 1. Please remember to set the MIME type properly on Bugzilla attachments, since otherwise most browsers will simply download garbage into the existing window. The best way to do this is change the attachment type to auto-detect. If you are posting a tarball, do a "make clean" before you make your tarball, and remove unnecessary backup files. The only things that need to be in the tarball are your XML source files and the Makefile. 2. When last we conversed, you were going to get your steps technically checked by Arjan. Have you done so? 3. The overall style of your tutorial is much improved, but the Introduction still needs a little love. Try and replicate the Introduction structure used in a couple of other tutorials, such as my usb-hotplug-tutorial or mirror-tutorial. (You can search those terms in Bugzilla or simply find them in the "all fedora-docs bugs" list.) 4. You should probably use a <procedure> for this document instead of separate <section>s. You'll probably want to cut down to three actual <section>s, the Introduction, the procedure, and your bibliography or source list. As a side effect, the rendering for the actual procedure will be on one page, which makes sense for such a short tutorial. The XML would look like this: <procedure> <step> <title>Step 1</title> <para>This is step 1.</para> </step> <step> <para>Step 2 has many parts.</para> <substeps> <step> <title>Step 2A</title> <para>This is Step 2A.</para> </step> <step> <title>Step 2B</title> <para>This is Step 2B.</para> </step> </substeps> </step> </procedure> Glad you're still working on it and improving it! 1. I'll be more carefull about the MIME type in the future. I seem to recal that it was auto, but oh well. I'll clean up before making the tarball. I thought including the renderd html was a good idea but I suppose I can just add a link. 2. I have now e-mailed Arjan. His comments to the boiled bown instructions were: > mkdir ~/rpmbuild ~/rpmbuild/SOURCES ~/rpmbuild/RPMS ~/rpmbuild/BUILD > rpm -ihv kernel-XXX.src.rpm > rpmbuild -bp --target=ARCH ~/rpmbuild/SPECS/kernel-XXX.spec ARCH->noarch noarch ought to work just fine there > cd ~/rpmbuild/BUILD/kernel-XXX/linux-XXX/ at this point you can already custom build... > make menuconfig > cp .config ~/rpmbuild/SOURCES/kernel-XXX-ARCH.config > rpmbuild -bb --target=ARCH ~/rpmbuild/SPECS/kernel-XXX.spec > sudo rpm -ihv ~/rpmbuild/RPMS/ARCH/kernel-XXX.NAME.ARCH.rpm ....and these steps are if you want your modified config to be there in rpm format. 3. OK I'll take a look. 4. Sounds good I'll try that out. I'm also concerned with my use of XXX, ARCH, and NAME. The ARCH confused one of my coworkers who tried out the tutorail this weekend. I considered using shellisms like $(arch) but I didn't think they would be more clear to the reader. To simplify your directions, make ARCH consistently "i686," "x86_64," or whatever is appropriate, rather than mixing "noarch" in there. (I'm uncertain whether "noarch" should be used during the rpmbuild -bp step; there's no compelling reason to make it different from the -bb step, though.) I can think of three ways to handle the "ARCH" question. First, in your directions you could simply include a step before the rpmbuild -bp step: export ARCH="i686" Then you can simply write the rest of your steps using $ARCH. However, DocBook XML allows you to use the tag <replaceable> around ARCH. It will render in a way that visually cues the user that they should substitute their own value. So as a second alternative you could code the XML as (ignore the line break): <screen> <userinput>rpmbuild -bb --target=<replaceable>ARCH</replaceable> ~/rpmbuild/SPECS/kernel-2.6.spec</userinput> </screen> A third way to do this -- and I think *this* is the best way -- is to eliminate the term "ARCH" entirely and use uname to generate the right arch for the user: rpmbuild -bb --target=`uname -p` ~/rpmbuild/SPECS/kernel-2.6.spec This is the most elegant solution... it doesn't require an extra step, the user types it verbatim without having to substitute or guess, and you can drop `uname -p` (remember those are backquotes, not single quotes) anywhere you are currently using ARCH in your text. Chapter : 7. Reconfigure the kernel make menuconfig => ncurses-devel require make xmenu => qt-devel require make gmenu => gtk2-devel require Chapter 2, rpmmacros: mkdir ~/rpmbuild/SRPMS Chapter 5, Disable SMP Build: This optional action saves build time when the SMP kernel will not be used. This step is not needed for correctness. Chapter 6, Apply Patches: If you modify any files within the kernel source tree, then in order to carry your modifications forward into the actual rebuild (Chapter 8, Build the Kernel), you must make your changes into an actual patch. Use "diff -Nru", put the output into a new file in ~/rpmbuild/SOURCES, give the file a PatchNNN number in the ~/rpmbuild/SPECS/kernel-XXX.spec file, insert a "%patchNNN -p1" command there. Chapter 7: Reconfigure the Kernel: Change the "%define release" string in the ~/rpmbuild/SPECS/kernel-XXX.spec file, by appending some text that will identify your changes. When your new kernel is running, then this string will appear in the output from "uname -a" and "cat /proc/version", and will distinguish your kernel from the plain distributed kernel. Chapter 8: Build the Kernel: Use "-ba" instead of "-bb". This will build the ~/rpmbuild/SRPMS/kernel-XXX.src.rpm file in addition to the ~/rpmbuild/RPMS/i686/kernel-XXX.i686.rpm file. This will help you comply with the GPL if you want to distribute your kernel to someone else, because the .src.rpm contains the _complete_ _reproducible_ recipe (with all sources and patches, both original and yours) to build _your_ kernel. This requires 50MB more diskspace and a few more minutes of time, but is well worth it. Saving your new .src.rpm file allows you (or anyone else) to reproduce your new kernel. Chapter 8: Build the Kernel: Note that one of the first things that rpmbuild does, is throw away any existing ~/rpmbuild/BUILD/kernel-XXX tree! As a result, all of your changes to the kernel source tree must be properly contained in .patch files within the ~/rpmbuild/SOURCES directory, and managed with "PatchNNN" and "%patchNNN -p1" in the .spec file. Chapter 8: Build the Kernel: Do this step using a gcc that is compatible with the particular kernel-XXX. The easiest way is to perform the build on a system that is running the original gcc that shipped with kernel-XXX. Check by comparing "uname -a" with "gcc --version". Unfortunately, in general the kernel is neither forward compatible, nor backward compatible, with any other version of gcc. Sometimes the incompatibilies are obvious at build time (such as the syntax for writing a string constant across multiple lines), but sometimes the incompatibilities are subtle (such as bad generated code for "corner cases".) [Yeah, this sucks; but that's the way it is.] Many of the corrections in #9 above aren't really valid: The SRPMS folder isn't needed since the instructions only build a binary kernel (rpmbuild -bb --target=<arch>). Disabling the SMP build is given as an "if" step. If you don't run SMP, why would you want to spend many minutes (or even hours, remember that not everyone is running a multi-GHz P4 or Athlon) compiling a useless kernel? Applying patches and reconfiguring: I think the author of comment #9 does not fully grok the way the kernel build happens. The release is automatically tagged by the build process unless a special file exists on the building host. No patching is necessary since the user is only replacing the normal kernel*-<arch>.config with the new one he just built. The rpmbuild -bp is run only to insure that any previous patches are applied correctly. As for making .src.rpm files, this is not necessary *unless* you distribute them, as the above author noted. This should be in a "Notes" or "Appendix" section after the main procedure. Most users building their own kernel are doing it for personal use, and that should be noted in the Introduction, when it is brought into compliance with fedora-docs standards later. I want a tutorial to result in more understanding of what is going on, and why, than is commonly obtained from a "cookbook recipe." Refering to Comment #9 and the criticisms of Comment #10: The SRPMS directory is required by "-bb", which will fail if it does not exist when "rpmbuild -bb" begins. [Perhaps this is a misfeature in rpmbuild and/or the .spec file.] Other parts of Comment #9 depend on -bb, so making the SRPMS directory is necessary to be consistent. The default Fedora Core configuration sets CONFIG_OPROFILE the same as CONFIG_SMP, and more than a few administrators want to run oprofile even on a uniprocessor box. There are also technical details involving the configuration and use of local APIC [Advanced Programmable Interrupt Controller] hardware: if you have the hardware on a UP board (often it is cheaper to put it on a UP board because of design commonality with SMP boards), then it can be better to run a SMP kernel to take advantage of the local APIC, than to run a UP kernel which might not utilize the local APIC as well. Also, part of the purpose of a tutorial is to explain, and understanding the difference between "needed for correctness" and "useful to save build time+space" can be helpful. As someone who customizes and builds dozens of kernels each year, and distributes them to customers who pay for this service, explicitly editing the "%define release" string has proved to be worth it many times over. The default tagging is only enough to "shift the blame" for bad results off of Fedora Core; it is not helpful enough to the builder herself. My experience is that if anyone finds out that you _have_ successfully built a kernel using rpm, then they ask you for a copy! You become a distributor, so building the .src.rpm in the first place actually saves time. Comment #10 paragraph 4, "No patching is necesssary ... [because only the .config is changing]." This may be true for the most common changes to .config, but for other sets of changes there is a fair chance that patching will be required to fixup minor [or major] incompatibilities. The set of possible .config is not exhaustively tested. I see a fair demand for feature additions which definitely do require patching. When patching is required, then a newbie most probably _will_ be surprised and disappointed by the "loss" of edits [from the "-bp" BUILD directory that did not get made into patches before the "-ba" build is started] every time rpmbuild is run. It is vital for a tutorial to explain and emphasize this property of rpmbuild, and the reason behind it. [If builds are performed using the known-reproducible procedure only, then it never can happen that there is a completed, non-reproducible build.] Comment #11, paragraph 3: all instances of "-bb" should say "-ba" instesad. Created attachment 105977 [details] a kernel compilation tutorial I've moved most of the steps into a procedure. I've added some of the things people have said. The directory was cleaned up in the tarball, so I put the html up on our lab web server. http://crab-lab.zool.ohiou.edu/kevin/kernel-compilation-tutorial-en/ John, you make good arguments for your points, so I'll concur. Thanks for taking the time to explain. In that case, I think the tutorial probably needs to address those points briefly and give concise alternatives for the reader. Created attachment 106566 [details] A snip from the kernel build output With the upgrade to Fedora Core 3 even a simple "rpmbuild --rebuild kernel-2.6.9-1.667.src.rpm" fails complaining about gpg keys. I'm assuming this can be fixed in ~/.rpmmacros by pointing rpm at the batch generated gpg key. I've missed, and haven't been able to find, usefull discussion on this. The best link I've found so far is from a mirror of the kernel archive from a month ago: http://www.uwsg.iu.edu/hypermail/linux/kernel/0410.1/index.html#1522 Any other urls would really be apreciated Are you certain this isn't a configuration problem on your system? This part works fine for me when I tested, e.g.: ++ find /var/tmp/kernel-2.6.9-1.667.pfrields-root/lib/modules/2.6.9-1.667.pfrields -name '*.ko' -type f + for i in '` find $RPM_BUILD_ROOT/lib/modules/$KernelVer -name "*.ko" -type f`'+ sh ./scripts/modsign/modsign.sh /var/tmp/kernel-2.6.9-1.667.pfrields-root/lib/modules/2.6.9-1.667.pfrields/kernel/crypto/blowfish.ko + mv -f /var/tmp/kernel-2.6.9-1.667.pfrields-root/lib/modules/2.6.9-1.667.pfrields/kernel/crypto/blowfish.ko.signed /var/tmp/kernel-2.6.9-1.667.pfrields-root/lib/modules/2.6.9-1.667.pfrields/kernel/crypto/blowfish.ko + for i in '` find $RPM_BUILD_ROOT/lib/modules/$KernelVer -name "*.ko" -type f`'+ sh ./scripts/modsign/modsign.sh /var/tmp/kernel-2.6.9-1.667.pfrields-root/lib/modules/2.6.9-1.667.pfrields/kernel/crypto/khazad.ko ...and so on. I tried this twice; the first time I commented out the %_signature and %_gpg_name defines in my ~/.rpmmacros to duplicate "clueless user" status, and the second time with these defaults in place. I'm not convinced these would have any effect on the build, from reading the .spec file. (Although you did that as well, right?) I suppose it's possible I have something else configured "correctly" here through dumb luck, but I doubt it. With kernel-2.6.9-1.678_FC3 my module signing problem went away. Please make the tutorial printer friendly. Several sections on page 2 of Kevin's tutorial exceeds the width of an 8-1/2 inch sheet of paper. It would be fairly easy to fold the long lines so that they are not truncated when printed. If you can be more specific about which lines maybe I could do a little more. The only line that gets chopped when I print the Steps HTML page from Firefox is the mkdir line. It gets chopped by the box it is in not the page. I'll split that up. Is this updated and available for FC4? Can we get it into the CVS? We can add a component in bugzilla, finish editing, and publish this. 'Twould be nice to remove this type of content from the release notes, once and for all. Grumble grumble. I've been lurking since the purchase of compatible hardware eliminated the need to compile kernels at work. However, I know it's important to get the Doc in CVS the webserver gets hit constantly for this. I'll go read the "Get CVS Access" stuff. Has the author done CVS access for this? Came up at 11 Oct 2005 FDSCo meeting. I can no longer spend work time on this, and I do not have the knowledge to add more detail. I must throw the kernel compilation tutorial "over the wall" so that I am not holding up the process any more. This has been addressed in the relnotes as of FC5t1, so in conjunction with the lack of interest in maintaining this tutorial, closing this bug. |