Bug 840267 - Modules compiled in F16 can't be inserted into 3.5-pre kernel
Modules compiled in F16 can't be inserted into 3.5-pre kernel
Product: Fedora
Classification: Fedora
Component: kernel (Show other bugs)
Unspecified Linux
unspecified Severity low
: ---
: ---
Assigned To: Kernel Maintainer List
Fedora Extras Quality Assurance
Depends On:
  Show dependency treegraph
Reported: 2012-07-15 03:49 EDT by Mitch Davis
Modified: 2012-07-23 09:11 EDT (History)
6 users (show)

See Also:
Fixed In Version:
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Last Closed: 2012-07-23 09:11:00 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---

Attachments (Terms of Use)
Source and objects for module which won't load. DON'T TRUST MY .ko! (48.14 KB, application/x-bzip2)
2012-07-15 03:49 EDT, Mitch Davis
no flags Details
/tmp/modsign.diff (644 bytes, patch)
2012-07-17 10:17 EDT, David Howells
no flags Details | Diff

  None (edit)
Description Mitch Davis 2012-07-15 03:49:21 EDT
Created attachment 598285 [details]
Source and objects for module which won't load.  DON'T TRUST MY .ko!


I'm developing a kernel module for a graphics tablet.  It requires a USB HID feature that's only appeared in the 3.5 kernel series, so I'm running a 3.5-pre kernel RPM from koji.  

The environment I have is a completely up-to-date 64-bit Fedora 16 (stock everything, esp toolchain, except for the kernel)

I'm developing the kernel module out-of-tree for eventual mainline inclusion.  As well as installing "kernel", I'm installing "kernel-devel" so I can compile out-of-tree.

For this bug report, I'll show a "hello world" module, because the problem is not specific to my tablet code.

The problem is that insmod fails to load my module.  The error message is "error inserting 'hello.ko': -1 Accessing a corrupted shared library".  dmesg says "Module format error encountered"

I think it is somehow related to the new signed module code.  Maybe the toolchain that comes with F16 doesn't generate something the signed crypto module code needs.

Here is a transcript:

[mjd@xiaomao hello-ko]$ cat /etc/fedora-release 
Fedora release 16 (Verne)
[mjd@xiaomao hello-ko]$ uname -r
[mjd@xiaomao hello-ko]$ rpm -q kernel kernel-devel
[mjd@xiaomao hello-ko]$ ls -l
total 8
-rw-rw-r--. 1 mjd mjd 307 Jul 15 14:18 hello.c
-rw-rw-r--. 1 mjd mjd 141 Jul 15 14:20 Makefile
[mjd@xiaomao hello-ko]$ cat hello.c 
#include <linux/module.h>
#include <linux/kernel.h>

static int __init hello_init(void)
	printk(KERN_INFO "hello_init() called\n");
	return 0;

static void __exit hello_exit(void)
	printk(KERN_INFO "hello_exit() called\n");


[mjd@xiaomao hello-ko]$ cat Makefile 
obj-m += hello.o
KDIR=/usr/src/kernels/$(shell uname -r)

	make -C $(KDIR) M=$(PWD) modules
	make -C $(KDIR) M=$(PWD) clean

[mjd@xiaomao hello-ko]$ make
make -C /usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64 M=/home/mjd/src/hello-ko modules
make[1]: Entering directory `/usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64'
  CC [M]  /home/mjd/src/hello-ko/hello.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /home/mjd/src/hello-ko/hello.mod.o
  LD [M]  /home/mjd/src/hello-ko/hello.ko.unsigned
  NO SIGN [M] /home/mjd/src/hello-ko/hello.ko
make[1]: Leaving directory `/usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64'
[mjd@xiaomao hello-ko]$ ls -l
total 280
-rw-rw-r--. 1 mjd mjd   307 Jul 15 14:18 hello.c
-rw-rw-r--. 1 mjd mjd 82832 Jul 15 14:49 hello.ko
-rw-rw-r--. 1 mjd mjd 82832 Jul 15 14:49 hello.ko.unsigned
-rw-rw-r--. 1 mjd mjd   444 Jul 15 14:49 hello.mod.c
-rw-rw-r--. 1 mjd mjd 50944 Jul 15 14:49 hello.mod.o
-rw-rw-r--. 1 mjd mjd 41568 Jul 15 14:49 hello.o
-rw-rw-r--. 1 mjd mjd   141 Jul 15 14:20 Makefile
-rw-rw-r--. 1 mjd mjd    39 Jul 15 14:49 modules.order
-rw-rw-r--. 1 mjd mjd     0 Jul 15 14:49 Module.symvers
[mjd@xiaomao hello-ko]$ file hello.ko
hello.ko: ELF 64-bit LSB relocatable, x86-64, version 1 (SYSV), not stripped
[mjd@xiaomao hello-ko]$ su
[root@xiaomao hello-ko]# insmod hello.ko
insmod: error inserting 'hello.ko': -1 Accessing a corrupted shared library
[root@xiaomao hello-ko]# tail /var/log/messages
Jul 15 13:14:50 xiaomao kernel: [ 1286.420078] Module format error encountered
[root@xiaomao hello-ko]# 

I have tried this with 3.5.0-0.rc4.git4.1.fc18.x86_64 and  3.5.0-0.rc6.git4.1.fc18.x86_64 with the same result.  insmod *does* work if I build and run the module under the stock 3.4 kernel.

This result is the same whether or not SELinux is turned on or off (eg, still happens after setenforce 0)

I asked about this on #fedora-devel and had the help of nirik and mjg59, who suggested I file a bug report.  

mjg59 said:

> Well that's bizarre
> Right so in a way you're right - the error
> is from the module signature checking code.
> But that still shouldn't result in the problem you're seeing
> It looks like you get that error if your module
> doesn't have any relocation records
> I've no idea how that would happen
> But please do file a bug

And this is it.  Thank you for your help.
Comment 1 Mitch Davis 2012-07-15 05:11:34 EDT
make V=1.  Not sure what the "Kernel configuration is invalid" is about, but I think it should work anyway.

[mjd@xiaomao hello-ko]$ make V=1
make -C /usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64 M=/home/mjd/src/hello-ko modules
make[1]: Entering directory `/usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64'
test -e include/generated/autoconf.h -a -e include/config/auto.conf || (		\
echo;								\
echo "  ERROR: Kernel configuration is invalid.";		\
echo "         include/generated/autoconf.h or include/config/auto.conf are missing.";\
echo "         Run 'make oldconfig && make prepare' on kernel src to fix it.";	\
echo;								\
mkdir -p /home/mjd/src/hello-ko/.tmp_versions ; rm -f /home/mjd/src/hello-ko/.tmp_versions/*
make -f scripts/Makefile.build obj=/home/mjd/src/hello-ko
  gcc -Wp,-MD,/home/mjd/src/hello-ko/.hello.o.d  -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/4.6.3/include -I/usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64/arch/x86/include -Iarch/x86/include/generated -Iinclude  -include /usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64/include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -O2 -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -fstack-protector -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_AVX=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -Wframe-larger-than=2048 -Wno-unused-but-set-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -g -pg -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO  -DMODULE  -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(hello)"  -D"KBUILD_MODNAME=KBUILD_STR(hello)" -c -o /home/mjd/src/hello-ko/hello.o /home/mjd/src/hello-ko/hello.c
  if [ "-pg" = "-pg" ]; then if [ /home/mjd/src/hello-ko/hello.o != "scripts/mod/empty.o" ]; then /usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64/scripts/recordmcount  "/home/mjd/src/hello-ko/hello.o"; fi; fi;
(cat /dev/null;   echo kernel//home/mjd/src/hello-ko/hello.ko;) > /home/mjd/src/hello-ko/modules.order
make -f /usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64/scripts/Makefile.modpost
  scripts/mod/modpost   -i /usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64/Module.symvers -I /home/mjd/src/hello-ko/Module.symvers  -o /home/mjd/src/hello-ko/Module.symvers -S -w  -s
  gcc -Wp,-MD,/home/mjd/src/hello-ko/.hello.mod.o.d  -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/4.6.3/include -I/usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64/arch/x86/include -Iarch/x86/include/generated -Iinclude  -include /usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64/include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -O2 -m64 -mtune=generic -mno-red-zone -mcmodel=kernel -funit-at-a-time -maccumulate-outgoing-args -fstack-protector -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_FXSAVEQ=1 -DCONFIG_AS_AVX=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -Wframe-larger-than=2048 -Wno-unused-but-set-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -g -pg -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO  -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(hello.mod)"  -D"KBUILD_MODNAME=KBUILD_STR(hello)" -DMODULE  -c -o /home/mjd/src/hello-ko/hello.mod.o /home/mjd/src/hello-ko/hello.mod.c
  ld -r -m elf_x86_64 -T /usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64/scripts/module-common.lds --build-id  -o /home/mjd/src/hello-ko/hello.ko.unsigned /home/mjd/src/hello-ko/hello.o /home/mjd/src/hello-ko/hello.mod.o 
  cp /home/mjd/src/hello-ko/hello.ko.unsigned /home/mjd/src/hello-ko/hello.ko
make[1]: Leaving directory `/usr/src/kernels/3.5.0-0.rc4.git4.1.fc18.x86_64'
[mjd@xiaomao hello-ko]$
Comment 2 Josh Boyer 2012-07-16 10:43:39 EDT
The error message is similar to the one from bug 825944, however that was fixed quite some time ago and only impacted 32-bit.

The build output shows your module being built as I would expect given that the modsign code is enabled but you obviously do not have the keys.  So the mod-extract script isn't run to generate the ELF note that contains the signature and the unsigned module is just copied directly to the final module name.

David, do you have any ideas on this one?
Comment 3 David Howells 2012-07-16 19:17:45 EDT
With just a quick look at the modules provided:

    warthog>cmp hello.ko.unsigned hello.ko

there is no signature attached to the module.  This would seem to be corroborated by this bit of the build log excerpt:

    NO SIGN [M] /home/mjd/src/hello-ko/hello.ko

Presumably because the keys weren't available to sign the module.

I'll have to go and have a poke through the sources.

Note that this isn't anything to do with SELinux, so fiddling with that shouldn't make any difference.

Am I right in thinking that you're running the F-18 kernel under the F-16 system.  Did you upgrade any of the build tools (binutils or elfutils) also?
Comment 4 Mitch Davis 2012-07-16 21:11:37 EDT
Thanks for looking at the problem David.

(In reply to comment #3)
> With just a quick look at the modules provided:
>     warthog>cmp hello.ko.unsigned hello.ko
>     warthog>

I get the same here.

> Am I right in thinking that you're running the F-18 kernel under the F-16
> system.

Yes, that's right.

> Did you upgrade any of the build tools (binutils or elfutils) also?

No, it's a stock, up-to-date F16 system, just running and compiling against an F18 kernel pulled down from koji.

Comment 5 David Howells 2012-07-17 09:20:48 EDT
Turning on DEBUG in the module verify code, shows that section 26 is failing on this test:

		seccheck((section->sh_offset & (section->sh_addralign - 1)) == 0);

which doesn't work if sh_addralign is 0, like so:

  [26] .note.GNU-stack   PROGBITS         0000000000000000  00013850
       0000000000000000  0000000000000000           0     0     0

The modules I've built have alignment >= 1, like so:

  [34] .note.GNU-stack   PROGBITS         0000000000000000  0000b809
       0000000000000000  0000000000000000   X       0     0     1

However, the ELF spec says 0 and 1 specify no constraint.
Comment 6 David Howells 2012-07-17 10:17:00 EDT
Created attachment 598652 [details]

Skip the ELF section alignment check if the alignment is 0 or 1 as the ELF spec says this means no alignment constraint.  The problem with the sample kernel module is that one of the modules has a section with an alignment of 0, which causes the check to malfunction.
Comment 7 Josh Boyer 2012-07-17 11:02:02 EDT
Thanks David.

Mitch, I've started a scratch build of the latest rawhide kernel plus David's patch here:


When it completes, could you please install it, and grab the kernel-devel package as well and give your modules a test against it?
Comment 8 Mitch Davis 2012-07-22 01:20:03 EDT

Sorry, I didn't see this earlier.  I've gone to the koji link, but I can't see any RPMs to download.  Can you advise me what to do next please?
Comment 9 Mitch Davis 2012-07-22 01:38:22 EDT
Ahh I found them I think, here:


I'll see how I go.
Comment 10 Mitch Davis 2012-07-22 02:57:39 EDT
With that kernel I can now load kernel modules built in F16.  Thank you!

Any more testing I can do, please let me know.
Comment 11 Josh Boyer 2012-07-23 08:06:40 EDT
Just grab the latest rawhide kernel.  I updated the modsign patches in it after I built the scratch build and it included David's patch already.
Comment 12 Josh Boyer 2012-07-23 09:11:00 EDT
Sigh.  I should have read the rest of your messages before commenting earlier.

Thank you very much for testing.  As I said in the previous comment, we updated the modsign code and it contains this fix now.  Your bug report is much appreciated!

Note You need to log in before you can comment on or make changes to this bug.