Uninstalled all the kernel packages (using --nodeps), reinstalled kernel-headers-2.2.15-22 and kernel-source-2.2.15-22. Recompiled a custom kernel with SMP turned OFF. Rebooted, everything works fine, including NFS. Reconfigured the kernel - the ONLY THING I CHANGED was to turn on SMP support. The kernel source tree was exactly as I left it after the first compile, I did 'make menuconfig', turned SMP on, saved, and ran 'make dep && make bzImage && make modules'; then I used linuxconf to "Add a kernel you compiled" to LILO, then after that worked, I went back and did 'make modules_install'. (Otherwise insmod loop.o tends to fail if I do modules_install before installing the new kernel.) Now, when I reboot, NFS does not work. /var/log/messages contains the following: Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/scsi/aha1740.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/scsi/aha1542.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/scsi/aha152x.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/block/xd.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/block/ide-tape.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/block/ide-probe.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/block/ide-mod.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/ipv4/rarp.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/ipv4/ip_gre.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/ipv4/ipip.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/misc/wanrouter.o: unresolved symbol(s) ... then ,later I see: Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/misc/parport_pc.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/misc/parport.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5 Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/misc/lp.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol(s) Aug 9 20:36:14 triton rc.sysinit: Finding module dependencies succeeded Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/misc/appletalk.o: unresolved symbol(s) Aug 9 20:36:14 triton depmod: /lib/modules/2.2.5-22/misc/af_spx.o: unresolved symbol(s) ... and later, Aug 9 20:36:17 triton network: Bringing up interface lo succeeded Aug 9 20:36:17 triton insmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol __global_cli Aug 9 20:36:17 triton insmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol __global_restore_flags Aug 9 20:36:17 triton insmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol __global_cli Aug 9 20:36:17 triton insmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol __global_restore_flags Aug 9 20:36:18 triton insmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol __global_cli Aug 9 20:36:18 triton insmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol __global_restore_flags Aug 9 20:36:18 triton ifup: /usr/bin/ipx_interface: socket: Invalid argument Aug 9 20:36:18 triton network: Bringing up interface eth0 succeeded ...later, Aug 9 20:36:46 triton nwserv: Aug 9 20:36:46 triton insmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol __global_cli Aug 9 20:36:46 triton insmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol __global_save_flags Aug 9 20:36:46 triton insmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol __global_restore_flags Aug 9 20:36:46 triton insmod: /lib/modules/2.2.5-22/misc/ipx.o: unresolved symbol __global_sti Aug 9 20:36:46 triton nwserv: Aug 9 20:36:46 triton nwserv: !! 08.09,20:36:46 NWS 0 0:PANIC !! Aug 9 20:36:46 triton nwserv: 08.09,20:36:46 NWS 0 0:EMUTLI:init_ipx:Invalid argument Aug 9 20:36:46 triton mars-nwe: nwserv startup succeeded and then, later still... Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:52 triton exportfs: uranus.commerced.com:/usr/local/apache/htdocs: Function not implemented Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:52 triton exportfs: mercury.commerced.com:/usr/local/apache/htdocs: Function not implemented Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:52 triton exportfs: uranus.commerced.com:/usr/local/apache/auth: Function not implemented Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:52 triton exportfs: mercury.commerced.com:/usr/local/apache/auth: Function not implemented Aug 9 20:36:52 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:53 triton exportfs: uranus.commerced.com:/home: Function not implemented Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:53 triton exportfs: mercury.commerced.com:/home: Function not implemented Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:53 triton exportfs: uranus.commerced.com:/usr/local/apache/htdocs: Function not implemented Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:53 triton exportfs: mercury.commerced.com:/usr/local/apache/htdocs: Function not implemented Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:53 triton exportfs: uranus.commerced.com:/usr/local/apache/auth: Function not implemented Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:53 triton exportfs: mercury.commerced.com:/usr/local/apache/auth: Function not implemented Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flagAug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:53 triton exportfs: uranus.commerced.com:/home: Function not implemented Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:53 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:53 triton exportfs: mercury.commerced.com:/home: Function not implemented Aug 9 20:36:53 triton nfs: Starting NFS services: succeeded Aug 9 20:36:54 triton nfs: rpc.statd startup succeeded Aug 9 20:36:55 triton nfs: rpc.rquotad startup succeeded Aug 9 20:36:55 triton nfs: rpc.mountd startup succeeded Aug 9 20:36:55 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol global_bh_lock Aug 9 20:36:55 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_cli Aug 9 20:36:55 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol kernel_flag Aug 9 20:36:55 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_save_flags Aug 9 20:36:55 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol waitqueue_lock Aug 9 20:36:55 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol __global_restore_flags Aug 9 20:36:55 triton insmod: /lib/modules/2.2.5-22/misc/sunrpc.o: unresolved symbol synchronize_bh Aug 9 20:36:55 triton rpc.nfsd: nfssvc: Function not implemented Aug 9 20:36:55 triton nfs: rpc.nfsd startup failed Aug 9 20:36:55 triton rstatd: rpc.rstatd startup succeeded Aug 9 20:36:55 triton rusersd: rpc.rusersd startup succeeded
This probably isn't an NFS bug. You can't recompile modules from RH kernel-source package - SMP isn't set correctly in headers. Tarball kernels of course work fine. This seems to be related to bug# 3151 (SMP recompile troubles).
Downloaded v2.2.12 tarball, compiled with same options and configuration, NFS problem went away. However, I still have a directly related XNTPD problem; this ALSO goes away if I boot a non-SMP image (of the *IDENTICAL* configuration), where xntpd outputs to /dev/console a message repeatedly: set_rtc_mmss: can't update from 59 to 10 set_rtc_mmss: can't update from 59 to 11 set_rtc_mmss: can't update from 59 to 12 set_rtc_mmss: can't update from 59 to 13 and so on... It also gets logged to syslogd, so /var/log/messages has the following (filtered through "grep mmss"): [...omitted...] Sep 3 20:06:22 triton kernel: set_rtc_mmss: can't update from 53 to 6 Sep 3 20:07:23 triton kernel: set_rtc_mmss: can't update from 53 to 7 Sep 3 20:30:26 triton kernel: set_rtc_mmss: can't update from 49 to 0 Sep 3 20:31:27 triton kernel: set_rtc_mmss: can't update from 49 to 1 Sep 3 20:32:28 triton kernel: set_rtc_mmss: can't update from 49 to 2 Sep 3 20:33:29 triton kernel: set_rtc_mmss: can't update from 49 to 3 Sep 3 21:07:33 triton kernel: set_rtc_mmss: can't update from 56 to 7 Sep 3 21:08:34 triton kernel: set_rtc_mmss: can't update from 56 to 8 Sep 3 21:09:35 triton kernel: set_rtc_mmss: can't update from 56 to 9 Sep 3 21:10:36 triton kernel: set_rtc_mmss: can't update from 56 to 10 Sep 3 21:33:39 triton kernel: set_rtc_mmss: can't update from 52 to 3 Sep 3 21:34:40 triton kernel: set_rtc_mmss: can't update from 52 to 4 Sep 3 21:35:41 triton kernel: set_rtc_mmss: can't update from 52 to 5 Sep 3 21:36:42 triton kernel: set_rtc_mmss: can't update from 52 to 6 Sep 3 22:10:46 triton kernel: set_rtc_mmss: can't update from 59 to 10 Sep 3 22:11:47 triton kernel: set_rtc_mmss: can't update from 59 to 11 Sep 3 22:12:48 triton kernel: set_rtc_mmss: can't update from 59 to 12 Sep 3 22:13:49 triton kernel: set_rtc_mmss: can't update from 59 to 13 Sep 3 22:36:52 triton kernel: set_rtc_mmss: can't update from 55 to 6 Sep 3 22:37:53 triton kernel: set_rtc_mmss: can't update from 55 to 7 Sep 3 22:38:54 triton kernel: set_rtc_mmss: can't update from 55 to 8 Sep 3 22:39:55 triton kernel: set_rtc_mmss: can't update from 55 to 9