Last night, the Fedora openQA servers (openqa01.qa.fedoraproject.org and openqa-stg01.qa.fedoraproject.org, internally) were rebooted, and went from kernel 4.16 to 4.17. These servers each provide an NFS share which is supposed to be mounted by the various 'worker host' boxes (qa05, qa07, qa09, qa14 etc etc). When all the boxes came up after the reboot, all these mounts failed, and there were a bunch of 'general protection fault' tracebacks clearly related to NFS in the server logs. After downgrading all the boxes back to the most recent kernel 4.16 build and rebooting again, the mounts are working again. Here's the first occurrence of the GPF from the server, with kernel 4.17.4-100.fc27.x86_64: Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: general protection fault: 0000 [#1] SMP PTI Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: Modules linked in: rpcsec_gss_krb5 xt_CHECKSUM iptable_mangle ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_nat_ipv4 nf_nat bridge stp llc tun ebtable_filter ebtables nfsv3 nfs fscache binfmt_misc ip6t_REJECT nf_reject_ipv6 nf_conntrack_ipv6 nf_defrag_ipv6 ip6table_filter ip6_tables snd_hda_codec_generic sb_edac kvm_intel snd_hda_intel snd_hda_codec kvm nf_conntrack_ipv4 nf_defrag_ipv4 irqbypass xt_conntrack snd_hda_core nf_conntrack crct10dif_pclmul crc32_pclmul snd_hwdep libcrc32c crc32c_intel snd_seq ghash_clmulni_intel snd_seq_device snd_pcm joydev snd_timer virtio_balloon snd soundcore virtio_net i2c_piix4 i6300esb nfsd nfs_acl lockd auth_rpcgss grace sunrpc qxl drm_kms_helper ttm drm serio_raw virtio_console virtio_blk ata_generic pata_acpi qemu_fw_cfg Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: CPU: 0 PID: 821 Comm: nfsd Not tainted 4.17.4-100.fc27.x86_64 #1 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: Hardware name: Red Hat KVM, BIOS 1.11.0-2.el7 04/01/2014 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: RIP: 0010:prefetch_freepointer+0x10/0x20 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: RSP: 0018:ffffbedfc3ca7c48 EFLAGS: 00010286 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: RAX: 0000000000000000 RBX: b687250af1fc0395 RCX: 0000000000000006 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: RDX: 0000000000000005 RSI: b687250af1fc0395 RDI: ffff98b125dd6280 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: RBP: ffff98b125dd6280 R08: ffff98b12b02b240 R09: ffff98ac36fec000 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: R10: ffffbedfc3ca7cb0 R11: 0000000000000000 R12: 00000000014080c0 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: R13: ffffffffc0322a21 R14: ffff98ac36db2cb2 R15: ffff98b125dd6280 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: FS: 0000000000000000(0000) GS:ffff98b12b000000(0000) knlGS:0000000000000000 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: CR2: 0000562bb9b59000 CR3: 000000029920a006 CR4: 00000000000606f0 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: Call Trace: Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: kmem_cache_alloc+0xb4/0x1c0 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: ? nfsd4_free_file_rcu+0x20/0x20 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: nfs4_alloc_stid+0x21/0xa0 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: nfsd4_process_open2+0xb84/0x14c0 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: ? nfsd_permission+0x5a/0xf0 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: ? fh_verify+0x44b/0x600 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: ? nfsd4_open+0x2dd/0x700 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: nfsd4_open+0x2dd/0x700 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: nfsd4_proc_compound+0x4f9/0x6e0 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: nfsd_dispatch+0xf5/0x230 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: svc_process_common+0x4c3/0x720 [sunrpc] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: ? nfsd_destroy+0x60/0x60 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: svc_process+0xd7/0xf0 [sunrpc] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: nfsd+0xe3/0x150 [nfsd] Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: kthread+0x113/0x130 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: ? kthread_create_worker_on_cpu+0x70/0x70 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: ret_from_fork+0x35/0x40 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: Code: 75 58 48 c7 c7 a8 21 0d a4 e8 9b 7d ea ff eb 90 90 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 85 f6 74 13 8b 47 20 48 01 c6 <48> 33 36 48 33 b7 38 01 00 00 0f 18 0e f3 c3 90 66 66 66 66 90 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: RIP: prefetch_freepointer+0x10/0x20 RSP: ffffbedfc3ca7c48 Jul 04 03:54:55 openqa01.qa.fedoraproject.org kernel: ---[ end trace ddcc593e8aac0749 ]--- The share is configured on the server end with a /etc/exports.d file, and mounted on the client end with a systemd 'mount' unit.
Someone on fedoraforums ran into what looks like the same thing and filed a kernel bug: https://forums.fedoraforum.org/showthread.php?318645-houston-we-have-a-problem-! https://bugzilla.kernel.org/show_bug.cgi?id=200379
*** This bug has been marked as a duplicate of bug 1597559 ***