Bug 1474890

Summary: kernel oops - unable to handle kernel NULL pointer dereference at 0000000000000418
Product: [Fedora] Fedora Reporter: Micah Abbott <miabbott>
Component: kernelAssignee: Kernel Maintainer List <kernel-maint>
Status: CLOSED RAWHIDE QA Contact: Fedora Extras Quality Assurance <extras-qa>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: rawhideCC: dustymabe, gansalmon, ichavero, itamar, jonathan, kernel-maint, labbott, madhu.chinakonda, mchehab
Target Milestone: ---   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-08-14 15:02:07 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Micah Abbott 2017-07-25 15:04:51 UTC 
Description of problem:

After rebasing a F26 Atomic Host to Rawhide Atomic Host (Rawhide.20170719.n.0), docker fails to successfully activate and a kernel oops is observed in the journal.


Version-Release number of selected component (if applicable):

kernel-4.13.0-0.rc1.git1.1.fc27.x86_64


How reproducible:

Always

Steps to Reproduce:
1.  Boot F26 Atomic Host
2.  ostree remote add --no-gpg-verify rawhide https://kojipkgs.fedoraproject.org/atomic/rawhide/
3.  rpm-ostree rebase rawhide:fedora/rawhide/x86_64/atomic-host 0febd2c6eae093e3c2e44c0a920835103b2ea1a3177b89051a7fe7b7be82f82e
4.  Reboot
5.  Check status of 'docker' - 'systemctl is-active docker'
6.  Check for kernel oops in journal 


Actual results:

# systemctl is-active docker
activating

# journalctl -b | grep -i oops
Jul 25 15:00:45 micah-f26ah-vm0725a.localdomain kernel: Oops: 0000 [#1] SMP


Expected results:

Docker successfully activates, no kernel oops


Additional info:

Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain dockerd-current[918]: time="2017-07-24T18:54:11.732584705Z" level=info msg="Loading containers: done."
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000418
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: IP: ovl_dentry_real+0xd/0x40 [overlay]
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: PGD 752f4067 
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: P4D 752f4067 
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: PUD 75ee6067 
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: PMD 0 
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: 
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: Oops: 0000 [#1] SMP
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 xt_addrtype xt_conntrack nf_nat nf_conntr
ack br_netfilter bridge stp llc overlay kvm_intel kvm cirrus ttm irqbypass ppdev snd_pcsp drm_kms_helper snd_pcm drm snd_timer snd soundcore virtio_balloon joydev i2c_piix4 parport_pc parport xfs libcrc32c virti
o_blk virtio_net crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel serio_raw virtio_pci virtio_ring virtio ata_generic pata_acpi sunrpc scsi_transport_iscsi
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: CPU: 0 PID: 924 Comm: dockerd-current Not tainted 4.13.0-0.rc1.git1.1.fc27.x86_64 #1
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: Hardware name: Red Hat OpenStack Compute, BIOS 1.9.1-5.el7_3.2 04/01/2014
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: task: ffff9851b5c7b2c0 task.stack: ffffb57cc0e40000
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RIP: 0010:ovl_dentry_real+0xd/0x40 [overlay]
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RSP: 0018:ffffb57cc0e43900 EFLAGS: 00010282
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RAX: 0000000000000000 RBX: ffff9851b73af8c0 RCX: 00000000000000ff
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RDX: ffff9851b6810c00 RSI: ffffffffb9cdec8b RDI: ffff9851b73af8c0
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RBP: ffffb57cc0e43900 R08: ffff9851b6810c00 R09: 00000000000000ff
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: R10: ffffffffc0418110 R11: 0000000000000000 R12: ffff9851b73af8c0
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: R13: ffff9851b6810c00 R14: ffffffffb9cdec8b R15: ffff9851b6810c00
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: FS:  00007f979deca700(0000) GS:ffff9851bfc00000(0000) knlGS:0000000000000000
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: CR2: 0000000000000418 CR3: 0000000075edd000 CR4: 00000000001406f0
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: Call Trace:
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  ovl_xattr_get+0x28/0x70 [overlay]
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  ovl_other_xattr_get+0x1a/0x20 [overlay]
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  __vfs_getxattr+0x57/0x70
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  inode_doinit_with_dentry+0x3d9/0x5f0
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  selinux_d_instantiate+0x1c/0x20
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  security_d_instantiate+0x32/0x50
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  d_add+0x2d/0x1a0
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  ovl_lookup+0x2c7/0x710 [overlay]
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  lookup_slow+0x132/0x220
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  ? lookup_slow+0x132/0x220
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  walk_component+0x1bd/0x340
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  ? security_inode_permission+0x41/0x60
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  link_path_walk+0x1bc/0x5a0
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  path_openat+0xe6/0xc80
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  do_filp_open+0x9b/0x110
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  ? _raw_spin_unlock+0x27/0x40
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  ? __alloc_fd+0xf8/0x210
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  do_sys_open+0x1ba/0x250
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  ? do_sys_open+0x1ba/0x250
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  ? task_work_run+0x8e/0xb0
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  SyS_openat+0x14/0x20
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  do_syscall_64+0x6c/0x1c0
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  entry_SYSCALL64_slow_path+0x25/0x25
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel:  entry_SYSCALL64_slow_path+0x25/0x25
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RIP: 0033:0x4eb77a
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RSP: 002b:000000c4205d88f8 EFLAGS: 00000216 ORIG_RAX: 0000000000000101
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00000000004eb77a
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RDX: 0000000000080241 RSI: 000000c4201f4200 RDI: ffffffffffffff9c
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RBP: 000000c4205d8988 R08: 0000000000000000 R09: 0000000000000000
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: R10: 00000000000001a4 R11: 0000000000000216 R12: 00000000000000ce
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: R13: 00000000ffffffee R14: 0000000000000200 R15: 0000000000000080
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: Code: 10 85 d2 74 06 48 8b 40 20 5d c3 31 c0 5d c3 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 48 8b 47 68 55 48 89 e5 <48> 8b 80 18 04 00 
00 48 85 c0 75 12 48 8b 87 f8 00 00 00 8b 50 
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: RIP: ovl_dentry_real+0xd/0x40 [overlay] RSP: ffffb57cc0e43900
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: CR2: 0000000000000418
Jul 24 18:54:11 micah-f26ah-vm0724b.localdomain kernel: ---[ end trace 0ffe971a415f85ba ]---
Comment 1 Laura Abbott 2017-07-25 16:32:47 UTC 
This looks like it was fixed with https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=1d88f183734c0d916428911df006e645a6162cab , please test on the -rc2 rawhide build.
Comment 2 Micah Abbott 2017-08-14 15:02:07 UTC 
Tried today with -rc4 and no longer see the Oops.

# rpm-ostree status
State: idle
Deployments:
● rawhide:fedora/rawhide/x86_64/atomic-host
                   Version: Rawhide.20170813.n.0 (2017-08-13 13:44:24)
                    Commit: 858db1adf9eebed1447b6853e66854afcf60a5e1a0079831a48bc031f8fd68cd

# rpm -q kernel
kernel-4.13.0-0.rc4.git4.1.fc27.x86_64

# journalctl -b | grep -i oops
#