Created attachment 314065 [details] serial console log Description of problem: OpenVZ kernel team discovered reproduceable BUG inside NFS clinet due improper configuration of NFS server. SOme more details can be found in https://bugzilla.redhat.com/show_bug.cgi?id=458622 Version-Release number of selected component (if applicable): 2.6.18-92.1.10.el5 How reproducible: NFS server FC8, configuration: [root@tc27 proc]# cat /etc/exports /nfs_share ts27.qa.sw.ru(rw,no_root_squash,no_subtree_check,crossmnt) /nfs_share/boo *(rw,no_root_squash,no_subtree_check) [root@tc27 proc]# cat /proc/mounts /dev/mapper/vzvg-vz /nfs_share ext3 rw,data=ordered 0 0 /dev/sda1 /nfs_share/boo ext3 rw,data=ordered 0 0 Steps to Reproduce: use the following script on client side #!/bin/bash stat -f /mnt/nfs/ | grep -q 'Type: nfs' || mount -v -t nfs tc27:/nfs_share /mnt/nfs -o vers=3,tcp,hard,nointr { ls /mnt/nfs ls /mnt/nfs/boo } >/dev/null echo =========================== grep nfs /proc/mounts umount /mnt/nfs/ Actual results: "ls /mnt/nfs/boo" leads to kernel panic ----------- [cut here ] --------- [please bite here ] --------- Kernel BUG at fs/nfs/namespace.c:103 invalid opcode: 0000 [1] SMP last sysfs file: /devices/pci0000:00/0000:00:1c.1/0000:02:00.0/irq CPU 1 Modules linked in: nfs lockd fscache nfs_acl xt_length ipt_ttl xt_tcpmss ipt_TCPMSS iptable_mangle iptable_filter xt_multiport xt_limit ipt_tos ipt_REJECT ip_tables x_tables ipv6 xfrm_nalgo crypto_api autofs4 hidp rfcomm l2cap bluetooth sunrpc dm_multipath video sbs backlight i2c_ec button battery asus_acpi acpi_memhotplug ac lp sg snd_hda_intel snd_hda_codec snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq snd_seq_device snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc e1000e floppy ide_cd shpchp pcspkr i2c_i801 cdrom parport_pc serio_raw i2c_core parport dm_snapshot dm_zero dm_mirror dm_mod ata_piix libata sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd Pid: 3043, comm: ls Not tainted 2.6.18-92.1.10.el5 #1 RIP: 0010:[<ffffffff885aef20>] [<ffffffff885aef20>] :nfs:nfs_follow_mountpoint+0x2d/0x1d9 RSP: 0018:ffff8100b83ebaf8 EFLAGS: 00010246 RAX: ffff8100b792c400 RBX: ffff8100b83eeed0 RCX: 0000000000000002 RDX: 0000000000000000 RSI: ffff8100b83ebea8 RDI: ffff8100b83eeed0 RBP: ffff8100b83ebea8 R08: ffff8100b83ebb28 R09: ffff8100b83ebea8 R10: ffff8100b83ebc08 R11: 0000000000000048 R12: ffff8100b83ebea8 R13: ffff8100c4459a00 R14: 0000000000000000 R15: ffff8100b7d2e00c FS: 00002ba785ec1a50(0000) GS:ffff8100c77657c0(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b CR2: 0000003586294290 CR3: 00000000b7c58000 CR4: 00000000000006e0 Process ls (pid: 3043, threadinfo ffff8100b83ea000, task ffff8100c732a100) Stack: 0000000000000000 0000000000000002 ffff8100b83ebc08 ffffffff8011f531 ffff8100b83ebc08 ffff8100c379ac68 ffffffffffffffff ffffffff00000000 0000018b00000001 ffff8100c732a100 ffff8100b7f07a40 ffff8100b79a5420 Call Trace: [<ffffffff8011f531>] avc_has_perm+0x43/0x55 [<ffffffff8011fd84>] inode_has_perm+0x56/0x63 [<ffffffff8011f531>] avc_has_perm+0x43/0x55 [<ffffffff885a208e>] :nfs:nfs_access_get_cached+0xab/0xfa [<ffffffff801247f7>] selinux_inode_follow_link+0x5f/0x6a [<ffffffff8000a0e3>] __link_path_walk+0xb71/0xf42 [<ffffffff8000cd69>] file_read_actor+0x0/0x154 [<ffffffff8000e782>] link_path_walk+0x5c/0xe5 [<ffffffff8000c965>] do_path_lookup+0x270/0x2e8 [<ffffffff8012021f>] selinux_file_alloc_security+0x2a/0x53 [<ffffffff80023514>] __path_lookup_intent_open+0x56/0x97 [<ffffffff8001a98d>] open_namei+0x73/0x6d5 [<ffffffff800668a2>] do_page_fault+0x4fe/0x830 [<ffffffff80027363>] do_filp_open+0x1c/0x38 [<ffffffff80019759>] do_sys_open+0x44/0xbe [<ffffffff8005d28d>] tracesys+0xd5/0xe0 Code: 0f 0b 68 23 77 5c 88 c2 67 00 49 8b 3c 24 e8 f3 e0 a5 f7 8b RIP [<ffffffff885aef20>] :nfs:nfs_follow_mountpoint+0x2d/0x1d9 RSP <ffff8100b83ebaf8> <0>Kernel panic - not syncing: Fatal exception <0>Rebooting in 30 seconds.. Additional info: full serial console logs and tcpdump logs are attached please decode tcpdump.log file by using wireshark and pay attention to packets #96 and #97: packet #96: NFS client send GETATTR CALL for fsid 8001 packet #97: NFS server send GETATTR REPLY for incorrect fsid (fd00 instead 8001), that lead to BUG on clinet side
Created attachment 314066 [details] tcpdump binary logs, use wireshark to decode it
Created attachment 314067 [details] Patch from Denis V.Lunev
This request was evaluated by Red Hat Product Management for inclusion in a Red Hat Enterprise Linux maintenance release. Product Management has requested further review of this request by Red Hat Engineering, for potential inclusion in a Red Hat Enterprise Linux Update release for currently deployed products. This request is not yet committed for inclusion in an Update release.
patch posted on 10/18/08: http://post-office.corp.redhat.com/archives/rhkernel-list/2008-October/msg00492.html
in kernel-2.6.18-121.el5 You can download this test kernel from http://people.redhat.com/dzickus/el5
An advisory has been issued which should help the problem described in this bug report. This report is therefore being closed with a resolution of ERRATA. For more information on therefore solution and/or where to find the updated files, please follow the link below. You may reopen this bug report if the solution does not work for you. http://rhn.redhat.com/errata/RHSA-2009-0225.html