Bug 468192

Summary: writing data to file can fail and cause panic sometimes when using xattr on ecryptfs
Product: Red Hat Enterprise Linux 5 Reporter: Jan Tluka <jtluka>
Component: kernelAssignee: Eric Sandeen <esandeen>
Status: CLOSED ERRATA QA Contact: Martin Jenner <mjenner>
Severity: medium Docs Contact:
Priority: medium    
Version: 5.3CC: ddomingo, esandeen, syeghiay
Target Milestone: rcKeywords: Regression
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2009-01-20 20:10:59 UTC Type: ---
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
bug reproducer script none

Description Jan Tluka 2008-10-23 14:49:14 UTC 
Description of problem:
Running attached script can result in write error on mounted ecryptfs filesystem. Sometimes I get kernel panics.
The script performs specified count of loop mounts of ecryptfs filesystem using ecryptfs_xattr option.

Works fine on RHEL5.2 (1000 loops).

Version-Release number of selected component (if applicable):
kernel-2.6.18-120.el5
ecryptfs-utils-56-4.el5

How reproducible:
Not 100% but multiple runs of attached shell script should result in observed faults.

Steps to Reproduce:
Run attached script with at least 30 runs specified as first argument of the program. 
  
Actual results:

from shell script (just an example - loop number can be different):

Running loop #10 ...
Attempting to mount with the following options:
  ecryptfs_xattr_metadata
  ecryptfs_key_bytes=16
  ecryptfs_cipher=cast5
  ecryptfs_sig=a23137f90093200c
Mounted eCryptfs
./simple_xattr.sh: line 50: echo: write error: Invalid argument
/root/.secret on /root/.secret type ecryptfs (rw,ecryptfs_sig=a23137f90093200c,ecryptfs_cipher=cast5,ecryptfs_key_bytes=16,ecryptfs_xattr_metadata,ecryptfs_cipher=cast5,ecryptfs_key_bytes=16)
Test done with result: FAILED

on console I see ecryptfs related messages:

ecryptfs_read_lower: octets_read = [-13]; expected [4096]
ecryptfs_write: Error getting page at index [0] from eCryptfs inode mapping; rc = [-22]
ecryptfs_read_lower: octets_read = [-13]; expected [4096]
ecryptfs_prepare_write: Error attemping to read lower page segment; rc = [-22]


Sometimes machine panics.

general protection fault: 0000 [1] SMP 
last sysfs file: /fs/ecryptfs/version
CPU 0 
Modules linked in: ecryptfs md5 aes_generic testmgr_cipher testmgr aead crypto_blkcipher crypto_algapi aes_x86_64 autofs4 hidp rfcomm l2cap bluetooth sunrpc ipv6 xfrm_nalgo crypto_api dm_multipath scsi_dh video backlight sbs i2c_ec i2c_core button battery asus_acpi acpi_memhotplug ac parport_pc lp parport sg tg3 i5000_edac edac_mc libphy bnx2 pcspkr dm_snapshot dm_zero dm_mirror dm_log dm_mod ata_piix libata shpchp mptsas mptscsih mptbase scsi_transport_sas sd_mod scsi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
Pid: 557, comm: kauditd Tainted: G      2.6.18-120.el5 #1
RIP: 0010:[<ffffffff80235a72>]  [<ffffffff80235a72>] netlink_unicast+0x112/0x1d9
RSP: 0018:ffff81003e863e60  EFLAGS: 00010286
RAX: b45bb691789e5a60 RBX: ffff810037fbe2d0 RCX: 000000006fe3367c
RDX: 00000000778332f2 RSI: 00000000ba11d45b RDI: ffff810037fbe2d0
RBP: ffff81002f04a7c0 R08: 00000000799e35d8 R09: 0000000000000000
R10: 0000000000000000 R11: ffff81002f04a7c0 R12: 7fffffffffffffff
R13: 000000000000030b R14: ffff81003f196400 R15: 0000000000000000
FS:  0000000000000000(0000) GS:ffffffff803b7000(0000) knlGS:0000000000000000
CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
CR2: 00000000080c3140 CR3: 000000002fba7000 CR4: 00000000000006e0
Process kauditd (pid: 557, threadinfo ffff81003e862000, task ffff81003f841820)
Stack:  000009623e863ea0 ffff81002f04a7c0 0000000000000000 ffff81003f7979c8
 0000000000000282 ffff81003f7979b8 ffffffff8009d6d5 ffffffff800b003a
 0000000000000000 ffff81003f841820 ffffffff8008a22e 0000000000100100
Call Trace:
 [<ffffffff8009d6d5>] keventd_create_kthread+0x0/0xc4
 [<ffffffff800b003a>] kauditd_thread+0x5b/0x17b
 [<ffffffff8008a22e>] default_wake_function+0x0/0xe
 [<ffffffff800affdf>] kauditd_thread+0x0/0x17b
 [<ffffffff800324bd>] kthread+0xfe/0x132
 [<ffffffff8009d6d5>] keventd_create_kthread+0x0/0xc4
 [<ffffffff8005dfb1>] child_rip+0xa/0x11
 [<ffffffff8009d6d5>] keventd_create_kthread+0x0/0xc4
 [<ffffffff800323bf>] kthread+0x0/0x132
 [<ffffffff8005dfa7>] child_rip+0x0/0x11


Code: 48 8b 10 0f 18 0a 48 8d 58 f8 8b 44 24 04 39 83 28 02 00 00 
RIP  [<ffffffff80235a72>] netlink_unicast+0x112/0x1d9
 RSP <ffff81003e863e60>
 <0>Kernel panic - not syncing: Fatal exception

Expected results:
No panics, no write errors. At least 1000 loops without any problem.

Additional info:
Patch from Eric Sandeen available, tested patched module on kernel-2.6.18-120.el5 and bug did not occur. 1000 mount loops without errors.
Comment 1 Eric Sandeen 2008-10-23 14:56:13 UTC 
Thanks Jan; the patch was recently sent upstream & acked by the ecryptfs maintainer.  I'll see if this makes sense to get this into RHEL5.3, yet.

It's a bad bug, but only if this (somewhat obscure) mount option is used; it allocates 0 bytes, and then writes into that memory location.  :(

-Eric
Comment 2 Jan Tluka 2008-10-23 15:00:43 UTC 
Created attachment 321298 [details]
bug reproducer script

Reproducer shell script.
Comment 3 RHEL Program Management 2008-10-23 15:09:31 UTC 
This bugzilla has Keywords: Regression.  

Since no regressions are allowed between releases, 
it is also being proposed as a blocker for this release.  

Please resolve ASAP.
Comment 4 RHEL Program Management 2008-10-23 15:16:46 UTC 
This request was evaluated by Red Hat Product Management for inclusion in a Red
Hat Enterprise Linux maintenance release.  Product Management has requested
further review of this request by Red Hat Engineering, for potential
inclusion in a Red Hat Enterprise Linux Update release for currently deployed
products.  This request is not yet committed for inclusion in an Update
release.
Comment 6 Eric Sandeen 2008-10-28 20:05:51 UTC 
Release note added. If any revisions are required, please set the 
"requires_release_notes" flag to "?" and edit the "Release Notes" field accordingly.
All revisions will be proofread by the Engineering Content Services team.

New Contents:
Storing cryptographic metadata in file extended attributes causes memory corruption.

(until this bugfix is released, that is)
Comment 8 Don Zickus 2008-11-04 16:51:20 UTC 
in kernel-2.6.18-122.el5
You can download this test kernel from http://people.redhat.com/dzickus/el5
Comment 12 Eric Sandeen 2008-11-24 04:10:23 UTC 
Don, I think we can remove the release notes, thanks for catching that.

-Eric
Comment 13 Eric Sandeen 2008-11-24 04:10:24 UTC 
Deleted Release Notes Contents.

Old Contents:
Storing cryptographic metadata in file extended attributes causes memory corruption.

(until this bugfix is released, that is)
Comment 15 errata-xmlrpc 2009-01-20 20:10:59 UTC 
An advisory has been issued which should help the problem
described in this bug report. This report is therefore being
closed with a resolution of ERRATA. For more information
on therefore solution and/or where to find the updated files,
please follow the link below. You may reopen this bug report
if the solution does not work for you.

http://rhn.redhat.com/errata/RHSA-2009-0225.html