Bug 837414

Created attachment 596084 [details]
upstream git commit 20d9600cb407b0b55fef6ee814b60345c6f58264

Description of problem:

The RHEL 6.x kernel contains the following bug:

https://lkml.org/lkml/2011/1/13/420

which has been fixed upstream:

http://git.kernel.org/?p=linux/kernel/git/torvalds/linux.git;a=commitdiff;h=20d9600cb407b0b55fef6ee814b60345c6f58264

For example, fire up a VM and give it an iscsi target.  The iscsi driver
has an sg_tablesize of 2048.

Then crank up the max request size to 4 MiB, e.g.

# echo 4096 > /sys/block/sda/queue/max_sectors_kb

Then run an fio job against it which does 4 MiB direct IOs:

$ cat seq-write-4m-q32.job
[seq-write-4m-q32.job]
filename=/dev/sda
bs=4m
rw=write
ioengine=libaio
direct=1
iodepth=32
ioscheduler=noop

and boom!

BUG: unable to handle kernel NULL pointer dereference at 0000000000000010
IP: [<ffffffff811b57cf>] dio_new_bio+0x6f/0x130
PGD 3d10d067 PUD 37669067 PMD 0 
Oops: 0002 [#1] SMP 
last sysfs file: /sys/devices/platform/host2/session1/target2:0:0/2:0:0:1/block/sda/queue/scheduler
CPU 0 
Modules linked in: autofs4 sunrpc sg sd_mod crc_t10dif ipt_REJECT nf_conntrack_ipv4 nf_defrag_ipv4 iptable_filter ip_tables ip6t_REJECT nf_conntrack_ipv6 nf_defrag_ipv6 xt_state nf_conntrack ip6table_filter ip6_tables be2iscsi iscsi_boot_sysfs bnx2i cnic uio cxgb4i cxgb4 cxgb3i libcxgbi cxgb3 mdio ib_iser rdma_cm ib_cm iw_cm ib_sa ib_mad ib_core ib_addr ipv6 iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi microcode virtio_balloon snd_hda_intel snd_hda_codec snd_hwdep snd_seq snd_seq_device snd_pcm snd_timer snd soundcore snd_page_alloc virtio_net i2c_piix4 i2c_core ext4 mbcache jbd2 virtio_blk virtio_pci virtio_ring virtio pata_acpi ata_generic ata_piix dm_mirror dm_region_hash dm_log dm_mod [last unloaded: speedstep_lib]

Pid: 1745, comm: fio Not tainted 2.6.32-279.el6.x86_64 #1 Bochs Bochs
RIP: 0010:[<ffffffff811b57cf>]  [<ffffffff811b57cf>] dio_new_bio+0x6f/0x130
RSP: 0018:ffff88003df7da48  EFLAGS: 00010246
RAX: 0000000000000000 RBX: ffff880037dd7000 RCX: 0000000000000003
RDX: ffffffff811b5e40 RSI: ffff88003ef8dd40 RDI: 0000000000000286
RBP: ffff88003df7da78 R08: 0000000000000001 R09: ffff88003eb791c0
R10: 0000000000000000 R11: 0000000000000fff R12: 0000000000000000
R13: 000000000000000c R14: ffff88003eb791c0 R15: 0000000000000001
FS:  00007f2bbc526700(0000) GS:ffff880002200000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000000000010 CR3: 000000003e3bc000 CR4: 00000000000006f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process fio (pid: 1745, threadinfo ffff88003df7c000, task ffff8800379f9540)
Stack:
 00000000008c4430 ffff880037dd7000 ffffea00008d4200 0000000000001000
<d> 0000000000000000 0000000000000001 ffff88003df7da98 ffffffff811b5b0f
<d> ffff88003df7dba8 ffff880037dd7000 ffff88003df7dae8 ffffffff811b5bb1
Call Trace:
 [<ffffffff811b5b0f>] dio_send_cur_page+0x7f/0xc0
 [<ffffffff811b5bb1>] submit_page_section+0x61/0x140
 [<ffffffff811b647e>] __blockdev_direct_IO_newtrunc+0x53e/0xb90
 [<ffffffff811b6b2e>] __blockdev_direct_IO+0x5e/0xd0
 [<ffffffff811b33e0>] ? blkdev_get_blocks+0x0/0xc0
 [<ffffffff812140f2>] ? security_inode_getsecurity+0x22/0x30
 [<ffffffff8119f6fc>] ? xattr_getsecurity+0x3c/0xa0
 [<ffffffff811b4247>] blkdev_direct_IO+0x57/0x60
 [<ffffffff811b33e0>] ? blkdev_get_blocks+0x0/0xc0
 [<ffffffff81114d32>] generic_file_direct_write+0xc2/0x190
 [<ffffffff81116545>] __generic_file_aio_write+0x345/0x480
 [<ffffffff811625e0>] ? cache_alloc_refill+0x1c0/0x240
 [<ffffffff811b39dc>] blkdev_aio_write+0x3c/0xa0
 [<ffffffff811b39a0>] ? blkdev_aio_write+0x0/0xa0
 [<ffffffff811c22c4>] aio_rw_vect_retry+0x84/0x200
 [<ffffffff811c3a04>] aio_run_iocb+0x64/0x170
 [<ffffffff811c4e31>] do_io_submit+0x291/0x920
 [<ffffffff811c54d0>] sys_io_submit+0x10/0x20
 [<ffffffff8100b0f2>] system_call_fastpath+0x16/0x1b
Code: f6 44 39 f0 0f 4e f0 85 f6 0f 8e d0 00 00 00 bf d0 00 00 00 4c 8b b3 c8 00 00 00 e8 8c cd ff ff 41 8d 4d f7 48 c7 c2 40 5e 1b 81 <4c> 89 70 10 49 d3 e4 48 c7 c1 90 58 1b 81 4c 89 20 44 8b 83 48 
RIP  [<ffffffff811b57cf>] dio_new_bio+0x6f/0x130
 RSP <ffff88003df7da48>
CR2: 0000000000000010
---[ end trace de64435a49aa379c ]---
Kernel panic - not syncing: Fatal exception



Version-Release number of selected component (if applicable):

RHEL 6.x

How reproducible:

Every time.

Steps to Reproduce:
1. deploy VM
2. login to an iscsi target on the VM
3. configure the max request size for the device to 4 MiB
4. do a direct IO > 1 MiB to the device
  
Actual results:

kernel panic


Expected results:

no kernel panic

Additional info: