Bug 1031451

Summary: RHEL6.5-32 guest can not poweroff/restart after hot-unplugging disk
Product: Red Hat Enterprise Linux 7 Reporter: Jun Li <juli>
Component: qemu-kvmAssignee: Paolo Bonzini <pbonzini>
Status: CLOSED CURRENTRELEASE QA Contact: Suqin Huang <shuang>
Severity: high Docs Contact:
Priority: medium    
Version: 7.0CC: aliang, coli, hhuang, juzhang, knoel, meyang, michen, ngu, pbonzini, pingl, rbalakri, virt-bugs, virt-maint, xfu, xuwei
Target Milestone: rcKeywords: Reopened
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2017-06-14 20:14:01 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:

Description Jun Li 2013-11-18 02:27:34 UTC
Description of problem:
Boot rhel6.5-32 guest with libiscsi data disk. Guest can not poweroff/restart/ping after hot-unplug this libiscsi disk. 

Version-Release number of selected component (if applicable):
libiscsi-1.9.0-3.el7.x86_64
qemu-kvm-rhev-1.5.3-19.el7.x86_64
Guest os kernel: 2.6.32-430.el6.i686
Host os kernel: 3.10.0-48.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
1.Boot guest with a libiscsi data disk.
<cli>:
# /usr/libexec/qemu-kvm -S -M pc-i440fx-rhel7.0.0 -cpu SandyBridge -enable-kvm -m 4G -smp 4,sockets=2,cores=2,threads=1 -name juli -uuid 355a2475-4e03-4cdd-bf7b-5d6a59edaa68 -rtc base=localtime,clock=host,driftfix=slew \
-device virtio-scsi-pci,bus=pci.0,addr=0x5,id=scsi0 -drive file=iscsi://10.66.6.82:3260/iqn.2013-11.com.example:storage.disk1.juli.xyz/1,if=none,id=drive-scsi0-0-0,media=disk,cache=none,format=qcow2,werror=stop,rerror=stop,aio=native  -device scsi-hd,drive=drive-scsi0-0-0,bus=scsi0.0,scsi-id=0,lun=0,id=juli,bootindex=0 \
-drive file=iscsi://10.66.6.82:3260/iqn.2013-11.com.example:storage.disk1.juli.xyz/3,if=none,id=drive-virtio0-0-0,media=disk,werror=stop,rerror=stop,cache=none,format=qcow2 -device virtio-blk-pci,bus=pci.0,drive=drive-virtio0-0-0,id=virtio0-0-0 \
-drive file=/home/ISO/en_windows_8.1_preview_x86_dvd_2358833.iso,if=none,media=cdrom,format=raw,aio=native,id=drive-ide1-0-0 -device ide-drive,drive=drive-ide1-0-0,id=ide1-0-0,bus=ide.0,unit=0,bootindex=4 -device virtio-balloon-pci,id=ballooning -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -net none -k en-us -boot menu=on,reboot-timeout=-1,strict=on -qmp tcp:0:4477,server,nowait -serial unix:/tmp/ttyS0,server,nowait -vnc :3 -spice port=5939,disable-ticketing  -vga qxl -global qxl-vga.revision=3 -monitor stdio -monitor tcp:0:7777,server,nowait -monitor unix:/tmp/monitor1,server,nowait \
-netdev tap,id=tap1,vhost=on,queues=4,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown,ifname=tap-juli -device virtio-net-pci,netdev=tap1,id=nic1,mq=on,vectors=17,mac=1a:59:0a:4b:aa:94
2.dd large file into the libiscsi data disk inside guest.
# dd if=/dev/urandom of=/mnt/aa bs=1M count=2048
3.hot-unplug libiscsi data disk via HMP.
(qemu) device_del virtio0-0-0
4.Reboot guest.
#reboot

Actual results:
# reboot
-bash: /sbin/reboot: Input/output error
#dmesg
...
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
__ratelimit: 11269 callbacks suppressed
Buffer I/O error on device vda1, logical block 1
lost page write due to I/O error on vda1
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
Buffer I/O error on device vda1, logical block 1
lost page write due to I/O error on vda1
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
Buffer I/O error on device vda1, logical block 1
lost page write due to I/O error on vda1
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
Buffer I/O error on device vda1, logical block 1
lost page write due to I/O error on vda1
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
Buffer I/O error on device vda1, logical block 1
lost page write due to I/O error on vda1
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
Buffer I/O error on device vda1, logical block 1
lost page write due to I/O error on vda1
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
Buffer I/O error on device vda1, logical block 1
lost page write due to I/O error on vda1
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
Buffer I/O error on device vda1, logical block 1
lost page write due to I/O error on vda1
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
Buffer I/O error on device vda1, logical block 1
lost page write due to I/O error on vda1
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
Buffer I/O error on device vda1, logical block 1
lost page write due to I/O error on vda1
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): __ext4_get_inode_loc: unable to read inode block - inode=19, block=294
EXT4-fs (vda1): previous I/O error to superblock detected
Aborting journal on device vda1-8.
JBD2: I/O error detected when updating journal superblock for vda1-8.

EXT4-fs error (device vda1) in ext4_reserve_inode_write: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_dirty_inode: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1) in ext4_da_write_end: IO failure
EXT4-fs (vda1): previous I/O error to superblock detected
EXT4-fs error (device vda1): ext4_journal_start_sb: Detected aborted journal
EXT4-fs (vda1): Remounting filesystem read-only
EXT4-fs error (device vda1): ext4_find_entry: reading directory #2 offset 0
EXT4-fs error (device vda1): ext4_find_entry: reading directory #2 offset 0
EXT4-fs error (device vda1): ext4_find_entry: reading directory #2 offset 0
Core dump to |/usr/libexec/abrt-hook-ccpp 7 0 2177 42 42 1384769963 e pipe failed
Core dump to |/usr/libexec/abrt-hook-ccpp 7 0 1989 42 42 1384769963 e pipe failed
EXT4-fs error (device vda1): ext4_find_entry: reading directory #2 offset 0
EXT4-fs error (device vda1): ext4_find_entry: reading directory #2 offset 0


Expected results:
After hot-unplug libiscsi data disk, the guest could works well, no error.

Additional info:
Also test with local data disk, no this issue.
<cli> ... -drive file=/root/test.img,if=none,id=drive-virtio0-0-0,media=disk,werror=stop,rerror=stop,cache=none,format=qcow2 -device virtio-blk-pci,bus=pci.0,drive=drive-virtio0-0-0,id=virtio0-0-0  ...

Comment 1 Jun Li 2013-11-18 02:44:14 UTC
Hot-unplug libiscsi data disk during live commit also hit this issue.
Do live commit via QMP as following steps:
{ "execute": "blockdev-snapshot-sync", "arguments": { "device": "drive-virtio0-0-0", "snapshot-file": "/root/sn1", "format": "qcow2" } }

{ "execute": "blockdev-snapshot-sync", "arguments": { "device": "drive-virtio0-0-0", "snapshot-file": "/root/sn2", "format": "qcow2" } }

{ "execute": "blockdev-snapshot-sync", "arguments": { "device": "drive-virtio0-0-0", "snapshot-file": "/root/sn3", "format": "qcow2" } }   

{ "execute": "blockdev-snapshot-sync", "arguments": { "device": "drive-virtio0-0-0", "snapshot-file": "/root/sn4", "format": "qcow2" } }

{ "execute": "block-commit", "arguments": { "device": "drive-virtio0-0-0", "base": "/root/sn1", "top": "/root/sn3", "speed": 1000000000 } }

Comment 3 Jun Li 2013-11-18 02:55:15 UTC
Hot-unplug libiscsi data disk during image streaming hit this issue again.
Steps as followings(boot guest with cli the same as comment 0):
(qemu) snapshot_blkdev drive-virtio0-0-0 /home/sn1 qcow2
(qemu) block_stream drive-virtio0-0-0
(qemu) device_del virtio0-0-0
-----
After above operation, can not do reboot/poweroff inside guest:
# reboot 
-bash: /sbin/reboot: Input/output error
# poweroff 
-bash: /sbin/poweroff: Input/output error
# dmesg
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ5 used by override.
ACPI: IRQ9 used by override.
ACPI: IRQ10 used by override.
ACPI: IRQ11 used by override.
Using ACPI (MADT) for SMP configuration information
SMP: Allowing 4 CPUs, 0 hotplug CPUs
nr_irqs_gsi: 24
PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
Allocating PCI resources starting at e0000000 (gap: e0000000:1effc000)
Booting paravirtualized kernel on KVM
NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:4 nr_node_ids:1
PERCPU: Embedded 14 pages/cpu @c1e00000 s36056 r0 d21288 u524288
pcpu-alloc: s36056 r0 d21288 u524288 alloc=1*2097152
pcpu-alloc: [0] 0 1 2 3 
kvm-clock: cpu 0, msr 0:1e086c1, primary cpu clock
kvm-stealtime: cpu 0, msr 1e03800
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1039259
Kernel command line: ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=128M rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
PID hash table entries: 4096 (order: 2, 16384 bytes)
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
Initializing HighMem for node 0 (000375fe:00120000)
Memory: 4002996k/4718592k available (4504k kernel code, 189736k reserved, 2561k data, 524k init, 3287040k highmem)
virtual kernel memory layout:
    fixmap  : 0xffad5000 - 0xfffff000   (5288 kB)
    pkmap   : 0xff600000 - 0xff800000   (2048 kB)
    vmalloc : 0xf7dfe000 - 0xff5fe000   ( 120 MB)
    lowmem  : 0xc0000000 - 0xf75fe000   ( 885 MB)
      .init : 0xc0ae7000 - 0xc0b6a000   ( 524 kB)
      .data : 0xc0866238 - 0xc0ae6808   (2561 kB)
      .text : 0xc0400000 - 0xc0866238   (4504 kB)
Checking if this processor honours the WP bit even in supervisor mode...Ok.
Hierarchical RCU implementation.
NR_IRQS:2304 nr_irqs:440
Console: colour VGA+ 80x25
console [tty0] enabled
allocated 8388608 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Detected 2000.040 MHz processor.
Marking TSC unstable due to TSCs unsynchronized
Calibrating delay loop (skipped) preset value.. 4000.08 BogoMIPS (lpj=2000040)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
Mount-cache hash table entries: 512
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
Initializing cgroup subsys perf_event
Initializing cgroup subsys net_prio
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 10 MCE banks
alternatives: switching to unfair spinlock
ACPI: Core revision 20090903
Enabling APIC mode:  Flat.  Using 1 I/O APICs
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: AMD Intel Xeon E312xx (Sandy Bridge) stepping 01
Performance Events: Broken PMU hardware detected, using software events only.
Booting Node   0, Processors  #1
Initializing CPU#1
kvm-clock: cpu 1, msr 0:1e886c1, secondary cpu clock
------------[ cut here ]------------
WARNING: at arch/x86/kernel/cpu/amd.c:187 init_amd+0x388/0x641() (Not tainted)
Hardware name: KVM
WARNING: This combination of AMD processors is not suitable for SMP.
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.32-430.el6.i686 #1
Call Trace:
 [<c045b0d1>] ? warn_slowpath_common+0x81/0xc0
 [<c085585d>] ? init_amd+0x388/0x641
 [<c085585d>] ? init_amd+0x388/0x641
 [<c045b1a3>] ? warn_slowpath_fmt+0x33/0x40
 [<c085585d>] ? init_amd+0x388/0x641
 [<c0854bda>] ? identify_cpu+0x28b/0x42d
 [<c042d65d>] ? default_init_apic_ldr+0x4d/0x50
 [<c08599a0>] ? setup_local_APIC+0x337/0x355
 [<c0854d8c>] ? identify_secondary_cpu+0x10/0x1a
 [<c0858e43>] ? start_secondary+0x138/0x2f4
---[ end trace a7919e7f17c0a725 ]---
Disabling lock debugging due to kernel taint
kvm-stealtime: cpu 1, msr 1e83800
 #2
Initializing CPU#2
kvm-clock: cpu 2, msr 0:1f086c1, secondary cpu clock
 #3 Ok.
kvm-stealtime: cpu 2, msr 1f03800
Initializing CPU#3
kvm-clock: cpu 3, msr 0:1f886c1, secondary cpu clock
Brought up 4 CPUs
Total of 4 processors activated (16000.32 BogoMIPS).
kvm-stealtime: cpu 3, msr 1f83800
sizeof(vma)=100 bytes
sizeof(page)=32 bytes
sizeof(inode)=352 bytes
sizeof(dentry)=132 bytes
sizeof(ext3inode)=508 bytes
sizeof(buffer_head)=56 bytes
sizeof(skbuff)=184 bytes
sizeof(task_struct)=1348 bytes
devtmpfs: initialized
regulator: core version 0.5
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfc670, last bus=0
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S3 S4 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7]
pci_root PNP0A03:00: host bridge window [io  0x0d00-0xffff]
pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff]
pci_root PNP0A03:00: host bridge window [mem 0xe0000000-0xfebfffff]
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: root bus resource [mem 0xe0000000-0xfebfffff]
pci 0000:00:01.1: reg 20: [io  0xc100-0xc10f]
pci 0000:00:01.3: quirk: [io  0xb000-0xb03f] claimed by PIIX4 ACPI
pci 0000:00:01.3: quirk: [io  0xb100-0xb10f] claimed by PIIX4 SMB
pci 0000:00:02.0: reg 10: [mem 0xf4000000-0xf7ffffff]
pci 0000:00:02.0: reg 14: [mem 0xf8000000-0xfbffffff]
pci 0000:00:02.0: reg 18: [mem 0xfc030000-0xfc031fff]
pci 0000:00:02.0: reg 1c: [io  0xc0c0-0xc0df]
pci 0000:00:02.0: reg 30: [mem 0xfc020000-0xfc02ffff pref]
pci 0000:00:03.0: reg 10: [io  0xc000-0xc03f]
pci 0000:00:03.0: reg 14: [mem 0xfc032000-0xfc032fff]
pci 0000:00:04.0: reg 10: [io  0xc0e0-0xc0ff]
pci 0000:00:05.0: reg 10: [io  0xc040-0xc07f]
pci 0000:00:05.0: reg 14: [mem 0xfc033000-0xfc033fff]
pci 0000:00:06.0: reg 10: [io  0xc080-0xc0bf]
pci 0000:00:06.0: reg 14: [mem 0xfc034000-0xfc034fff]
pci 0000:00:06.0: reg 30: [mem 0xfc000000-0xfc01ffff pref]
pci_bus 0000:00: on NUMA node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0000:00:02.0
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
PCI: pci_cache_line_size set to 64 bytes
lo: Dropping TSO features since no CSUM feature.
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
Switching to clocksource kvm-clock
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp 00:00: [io  0x0cf8-0x0cff]
pnp 00:00: Plug and Play ACPI device, IDs PNP0a03 (active)
pnp 00:01: [io  0x0070-0x0071]
pnp 00:01: [irq 8]
pnp 00:01: [io  0x0072-0x0077]
pnp 00:01: Plug and Play ACPI device, IDs PNP0b00 (active)
pnp 00:02: [io  0x0060]
pnp 00:02: [io  0x0064]
pnp 00:02: [irq 1]
pnp 00:02: Plug and Play ACPI device, IDs PNP0303 (active)
pnp 00:03: [irq 12]
pnp 00:03: Plug and Play ACPI device, IDs PNP0f13 (active)
pnp 00:04: [io  0x03f2-0x03f5]
pnp 00:04: [io  0x03f7]
pnp 00:04: [irq 6]
pnp 00:04: [dma 2]
pnp 00:04: Plug and Play ACPI device, IDs PNP0700 (active)
pnp 00:05: [io  0x03f8-0x03ff]
pnp 00:05: [irq 4]
pnp 00:05: Plug and Play ACPI device, IDs PNP0501 (active)
pnp: PnP ACPI: found 6 devices
ACPI: ACPI bus type pnp unregistered
PCI: max bus depth: 0 pci_try_num: 1
pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: resource 7 [mem 0xe0000000-0xfebfffff]
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
NET: Registered protocol family 1
pci 0000:00:00.0: Limiting direct PCI/PCI transfers
pci 0000:00:01.0: PIIX3: Enabling Passive Release
pci 0000:00:01.0: Activating ISA DMA hang workarounds
pci 0000:00:02.0: Boot video device
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 15837k freed
apm: BIOS not found.
audit: initializing netlink socket (disabled)
type=2000 audit(1384742705.604:1): initialized
highmem bounce pool size: 64 pages
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
msgmni has been set to 1431
SELinux:  Registering netfilter hooks
alg: No test for stdrng (krng)
ksign: Installing public key data
Loading keyring
- Added public key 6D8C1D71AB3678E6
- User ID: Red Hat, Inc. (Kernel Module GPG key)
- Added public key D4A26C9CCD09BEDA
- User ID: Red Hat Enterprise Linux Driver Update Program <secalert>
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
acpiphp: Slot [3] registered
acpiphp: Slot [4] registered
acpiphp: Slot [5] registered
acpiphp: Slot [6] registered
acpiphp: Slot [7] registered
acpiphp: Slot [8] registered
acpiphp: Slot [9] registered
acpiphp: Slot [10] registered
acpiphp: Slot [11] registered
acpiphp: Slot [12] registered
acpiphp: Slot [13] registered
acpiphp: Slot [14] registered
acpiphp: Slot [15] registered
acpiphp: Slot [16] registered
acpiphp: Slot [17] registered
acpiphp: Slot [18] registered
acpiphp: Slot [19] registered
acpiphp: Slot [20] registered
acpiphp: Slot [21] registered
acpiphp: Slot [22] registered
acpiphp: Slot [23] registered
acpiphp: Slot [24] registered
acpiphp: Slot [25] registered
acpiphp: Slot [26] registered
acpiphp: Slot [27] registered
acpiphp: Slot [28] registered
acpiphp: Slot [29] registered
acpiphp: Slot [30] registered
acpiphp: Slot [31] registered
ipmi message handler version 39.2
IPMI System Interface driver.
ipmi_si: Adding default-specified kcs state machine
ipmi_si: Trying default-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Adding default-specified smic state machine
ipmi_si: Trying default-specified smic state machine at i/o address 0xca9, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Adding default-specified bt state machine
ipmi_si: Trying default-specified bt state machine at i/o address 0xe4, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Unable to find any System Interface(s)
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
ACPI: Power Button [PWRF]
ACPI: acpi_idle registered with cpuidle
ACPI Exception: AE_NOT_FOUND, No or invalid critical threshold (20090903/thermal-386)
GHES: HEST is not enabled!
isapnp: Scanning for PnP cards...
isapnp: No Plug & Play device found
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
crash memory driver: version 1.1
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
brd: module loaded
loop: module loaded
input: Macintosh mouse button emulation as /devices/virtual/input/input1
Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
rtc_cmos 00:01: RTC can wake from S4
rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one day, 114 bytes nvram
cpuidle: using governor ladder
cpuidle: using governor menu
EFI Variables Facility v0.08 2004-May-17
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
GRE over IPv4 demultiplexor driver
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Using IPI No-Shortcut mode
registered taskstats version 1
input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
rtc_cmos 00:01: setting system clock to 2013-11-18 10:45:06 UTC (1384771506)
Initalizing network drop monitor service
Freeing unused kernel memory: 524k freed
Write protecting the kernel text: 4508k
Write protecting the kernel read-only data: 1980k
dracut: dracut-004-335.el6
dracut: rd_NO_LUKS: removing cryptoluks activation
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.24.6-ioctl (2013-01-15) initialised: dm-devel
udev: starting version 147
psmouse serio1: ID: 10 00 64
dracut: Starting plymouth daemon
dracut: rd_NO_DM: removing DM RAID activation
dracut: rd_NO_MD: removing MD RAID activation
ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
virtio-pci 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 11 (level, high) -> IRQ 11
ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 10
virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 10 (level, high) -> IRQ 10
ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
virtio-pci 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 11
virtio-pci 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 11 (level, high) -> IRQ 11
ata_piix 0000:00:01.1: version 2.13
ata_piix 0000:00:01.1: setting latency timer to 64
scsi0 : ata_piix
scsi1 : ata_piix
ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc100 irq 14
ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc108 irq 15
ata1.01: NODEV after polling detection
ata1.00: ATAPI: QEMU DVD-ROM, 1.5.3, max UDMA/100
ata1.00: configured for MWDMA2
scsi 0:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     1.5. PQ: 0 ANSI: 5
input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input3
  alloc irq_desc for 24 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:03.0: irq 24 for MSI/MSI-X
  alloc irq_desc for 25 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:03.0: irq 25 for MSI/MSI-X
 vda: vda1 vda2
  alloc irq_desc for 26 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 26 for MSI/MSI-X
  alloc irq_desc for 27 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 27 for MSI/MSI-X
  alloc irq_desc for 28 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 28 for MSI/MSI-X
  alloc irq_desc for 29 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 29 for MSI/MSI-X
scsi2 : Virtio SCSI HBA
scsi 2:0:0:0: Direct-Access     QEMU     QEMU HARDDISK    1.5. PQ: 0 ANSI: 5
sr0: scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
Uniform CD-ROM driver Revision: 3.20
sr 0:0:0:0: Attached scsi CD-ROM sr0
sd 2:0:0:0: [sda] 62914560 512-byte logical blocks: (32.2 GB/30.0 GiB)
sd 2:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sda] Mode Sense: 63 00 00 08
sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2
sd 2:0:0:0: [sda] Attached SCSI disk
dracut: Scanning devices sda2 vda2  for LVM logical volumes vg_unused/lv_swap vg_unused/lv_root 
dracut: Found duplicate PV KR0iAw7ZIxbzalquR0vjeyAEWGK3uRGh: using /dev/vda2 not /dev/sda2
dracut: inactive '/dev/vg_unused/lv_root' [26.51 GiB] inherit
dracut: inactive '/dev/vg_unused/lv_swap' [3.00 GiB] inherit
dracut: Found duplicate PV KR0iAw7ZIxbzalquR0vjeyAEWGK3uRGh: using /dev/vda2 not /dev/sda2
EXT4-fs (dm-0): INFO: recovery required on readonly filesystem
EXT4-fs (dm-0): write access will be enabled during recovery
EXT4-fs (dm-0): recovery complete
EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: 
dracut: Mounted root filesystem /dev/mapper/vg_unused-lv_root
dracut: Loading SELinux policy
type=1404 audit(1384771508.444:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295
SELinux: 2048 avtab hash slots, 277800 rules.
SELinux: 2048 avtab hash slots, 277800 rules.
SELinux:  9 users, 12 roles, 3917 types, 217 bools, 1 sens, 1024 cats
SELinux:  81 classes, 277800 rules
SELinux:  Completing initialization.
SELinux:  Setting up existing superblocks.
SELinux: initialized (dev dm-0, type ext4), uses xattr
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev selinuxfs, type selinuxfs), uses genfs_contexts
SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs
SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses transition SIDs
SELinux: initialized (dev devpts, type devpts), uses transition SIDs
SELinux: initialized (dev inotifyfs, type inotifyfs), uses genfs_contexts
SELinux: initialized (dev anon_inodefs, type anon_inodefs), uses genfs_contexts
SELinux: initialized (dev pipefs, type pipefs), uses task SIDs
SELinux: initialized (dev debugfs, type debugfs), uses genfs_contexts
SELinux: initialized (dev sockfs, type sockfs), uses task SIDs
SELinux: initialized (dev devtmpfs, type devtmpfs), uses transition SIDs
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev proc, type proc), uses genfs_contexts
SELinux: initialized (dev bdev, type bdev), uses genfs_contexts
SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts
SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
type=1403 audit(1384771509.017:3): policy loaded auid=4294967295 ses=4294967295
dracut: 
dracut: Switching root
SELinux: initialized (dev usbfs, type usbfs), uses genfs_contexts
udev: starting version 147
piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0
  alloc irq_desc for 30 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:06.0: irq 30 for MSI/MSI-X
  alloc irq_desc for 31 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:06.0: irq 31 for MSI/MSI-X
  alloc irq_desc for 32 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:06.0: irq 32 for MSI/MSI-X
sr 0:0:0:0: Attached scsi generic sg0 type 5
sd 2:0:0:0: Attached scsi generic sg1 type 0
platform microcode: firmware: requesting amd-ucode/microcode_amd.bin
microcode: CPU0: AMD CPU family 0x6 not supported
microcode: CPU1: AMD CPU family 0x6 not supported
microcode: CPU2: AMD CPU family 0x6 not supported
microcode: CPU3: AMD CPU family 0x6 not supported
Microcode Update Driver: v2.00 <tigran.co.uk>, Peter Oruba
EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: 
SELinux: initialized (dev sda1, type ext4), uses xattr
Adding 3145720k swap on /dev/mapper/vg_unused-lv_swap.  Priority:-1 extents:1 across:3145720k 
SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses genfs_contexts
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
ip6_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
ip_tables: (C) 2000-2006 Netfilter Core Team
802.1Q VLAN Support v1.8 Ben Greear <greearb>
All bugs added by David S. Miller <davem>
8021q: adding VLAN 0 to HW filter on device eth0
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
eth0: no IPv6 routers present
virtio-pci 0000:00:03.0: PCI INT A disabled
Buffer I/O error on device dm-0, logical block 1617943
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1617983
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 61873
lost page write due to I/O error on dm-0
JBD2: Detected IO errors while flushing file data on dm-0-8
Aborting journal on device dm-0-8.
Buffer I/O error on device dm-0, logical block 3178496
lost page write due to I/O error on dm-0
JBD2: I/O error detected when updating journal superblock for dm-0-8.
EXT4-fs error (device dm-0): ext4_journal_start_sb: Detected aborted journal
EXT4-fs (dm-0): Remounting filesystem read-only
Buffer I/O error on device dm-0, logical block 1362432
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1027
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048577
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048585
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048590
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048645
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048646
lost page write due to I/O error on dm-0
Core dump to |/usr/libexec/abrt-hook-ccpp 7 0 2150 42 42 1384771744 e pipe failed
Core dump to |/usr/libexec/abrt-hook-ccpp 7 0 1993 42 42 1384771744 e pipe failed

Comment 4 Paolo Bonzini 2013-11-18 16:26:44 UTC
Before hot-unplug, make sure that the disk is unmounted in the guest.

Comment 5 Jun Li 2013-11-19 02:56:01 UTC
(In reply to Paolo Bonzini from comment #4)
> Before hot-unplug, make sure that the disk is unmounted in the guest.

retest this issue, using comment 0 cli. umount this data disk before hot-unplug,  hit this issue still. Steps as followings:
1.Boot guest with a libiscsi data disk.
<cli>:
# /usr/libexec/qemu-kvm -S -M pc-i440fx-rhel7.0.0 -cpu SandyBridge -enable-kvm -m 4G -smp 4,sockets=2,cores=2,threads=1 -name juli -uuid 355a2475-4e03-4cdd-bf7b-5d6a59edaa68 -rtc base=localtime,clock=host,driftfix=slew \
-device virtio-scsi-pci,bus=pci.0,addr=0x5,id=scsi0 -drive file=iscsi://10.66.6.82:3260/iqn.2013-11.com.example:storage.disk1.juli.xyz/1,if=none,id=drive-scsi0-0-0,media=disk,cache=none,format=qcow2,werror=stop,rerror=stop,aio=native  -device scsi-hd,drive=drive-scsi0-0-0,bus=scsi0.0,scsi-id=0,lun=0,id=juli,bootindex=0 \
-drive file=iscsi://10.66.6.82:3260/iqn.2013-11.com.example:storage.disk1.juli.xyz/3,if=none,id=drive-virtio0-0-0,media=disk,werror=stop,rerror=stop,cache=none,format=qcow2 -device virtio-blk-pci,bus=pci.0,drive=drive-virtio0-0-0,id=virtio0-0-0 \
-drive file=/home/ISO/en_windows_8.1_preview_x86_dvd_2358833.iso,if=none,media=cdrom,format=raw,aio=native,id=drive-ide1-0-0 -device ide-drive,drive=drive-ide1-0-0,id=ide1-0-0,bus=ide.0,unit=0,bootindex=4 -device virtio-balloon-pci,id=ballooning -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -net none -k en-us -boot menu=on,reboot-timeout=-1,strict=on -qmp tcp:0:4477,server,nowait -serial unix:/tmp/ttyS0,server,nowait -vnc :3 -spice port=5939,disable-ticketing  -vga qxl -global qxl-vga.revision=3 -monitor stdio -monitor tcp:0:7777,server,nowait -monitor unix:/tmp/monitor1,server,nowait \
-netdev tap,id=tap1,vhost=on,queues=4,script=/etc/qemu-ifup,downscript=/etc/qemu-ifdown,ifname=tap-juli -device virtio-net-pci,netdev=tap1,id=nic1,mq=on,vectors=17,mac=1a:59:0a:4b:aa:94
2.umount virtio-blk data disk.
# umount /dev/vda
# umount /dev/vda1
# umount /dev/vda2
3.hot-unplug libiscsi data disk via HMP.
(qemu) device_del virtio0-0-0
4.Reboot guest.
#reboot

Actual results:
# reboot
-bash: /sbin/reboot: Input/output error
# poweroff 
-bash: /sbin/poweroff: Input/output error

#dmesg
[root@unused ~]# dmesg 
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.32-430.el6.i686 (mockbuild.eng.bos.redhat.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Tue Nov 5 15:07:44 EST 2013
KERNEL supported cpus:
  Intel GenuineIntel
  AMD AuthenticAMD
  NSC Geode by NSC
  Cyrix CyrixInstead
  Centaur CentaurHauls
  Transmeta GenuineTMx86
  Transmeta TransmetaCPU
  UMC UMC UMC UMC
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000dfffe000 (usable)
 BIOS-e820: 00000000dfffe000 - 00000000e0000000 (reserved)
 BIOS-e820: 00000000feffc000 - 00000000ff000000 (reserved)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000120000000 (usable)
DMI 2.4 present.
SMBIOS version 2.4 @ 0xFDF10
DMI: Red Hat KVM, BIOS Bochs 01/01/2011
Hypervisor detected: KVM
e820 update range: 0000000000000000 - 0000000000001000 (usable) ==> (reserved)
e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
last_pfn = 0x120000 max_arch_pfn = 0x400000
MTRR default type: write-back
MTRR fixed ranges enabled:
  00000-9FFFF write-back
  A0000-BFFFF uncachable
  C0000-FFFFF write-protect
MTRR variable ranges enabled:
  0 base 00E0000000 mask FFE0000000 uncachable
  1 disabled
  2 disabled
  3 disabled
  4 disabled
  5 disabled
  6 disabled
  7 disabled
x86 PAT enabled: cpu 0, old 0x70406, new 0x7010600070106
initial memory mapped : 0 - 01000000
init_memory_mapping: 0000000000000000-00000000375fe000
NX (Execute Disable) protection: active
 0000000000 - 0000200000 page 4k
 0000200000 - 0037400000 page 2M
 0037400000 - 00375fe000 page 4k
kernel direct mapping tables up to 375fe000 @ 7000-f000
RAMDISK: 37078000 - 37fef43f
Allocated new RAMDISK: 00ca2000 - 01c1943f
Move RAMDISK from 0000000037078000 - 0000000037fef43e to 00ca2000 - 01c1943e
ACPI: RSDP 000fdce0 00014 (v00 BOCHS )
ACPI: RSDT dfffe310 00030 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
ACPI: FACP dfffff80 00074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
ACPI: DSDT dfffe340 011A9 (v01   BXPC   BXDSDT 00000001 INTL 20130823)
ACPI: FACS dfffff40 00040
ACPI: SSDT dffff620 00919 (v01 BOCHS  BXPCSSDT 00000001 BXPC 00000001)
ACPI: APIC dffff4f0 00090 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
ACPI: Local APIC address 0xfee00000
3722MB HIGHMEM available.
885MB LOWMEM available.
  mapped low ram: 0 - 375fe000
  low ram: 0 - 375fe000
  node 0 low ram: 00000000 - 375fe000
  node 0 bootmap 0000b000 - 00011ec0
(9 early reservations) ==> bootmem [0000000000 - 00375fe000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 0000002000]
  #2 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - 0000007000]
  #3 [0000400000 - 0000c981d0]    TEXT DATA BSS ==> [0000400000 - 0000c981d0]
  #4 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
  #5 [0000c99000 - 0000ca1095]              BRK ==> [0000c99000 - 0000ca1095]
  #6 [0000007000 - 000000b000]          PGTABLE ==> [0000007000 - 000000b000]
  #7 [0000ca2000 - 0001c1943f]      NEW RAMDISK ==> [0000ca2000 - 0001c1943f]
  #8 [000000b000 - 0000012000]          BOOTMAP ==> [000000b000 - 0000012000]
found SMP MP-table at [c00fdf30] fdf30
Reserving 128MB of memory at 32MB for crashkernel (System RAM: 4608MB)
kvm-clock: Using msrs 4b564d01 and 4b564d00
kvm-clock: cpu 0, msr 0:b696c1, boot clock
Zone PFN ranges:
  DMA      0x00000001 -> 0x00001000
  Normal   0x00001000 -> 0x000375fe
  HighMem  0x000375fe -> 0x00120000
Movable zone start PFN for each node
early_node_map[3] active PFN ranges
    0: 0x00000001 -> 0x0000009f
    0: 0x00000100 -> 0x000dfffe
    0: 0x00100000 -> 0x00120000
On node 0 totalpages: 1048476
  DMA zone: 32 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 3966 pages, LIFO batch:0
  Normal zone: 1740 pages used for memmap
  Normal zone: 220978 pages, LIFO batch:31
  HighMem zone: 7445 pages used for memmap
  HighMem zone: 814315 pages, LIFO batch:31
Using APIC driver default
ACPI: PM-Timer IO Port: 0xb008
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ5 used by override.
ACPI: IRQ9 used by override.
ACPI: IRQ10 used by override.
ACPI: IRQ11 used by override.
Using ACPI (MADT) for SMP configuration information
SMP: Allowing 4 CPUs, 0 hotplug CPUs
nr_irqs_gsi: 24
PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
Allocating PCI resources starting at e0000000 (gap: e0000000:1effc000)
Booting paravirtualized kernel on KVM
NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:4 nr_node_ids:1
PERCPU: Embedded 14 pages/cpu @c1e00000 s36056 r0 d21288 u524288
pcpu-alloc: s36056 r0 d21288 u524288 alloc=1*2097152
pcpu-alloc: [0] 0 1 2 3 
kvm-clock: cpu 0, msr 0:1e086c1, primary cpu clock
kvm-stealtime: cpu 0, msr 1e03800
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1039259
Kernel command line: ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=128M rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
PID hash table entries: 4096 (order: 2, 16384 bytes)
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
Initializing HighMem for node 0 (000375fe:00120000)
Memory: 4002996k/4718592k available (4504k kernel code, 189736k reserved, 2561k data, 524k init, 3287040k highmem)
virtual kernel memory layout:
    fixmap  : 0xffad5000 - 0xfffff000   (5288 kB)
    pkmap   : 0xff600000 - 0xff800000   (2048 kB)
    vmalloc : 0xf7dfe000 - 0xff5fe000   ( 120 MB)
    lowmem  : 0xc0000000 - 0xf75fe000   ( 885 MB)
      .init : 0xc0ae7000 - 0xc0b6a000   ( 524 kB)
      .data : 0xc0866238 - 0xc0ae6808   (2561 kB)
      .text : 0xc0400000 - 0xc0866238   (4504 kB)
Checking if this processor honours the WP bit even in supervisor mode...Ok.
Hierarchical RCU implementation.
NR_IRQS:2304 nr_irqs:440
Console: colour VGA+ 80x25
console [tty0] enabled
allocated 8388608 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Detected 2000.040 MHz processor.
Marking TSC unstable due to TSCs unsynchronized
Calibrating delay loop (skipped) preset value.. 4000.08 BogoMIPS (lpj=2000040)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
Mount-cache hash table entries: 512
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
Initializing cgroup subsys perf_event
Initializing cgroup subsys net_prio
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 10 MCE banks
alternatives: switching to unfair spinlock
ACPI: Core revision 20090903
Enabling APIC mode:  Flat.  Using 1 I/O APICs
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: AMD Intel Xeon E312xx (Sandy Bridge) stepping 01
Performance Events: Broken PMU hardware detected, using software events only.
Booting Node   0, Processors  #1
Initializing CPU#1
kvm-clock: cpu 1, msr 0:1e886c1, secondary cpu clock
------------[ cut here ]------------
WARNING: at arch/x86/kernel/cpu/amd.c:187 init_amd+0x388/0x641() (Not tainted)
Hardware name: KVM
WARNING: This combination of AMD processors is not suitable for SMP.
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.32-430.el6.i686 #1
Call Trace:
 [<c045b0d1>] ? warn_slowpath_common+0x81/0xc0
 [<c085585d>] ? init_amd+0x388/0x641
 [<c085585d>] ? init_amd+0x388/0x641
 [<c045b1a3>] ? warn_slowpath_fmt+0x33/0x40
 [<c085585d>] ? init_amd+0x388/0x641
 [<c0854bda>] ? identify_cpu+0x28b/0x42d
 [<c042d65d>] ? default_init_apic_ldr+0x4d/0x50
 [<c08599a0>] ? setup_local_APIC+0x337/0x355
 [<c0854d8c>] ? identify_secondary_cpu+0x10/0x1a
 [<c0858e43>] ? start_secondary+0x138/0x2f4
---[ end trace a7919e7f17c0a725 ]---
Disabling lock debugging due to kernel taint
kvm-stealtime: cpu 1, msr 1e83800
 #2
Initializing CPU#2
kvm-clock: cpu 2, msr 0:1f086c1, secondary cpu clock
kvm-stealtime: cpu 2, msr 1f03800
 #3 Ok.
Initializing CPU#3
kvm-clock: cpu 3, msr 0:1f886c1, secondary cpu clock
Brought up 4 CPUs
Total of 4 processors activated (16000.32 BogoMIPS).
kvm-stealtime: cpu 3, msr 1f83800
sizeof(vma)=100 bytes
sizeof(page)=32 bytes
sizeof(inode)=352 bytes
sizeof(dentry)=132 bytes
sizeof(ext3inode)=508 bytes
sizeof(buffer_head)=56 bytes
sizeof(skbuff)=184 bytes
sizeof(task_struct)=1348 bytes
devtmpfs: initialized
regulator: core version 0.5
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfc670, last bus=0
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S3 S4 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7]
pci_root PNP0A03:00: host bridge window [io  0x0d00-0xffff]
pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff]
pci_root PNP0A03:00: host bridge window [mem 0xe0000000-0xfebfffff]
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: root bus resource [mem 0xe0000000-0xfebfffff]
pci 0000:00:01.1: reg 20: [io  0xc100-0xc10f]
pci 0000:00:01.3: quirk: [io  0xb000-0xb03f] claimed by PIIX4 ACPI
pci 0000:00:01.3: quirk: [io  0xb100-0xb10f] claimed by PIIX4 SMB
pci 0000:00:02.0: reg 10: [mem 0xf4000000-0xf7ffffff]
pci 0000:00:02.0: reg 14: [mem 0xf8000000-0xfbffffff]
pci 0000:00:02.0: reg 18: [mem 0xfc030000-0xfc031fff]
pci 0000:00:02.0: reg 1c: [io  0xc0c0-0xc0df]
pci 0000:00:02.0: reg 30: [mem 0xfc020000-0xfc02ffff pref]
pci 0000:00:03.0: reg 10: [io  0xc000-0xc03f]
pci 0000:00:03.0: reg 14: [mem 0xfc032000-0xfc032fff]
pci 0000:00:04.0: reg 10: [io  0xc0e0-0xc0ff]
pci 0000:00:05.0: reg 10: [io  0xc040-0xc07f]
pci 0000:00:05.0: reg 14: [mem 0xfc033000-0xfc033fff]
pci 0000:00:06.0: reg 10: [io  0xc080-0xc0bf]
pci 0000:00:06.0: reg 14: [mem 0xfc034000-0xfc034fff]
pci 0000:00:06.0: reg 30: [mem 0xfc000000-0xfc01ffff pref]
pci_bus 0000:00: on NUMA node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0000:00:02.0
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
PCI: pci_cache_line_size set to 64 bytes
lo: Dropping TSO features since no CSUM feature.
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
Switching to clocksource kvm-clock
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp 00:00: [io  0x0cf8-0x0cff]
pnp 00:00: Plug and Play ACPI device, IDs PNP0a03 (active)
pnp 00:01: [io  0x0070-0x0071]
pnp 00:01: [irq 8]
pnp 00:01: [io  0x0072-0x0077]
pnp 00:01: Plug and Play ACPI device, IDs PNP0b00 (active)
pnp 00:02: [io  0x0060]
pnp 00:02: [io  0x0064]
pnp 00:02: [irq 1]
pnp 00:02: Plug and Play ACPI device, IDs PNP0303 (active)
pnp 00:03: [irq 12]
pnp 00:03: Plug and Play ACPI device, IDs PNP0f13 (active)
pnp 00:04: [io  0x03f2-0x03f5]
pnp 00:04: [io  0x03f7]
pnp 00:04: [irq 6]
pnp 00:04: [dma 2]
pnp 00:04: Plug and Play ACPI device, IDs PNP0700 (active)
pnp 00:05: [io  0x03f8-0x03ff]
pnp 00:05: [irq 4]
pnp 00:05: Plug and Play ACPI device, IDs PNP0501 (active)
pnp: PnP ACPI: found 6 devices
ACPI: ACPI bus type pnp unregistered
PCI: max bus depth: 0 pci_try_num: 1
pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: resource 7 [mem 0xe0000000-0xfebfffff]
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
NET: Registered protocol family 1
pci 0000:00:00.0: Limiting direct PCI/PCI transfers
pci 0000:00:01.0: PIIX3: Enabling Passive Release
pci 0000:00:01.0: Activating ISA DMA hang workarounds
pci 0000:00:02.0: Boot video device
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 15837k freed
apm: BIOS not found.
audit: initializing netlink socket (disabled)
type=2000 audit(1384828980.571:1): initialized
highmem bounce pool size: 64 pages
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
msgmni has been set to 1431
SELinux:  Registering netfilter hooks
alg: No test for stdrng (krng)
ksign: Installing public key data
Loading keyring
- Added public key 6D8C1D71AB3678E6
- User ID: Red Hat, Inc. (Kernel Module GPG key)
- Added public key D4A26C9CCD09BEDA
- User ID: Red Hat Enterprise Linux Driver Update Program <secalert>
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
acpiphp: Slot [3] registered
acpiphp: Slot [4] registered
acpiphp: Slot [5] registered
acpiphp: Slot [6] registered
acpiphp: Slot [7] registered
acpiphp: Slot [8] registered
acpiphp: Slot [9] registered
acpiphp: Slot [10] registered
acpiphp: Slot [11] registered
acpiphp: Slot [12] registered
acpiphp: Slot [13] registered
acpiphp: Slot [14] registered
acpiphp: Slot [15] registered
acpiphp: Slot [16] registered
acpiphp: Slot [17] registered
acpiphp: Slot [18] registered
acpiphp: Slot [19] registered
acpiphp: Slot [20] registered
acpiphp: Slot [21] registered
acpiphp: Slot [22] registered
acpiphp: Slot [23] registered
acpiphp: Slot [24] registered
acpiphp: Slot [25] registered
acpiphp: Slot [26] registered
acpiphp: Slot [27] registered
acpiphp: Slot [28] registered
acpiphp: Slot [29] registered
acpiphp: Slot [30] registered
acpiphp: Slot [31] registered
ipmi message handler version 39.2
IPMI System Interface driver.
ipmi_si: Adding default-specified kcs state machine
ipmi_si: Trying default-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Adding default-specified smic state machine
ipmi_si: Trying default-specified smic state machine at i/o address 0xca9, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Adding default-specified bt state machine
ipmi_si: Trying default-specified bt state machine at i/o address 0xe4, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Unable to find any System Interface(s)
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
ACPI: Power Button [PWRF]
ACPI: acpi_idle registered with cpuidle
ACPI Exception: AE_NOT_FOUND, No or invalid critical threshold (20090903/thermal-386)
GHES: HEST is not enabled!
isapnp: Scanning for PnP cards...
isapnp: No Plug & Play device found
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
crash memory driver: version 1.1
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
brd: module loaded
loop: module loaded
input: Macintosh mouse button emulation as /devices/virtual/input/input1
Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
rtc_cmos 00:01: RTC can wake from S4
rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one day, 114 bytes nvram
cpuidle: using governor ladder
cpuidle: using governor menu
EFI Variables Facility v0.08 2004-May-17
input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
GRE over IPv4 demultiplexor driver
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Using IPI No-Shortcut mode
registered taskstats version 1
rtc_cmos 00:01: setting system clock to 2013-11-19 10:43:02 UTC (1384857782)
Initalizing network drop monitor service
Freeing unused kernel memory: 524k freed
Write protecting the kernel text: 4508k
Write protecting the kernel read-only data: 1980k
dracut: dracut-004-335.el6
dracut: rd_NO_LUKS: removing cryptoluks activation
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.24.6-ioctl (2013-01-15) initialised: dm-devel
udev: starting version 147
psmouse serio1: ID: 10 00 64
dracut: Starting plymouth daemon
dracut: rd_NO_DM: removing DM RAID activation
dracut: rd_NO_MD: removing MD RAID activation
ata_piix 0000:00:01.1: version 2.13
ata_piix 0000:00:01.1: setting latency timer to 64
scsi0 : ata_piix
scsi1 : ata_piix
ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc100 irq 14
ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc108 irq 15
ata1.01: NODEV after polling detection
ata1.00: ATAPI: QEMU DVD-ROM, 1.5.3, max UDMA/100
ata1.00: configured for MWDMA2
scsi 0:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     1.5. PQ: 0 ANSI: 5
ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
virtio-pci 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 11 (level, high) -> IRQ 11
ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 10
virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 10 (level, high) -> IRQ 10
ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
virtio-pci 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 11
virtio-pci 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 11 (level, high) -> IRQ 11
input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input3
sr0: scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
Uniform CD-ROM driver Revision: 3.20
sr 0:0:0:0: Attached scsi CD-ROM sr0
  alloc irq_desc for 24 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:03.0: irq 24 for MSI/MSI-X
  alloc irq_desc for 25 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:03.0: irq 25 for MSI/MSI-X
 vda: vda1 vda2
  alloc irq_desc for 26 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 26 for MSI/MSI-X
  alloc irq_desc for 27 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 27 for MSI/MSI-X
  alloc irq_desc for 28 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 28 for MSI/MSI-X
  alloc irq_desc for 29 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 29 for MSI/MSI-X
scsi2 : Virtio SCSI HBA
scsi 2:0:0:0: Direct-Access     QEMU     QEMU HARDDISK    1.5. PQ: 0 ANSI: 5
sd 2:0:0:0: [sda] 62914560 512-byte logical blocks: (32.2 GB/30.0 GiB)
sd 2:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sda] Mode Sense: 63 00 00 08
sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2
sd 2:0:0:0: [sda] Attached SCSI disk
dracut: Scanning devices sda2 vda2  for LVM logical volumes vg_unused/lv_swap vg_unused/lv_root 
dracut: Found duplicate PV KR0iAw7ZIxbzalquR0vjeyAEWGK3uRGh: using /dev/vda2 not /dev/sda2
dracut: inactive '/dev/vg_unused/lv_root' [26.51 GiB] inherit
dracut: inactive '/dev/vg_unused/lv_swap' [3.00 GiB] inherit
dracut: Found duplicate PV KR0iAw7ZIxbzalquR0vjeyAEWGK3uRGh: using /dev/vda2 not /dev/sda2
EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: 
dracut: Mounted root filesystem /dev/mapper/vg_unused-lv_root
dracut: Loading SELinux policy
type=1404 audit(1384857784.518:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295
SELinux: 2048 avtab hash slots, 277800 rules.
SELinux: 2048 avtab hash slots, 277800 rules.
SELinux:  9 users, 12 roles, 3917 types, 217 bools, 1 sens, 1024 cats
SELinux:  81 classes, 277800 rules
SELinux:  Completing initialization.
SELinux:  Setting up existing superblocks.
SELinux: initialized (dev dm-0, type ext4), uses xattr
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev selinuxfs, type selinuxfs), uses genfs_contexts
SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs
SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses transition SIDs
SELinux: initialized (dev devpts, type devpts), uses transition SIDs
SELinux: initialized (dev inotifyfs, type inotifyfs), uses genfs_contexts
SELinux: initialized (dev anon_inodefs, type anon_inodefs), uses genfs_contexts
SELinux: initialized (dev pipefs, type pipefs), uses task SIDs
SELinux: initialized (dev debugfs, type debugfs), uses genfs_contexts
SELinux: initialized (dev sockfs, type sockfs), uses task SIDs
SELinux: initialized (dev devtmpfs, type devtmpfs), uses transition SIDs
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev proc, type proc), uses genfs_contexts
SELinux: initialized (dev bdev, type bdev), uses genfs_contexts
SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts
SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
type=1403 audit(1384857785.074:3): policy loaded auid=4294967295 ses=4294967295
dracut: 
dracut: Switching root
SELinux: initialized (dev usbfs, type usbfs), uses genfs_contexts
udev: starting version 147
piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0
  alloc irq_desc for 30 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:06.0: irq 30 for MSI/MSI-X
  alloc irq_desc for 31 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:06.0: irq 31 for MSI/MSI-X
  alloc irq_desc for 32 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:06.0: irq 32 for MSI/MSI-X
sr 0:0:0:0: Attached scsi generic sg0 type 5
sd 2:0:0:0: Attached scsi generic sg1 type 0
platform microcode: firmware: requesting amd-ucode/microcode_amd.bin
microcode: CPU0: AMD CPU family 0x6 not supported
microcode: CPU1: AMD CPU family 0x6 not supported
microcode: CPU2: AMD CPU family 0x6 not supported
microcode: CPU3: AMD CPU family 0x6 not supported
Microcode Update Driver: v2.00 <tigran.co.uk>, Peter Oruba
EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: 
SELinux: initialized (dev sda1, type ext4), uses xattr
Adding 3145720k swap on /dev/mapper/vg_unused-lv_swap.  Priority:-1 extents:1 across:3145720k 
SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses genfs_contexts
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
ip6_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
ip_tables: (C) 2000-2006 Netfilter Core Team
802.1Q VLAN Support v1.8 Ben Greear <greearb>
All bugs added by David S. Miller <davem>
8021q: adding VLAN 0 to HW filter on device eth0
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
eth0: no IPv6 routers present
virtio-pci 0000:00:03.0: PCI INT A disabled
Buffer I/O error on device dm-0, logical block 1607374
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1617923
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 10590
lost page write due to I/O error on dm-0
JBD2: Detected IO errors while flushing file data on dm-0-8
Aborting journal on device dm-0-8.
Buffer I/O error on device dm-0, logical block 3178496
lost page write due to I/O error on dm-0
JBD2: I/O error detected when updating journal superblock for dm-0-8.
EXT4-fs error (device dm-0): ext4_journal_start_sb: Detected aborted journal
EXT4-fs (dm-0): Remounting filesystem read-only
Buffer I/O error on device dm-0, logical block 5244523
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048592
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048639
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048643
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048646
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1056781
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1572866
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1572896
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1573263
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1581858
lost page write due to I/O error on dm-0
[root@unused ~]# df -lh
Filesystem                     Size  Used Avail Use% Mounted on
/dev/mapper/vg_unused-lv_root   27G  7.8G   18G  32% /
tmpfs                          2.0G   72K  2.0G   1% /dev/shm
/dev/sda1                      485M   37M  423M   8% /boot
[root@unused ~]# 
[root@unused ~]# umount /dev/vda
umount: /dev/vda: not found
[root@unused ~]# umount /dev/vda1
umount: /dev/vda1: not found
[root@unused ~]# umount /dev/vda2
umount: /dev/vda2: not found
[root@unused ~]# reboot
-bash: /sbin/reboot: Input/output error
[root@unused ~]# poweroff 
-bash: /sbin/poweroff: Input/output error
[root@unused ~]# dmesg 
Initializing cgroup subsys cpuset
Initializing cgroup subsys cpu
Linux version 2.6.32-430.el6.i686 (mockbuild.eng.bos.redhat.com) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-4) (GCC) ) #1 SMP Tue Nov 5 15:07:44 EST 2013
KERNEL supported cpus:
  Intel GenuineIntel
  AMD AuthenticAMD
  NSC Geode by NSC
  Cyrix CyrixInstead
  Centaur CentaurHauls
  Transmeta GenuineTMx86
  Transmeta TransmetaCPU
  UMC UMC UMC UMC
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000f0000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 00000000dfffe000 (usable)
 BIOS-e820: 00000000dfffe000 - 00000000e0000000 (reserved)
 BIOS-e820: 00000000feffc000 - 00000000ff000000 (reserved)
 BIOS-e820: 00000000fffc0000 - 0000000100000000 (reserved)
 BIOS-e820: 0000000100000000 - 0000000120000000 (usable)
DMI 2.4 present.
SMBIOS version 2.4 @ 0xFDF10
DMI: Red Hat KVM, BIOS Bochs 01/01/2011
Hypervisor detected: KVM
e820 update range: 0000000000000000 - 0000000000001000 (usable) ==> (reserved)
e820 remove range: 00000000000a0000 - 0000000000100000 (usable)
last_pfn = 0x120000 max_arch_pfn = 0x400000
MTRR default type: write-back
MTRR fixed ranges enabled:
  00000-9FFFF write-back
  A0000-BFFFF uncachable
  C0000-FFFFF write-protect
MTRR variable ranges enabled:
  0 base 00E0000000 mask FFE0000000 uncachable
  1 disabled
  2 disabled
  3 disabled
  4 disabled
  5 disabled
  6 disabled
  7 disabled
x86 PAT enabled: cpu 0, old 0x70406, new 0x7010600070106
initial memory mapped : 0 - 01000000
init_memory_mapping: 0000000000000000-00000000375fe000
NX (Execute Disable) protection: active
 0000000000 - 0000200000 page 4k
 0000200000 - 0037400000 page 2M
 0037400000 - 00375fe000 page 4k
kernel direct mapping tables up to 375fe000 @ 7000-f000
RAMDISK: 37078000 - 37fef43f
Allocated new RAMDISK: 00ca2000 - 01c1943f
Move RAMDISK from 0000000037078000 - 0000000037fef43e to 00ca2000 - 01c1943e
ACPI: RSDP 000fdce0 00014 (v00 BOCHS )
ACPI: RSDT dfffe310 00030 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
ACPI: FACP dfffff80 00074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
ACPI: DSDT dfffe340 011A9 (v01   BXPC   BXDSDT 00000001 INTL 20130823)
ACPI: FACS dfffff40 00040
ACPI: SSDT dffff620 00919 (v01 BOCHS  BXPCSSDT 00000001 BXPC 00000001)
ACPI: APIC dffff4f0 00090 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
ACPI: Local APIC address 0xfee00000
3722MB HIGHMEM available.
885MB LOWMEM available.
  mapped low ram: 0 - 375fe000
  low ram: 0 - 375fe000
  node 0 low ram: 00000000 - 375fe000
  node 0 bootmap 0000b000 - 00011ec0
(9 early reservations) ==> bootmem [0000000000 - 00375fe000]
  #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
  #1 [0000001000 - 0000002000]    EX TRAMPOLINE ==> [0000001000 - 0000002000]
  #2 [0000006000 - 0000007000]       TRAMPOLINE ==> [0000006000 - 0000007000]
  #3 [0000400000 - 0000c981d0]    TEXT DATA BSS ==> [0000400000 - 0000c981d0]
  #4 [000009fc00 - 0000100000]    BIOS reserved ==> [000009fc00 - 0000100000]
  #5 [0000c99000 - 0000ca1095]              BRK ==> [0000c99000 - 0000ca1095]
  #6 [0000007000 - 000000b000]          PGTABLE ==> [0000007000 - 000000b000]
  #7 [0000ca2000 - 0001c1943f]      NEW RAMDISK ==> [0000ca2000 - 0001c1943f]
  #8 [000000b000 - 0000012000]          BOOTMAP ==> [000000b000 - 0000012000]
found SMP MP-table at [c00fdf30] fdf30
Reserving 128MB of memory at 32MB for crashkernel (System RAM: 4608MB)
kvm-clock: Using msrs 4b564d01 and 4b564d00
kvm-clock: cpu 0, msr 0:b696c1, boot clock
Zone PFN ranges:
  DMA      0x00000001 -> 0x00001000
  Normal   0x00001000 -> 0x000375fe
  HighMem  0x000375fe -> 0x00120000
Movable zone start PFN for each node
early_node_map[3] active PFN ranges
    0: 0x00000001 -> 0x0000009f
    0: 0x00000100 -> 0x000dfffe
    0: 0x00100000 -> 0x00120000
On node 0 totalpages: 1048476
  DMA zone: 32 pages used for memmap
  DMA zone: 0 pages reserved
  DMA zone: 3966 pages, LIFO batch:0
  Normal zone: 1740 pages used for memmap
  Normal zone: 220978 pages, LIFO batch:31
  HighMem zone: 7445 pages used for memmap
  HighMem zone: 814315 pages, LIFO batch:31
Using APIC driver default
ACPI: PM-Timer IO Port: 0xb008
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ5 used by override.
ACPI: IRQ9 used by override.
ACPI: IRQ10 used by override.
ACPI: IRQ11 used by override.
Using ACPI (MADT) for SMP configuration information
SMP: Allowing 4 CPUs, 0 hotplug CPUs
nr_irqs_gsi: 24
PM: Registered nosave memory: 000000000009f000 - 00000000000a0000
PM: Registered nosave memory: 00000000000a0000 - 00000000000f0000
PM: Registered nosave memory: 00000000000f0000 - 0000000000100000
Allocating PCI resources starting at e0000000 (gap: e0000000:1effc000)
Booting paravirtualized kernel on KVM
NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:4 nr_node_ids:1
PERCPU: Embedded 14 pages/cpu @c1e00000 s36056 r0 d21288 u524288
pcpu-alloc: s36056 r0 d21288 u524288 alloc=1*2097152
pcpu-alloc: [0] 0 1 2 3 
kvm-clock: cpu 0, msr 0:1e086c1, primary cpu clock
kvm-stealtime: cpu 0, msr 1e03800
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1039259
Kernel command line: ro root=/dev/mapper/vg_unused-lv_root rd_NO_LUKS  KEYBOARDTYPE=pc KEYTABLE=us LANG=en_US.UTF-8 rd_LVM_LV=vg_unused/lv_swap rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=128M rd_LVM_LV=vg_unused/lv_root rd_NO_DM rhgb quiet
PID hash table entries: 4096 (order: 2, 16384 bytes)
Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
Enabling fast FPU save and restore... done.
Enabling unmasked SIMD FPU exception support... done.
Initializing CPU#0
Initializing HighMem for node 0 (000375fe:00120000)
Memory: 4002996k/4718592k available (4504k kernel code, 189736k reserved, 2561k data, 524k init, 3287040k highmem)
virtual kernel memory layout:
    fixmap  : 0xffad5000 - 0xfffff000   (5288 kB)
    pkmap   : 0xff600000 - 0xff800000   (2048 kB)
    vmalloc : 0xf7dfe000 - 0xff5fe000   ( 120 MB)
    lowmem  : 0xc0000000 - 0xf75fe000   ( 885 MB)
      .init : 0xc0ae7000 - 0xc0b6a000   ( 524 kB)
      .data : 0xc0866238 - 0xc0ae6808   (2561 kB)
      .text : 0xc0400000 - 0xc0866238   (4504 kB)
Checking if this processor honours the WP bit even in supervisor mode...Ok.
Hierarchical RCU implementation.
NR_IRQS:2304 nr_irqs:440
Console: colour VGA+ 80x25
console [tty0] enabled
allocated 8388608 bytes of page_cgroup
please try 'cgroup_disable=memory' option if you don't want memory cgroups
Detected 2000.040 MHz processor.
Marking TSC unstable due to TSCs unsynchronized
Calibrating delay loop (skipped) preset value.. 4000.08 BogoMIPS (lpj=2000040)
pid_max: default: 32768 minimum: 301
Security Framework initialized
SELinux:  Initializing.
SELinux:  Starting in permissive mode
Mount-cache hash table entries: 512
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
Initializing cgroup subsys net_cls
Initializing cgroup subsys blkio
Initializing cgroup subsys perf_event
Initializing cgroup subsys net_prio
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
mce: CPU supports 10 MCE banks
alternatives: switching to unfair spinlock
ACPI: Core revision 20090903
Enabling APIC mode:  Flat.  Using 1 I/O APICs
..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
CPU0: AMD Intel Xeon E312xx (Sandy Bridge) stepping 01
Performance Events: Broken PMU hardware detected, using software events only.
Booting Node   0, Processors  #1
Initializing CPU#1
kvm-clock: cpu 1, msr 0:1e886c1, secondary cpu clock
------------[ cut here ]------------
WARNING: at arch/x86/kernel/cpu/amd.c:187 init_amd+0x388/0x641() (Not tainted)
Hardware name: KVM
WARNING: This combination of AMD processors is not suitable for SMP.
Modules linked in:
Pid: 0, comm: swapper Not tainted 2.6.32-430.el6.i686 #1
Call Trace:
 [<c045b0d1>] ? warn_slowpath_common+0x81/0xc0
 [<c085585d>] ? init_amd+0x388/0x641
 [<c085585d>] ? init_amd+0x388/0x641
 [<c045b1a3>] ? warn_slowpath_fmt+0x33/0x40
 [<c085585d>] ? init_amd+0x388/0x641
 [<c0854bda>] ? identify_cpu+0x28b/0x42d
 [<c042d65d>] ? default_init_apic_ldr+0x4d/0x50
 [<c08599a0>] ? setup_local_APIC+0x337/0x355
 [<c0854d8c>] ? identify_secondary_cpu+0x10/0x1a
 [<c0858e43>] ? start_secondary+0x138/0x2f4
---[ end trace a7919e7f17c0a725 ]---
Disabling lock debugging due to kernel taint
kvm-stealtime: cpu 1, msr 1e83800
 #2
Initializing CPU#2
kvm-clock: cpu 2, msr 0:1f086c1, secondary cpu clock
kvm-stealtime: cpu 2, msr 1f03800
 #3 Ok.
Initializing CPU#3
kvm-clock: cpu 3, msr 0:1f886c1, secondary cpu clock
Brought up 4 CPUs
Total of 4 processors activated (16000.32 BogoMIPS).
kvm-stealtime: cpu 3, msr 1f83800
sizeof(vma)=100 bytes
sizeof(page)=32 bytes
sizeof(inode)=352 bytes
sizeof(dentry)=132 bytes
sizeof(ext3inode)=508 bytes
sizeof(buffer_head)=56 bytes
sizeof(skbuff)=184 bytes
sizeof(task_struct)=1348 bytes
devtmpfs: initialized
regulator: core version 0.5
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: PCI BIOS revision 2.10 entry at 0xfc670, last bus=0
PCI: Using configuration type 1 for base access
bio: create slab <bio-0> at 0
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S3 S4 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: No dock devices found.
PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
pci_root PNP0A03:00: host bridge window [io  0x0000-0x0cf7]
pci_root PNP0A03:00: host bridge window [io  0x0d00-0xffff]
pci_root PNP0A03:00: host bridge window [mem 0x000a0000-0x000bffff]
pci_root PNP0A03:00: host bridge window [mem 0xe0000000-0xfebfffff]
PCI host bridge to bus 0000:00
pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: root bus resource [mem 0xe0000000-0xfebfffff]
pci 0000:00:01.1: reg 20: [io  0xc100-0xc10f]
pci 0000:00:01.3: quirk: [io  0xb000-0xb03f] claimed by PIIX4 ACPI
pci 0000:00:01.3: quirk: [io  0xb100-0xb10f] claimed by PIIX4 SMB
pci 0000:00:02.0: reg 10: [mem 0xf4000000-0xf7ffffff]
pci 0000:00:02.0: reg 14: [mem 0xf8000000-0xfbffffff]
pci 0000:00:02.0: reg 18: [mem 0xfc030000-0xfc031fff]
pci 0000:00:02.0: reg 1c: [io  0xc0c0-0xc0df]
pci 0000:00:02.0: reg 30: [mem 0xfc020000-0xfc02ffff pref]
pci 0000:00:03.0: reg 10: [io  0xc000-0xc03f]
pci 0000:00:03.0: reg 14: [mem 0xfc032000-0xfc032fff]
pci 0000:00:04.0: reg 10: [io  0xc0e0-0xc0ff]
pci 0000:00:05.0: reg 10: [io  0xc040-0xc07f]
pci 0000:00:05.0: reg 14: [mem 0xfc033000-0xfc033fff]
pci 0000:00:06.0: reg 10: [io  0xc080-0xc0bf]
pci 0000:00:06.0: reg 14: [mem 0xfc034000-0xfc034fff]
pci 0000:00:06.0: reg 30: [mem 0xfc000000-0xfc01ffff pref]
pci_bus 0000:00: on NUMA node 0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
vgaarb: loaded
vgaarb: bridge control possible 0000:00:02.0
SCSI subsystem initialized
libata version 3.00 loaded.
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
PCI: Using ACPI for IRQ routing
PCI: pci_cache_line_size set to 64 bytes
lo: Dropping TSO features since no CSUM feature.
NetLabel: Initializing
NetLabel:  domain hash size = 128
NetLabel:  protocols = UNLABELED CIPSOv4
NetLabel:  unlabeled traffic allowed by default
Switching to clocksource kvm-clock
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp 00:00: [io  0x0cf8-0x0cff]
pnp 00:00: Plug and Play ACPI device, IDs PNP0a03 (active)
pnp 00:01: [io  0x0070-0x0071]
pnp 00:01: [irq 8]
pnp 00:01: [io  0x0072-0x0077]
pnp 00:01: Plug and Play ACPI device, IDs PNP0b00 (active)
pnp 00:02: [io  0x0060]
pnp 00:02: [io  0x0064]
pnp 00:02: [irq 1]
pnp 00:02: Plug and Play ACPI device, IDs PNP0303 (active)
pnp 00:03: [irq 12]
pnp 00:03: Plug and Play ACPI device, IDs PNP0f13 (active)
pnp 00:04: [io  0x03f2-0x03f5]
pnp 00:04: [io  0x03f7]
pnp 00:04: [irq 6]
pnp 00:04: [dma 2]
pnp 00:04: Plug and Play ACPI device, IDs PNP0700 (active)
pnp 00:05: [io  0x03f8-0x03ff]
pnp 00:05: [irq 4]
pnp 00:05: Plug and Play ACPI device, IDs PNP0501 (active)
pnp: PnP ACPI: found 6 devices
ACPI: ACPI bus type pnp unregistered
PCI: max bus depth: 0 pci_try_num: 1
pci_bus 0000:00: resource 4 [io  0x0000-0x0cf7]
pci_bus 0000:00: resource 5 [io  0x0d00-0xffff]
pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff]
pci_bus 0000:00: resource 7 [mem 0xe0000000-0xfebfffff]
NET: Registered protocol family 2
IP route cache hash table entries: 32768 (order: 5, 131072 bytes)
TCP established hash table entries: 131072 (order: 8, 1048576 bytes)
TCP bind hash table entries: 65536 (order: 7, 524288 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
NET: Registered protocol family 1
pci 0000:00:00.0: Limiting direct PCI/PCI transfers
pci 0000:00:01.0: PIIX3: Enabling Passive Release
pci 0000:00:01.0: Activating ISA DMA hang workarounds
pci 0000:00:02.0: Boot video device
Trying to unpack rootfs image as initramfs...
Freeing initrd memory: 15837k freed
apm: BIOS not found.
audit: initializing netlink socket (disabled)
type=2000 audit(1384828980.571:1): initialized
highmem bounce pool size: 64 pages
HugeTLB registered 2 MB page size, pre-allocated 0 pages
VFS: Disk quotas dquot_6.5.2
Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
msgmni has been set to 1431
SELinux:  Registering netfilter hooks
alg: No test for stdrng (krng)
ksign: Installing public key data
Loading keyring
- Added public key 6D8C1D71AB3678E6
- User ID: Red Hat, Inc. (Kernel Module GPG key)
- Added public key D4A26C9CCD09BEDA
- User ID: Red Hat Enterprise Linux Driver Update Program <secalert>
Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
io scheduler noop registered
io scheduler anticipatory registered
io scheduler deadline registered
io scheduler cfq registered (default)
pci_hotplug: PCI Hot Plug PCI Core version: 0.5
pciehp: PCI Express Hot Plug Controller Driver version: 0.4
acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
acpiphp: Slot [3] registered
acpiphp: Slot [4] registered
acpiphp: Slot [5] registered
acpiphp: Slot [6] registered
acpiphp: Slot [7] registered
acpiphp: Slot [8] registered
acpiphp: Slot [9] registered
acpiphp: Slot [10] registered
acpiphp: Slot [11] registered
acpiphp: Slot [12] registered
acpiphp: Slot [13] registered
acpiphp: Slot [14] registered
acpiphp: Slot [15] registered
acpiphp: Slot [16] registered
acpiphp: Slot [17] registered
acpiphp: Slot [18] registered
acpiphp: Slot [19] registered
acpiphp: Slot [20] registered
acpiphp: Slot [21] registered
acpiphp: Slot [22] registered
acpiphp: Slot [23] registered
acpiphp: Slot [24] registered
acpiphp: Slot [25] registered
acpiphp: Slot [26] registered
acpiphp: Slot [27] registered
acpiphp: Slot [28] registered
acpiphp: Slot [29] registered
acpiphp: Slot [30] registered
acpiphp: Slot [31] registered
ipmi message handler version 39.2
IPMI System Interface driver.
ipmi_si: Adding default-specified kcs state machine
ipmi_si: Trying default-specified kcs state machine at i/o address 0xca2, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Adding default-specified smic state machine
ipmi_si: Trying default-specified smic state machine at i/o address 0xca9, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Adding default-specified bt state machine
ipmi_si: Trying default-specified bt state machine at i/o address 0xe4, slave address 0x0, irq 0
ipmi_si: Interface detection failed
ipmi_si: Unable to find any System Interface(s)
input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
ACPI: Power Button [PWRF]
ACPI: acpi_idle registered with cpuidle
ACPI Exception: AE_NOT_FOUND, No or invalid critical threshold (20090903/thermal-386)
GHES: HEST is not enabled!
isapnp: Scanning for PnP cards...
isapnp: No Plug & Play device found
Non-volatile memory driver v1.3
Linux agpgart interface v0.103
crash memory driver: version 1.1
Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
00:05: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
brd: module loaded
loop: module loaded
input: Macintosh mouse button emulation as /devices/virtual/input/input1
Fixed MDIO Bus: probed
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
uhci_hcd: USB Universal Host Controller Interface driver
PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
serio: i8042 KBD port at 0x60,0x64 irq 1
serio: i8042 AUX port at 0x60,0x64 irq 12
mice: PS/2 mouse device common for all mice
rtc_cmos 00:01: RTC can wake from S4
rtc_cmos 00:01: rtc core: registered rtc_cmos as rtc0
rtc0: alarms up to one day, 114 bytes nvram
cpuidle: using governor ladder
cpuidle: using governor menu
EFI Variables Facility v0.08 2004-May-17
input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
usbcore: registered new interface driver hiddev
usbcore: registered new interface driver usbhid
usbhid: v2.6:USB HID core driver
GRE over IPv4 demultiplexor driver
TCP cubic registered
Initializing XFRM netlink socket
NET: Registered protocol family 17
Using IPI No-Shortcut mode
registered taskstats version 1
rtc_cmos 00:01: setting system clock to 2013-11-19 10:43:02 UTC (1384857782)
Initalizing network drop monitor service
Freeing unused kernel memory: 524k freed
Write protecting the kernel text: 4508k
Write protecting the kernel read-only data: 1980k
dracut: dracut-004-335.el6
dracut: rd_NO_LUKS: removing cryptoluks activation
device-mapper: uevent: version 1.0.3
device-mapper: ioctl: 4.24.6-ioctl (2013-01-15) initialised: dm-devel
udev: starting version 147
psmouse serio1: ID: 10 00 64
dracut: Starting plymouth daemon
dracut: rd_NO_DM: removing DM RAID activation
dracut: rd_NO_MD: removing MD RAID activation
ata_piix 0000:00:01.1: version 2.13
ata_piix 0000:00:01.1: setting latency timer to 64
scsi0 : ata_piix
scsi1 : ata_piix
ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc100 irq 14
ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc108 irq 15
ata1.01: NODEV after polling detection
ata1.00: ATAPI: QEMU DVD-ROM, 1.5.3, max UDMA/100
ata1.00: configured for MWDMA2
scsi 0:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     1.5. PQ: 0 ANSI: 5
ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 11
virtio-pci 0000:00:03.0: PCI INT A -> Link[LNKC] -> GSI 11 (level, high) -> IRQ 11
ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 10
virtio-pci 0000:00:04.0: PCI INT A -> Link[LNKD] -> GSI 10 (level, high) -> IRQ 10
ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
virtio-pci 0000:00:05.0: PCI INT A -> Link[LNKA] -> GSI 10 (level, high) -> IRQ 10
ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 11
virtio-pci 0000:00:06.0: PCI INT A -> Link[LNKB] -> GSI 11 (level, high) -> IRQ 11
input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input3
sr0: scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
Uniform CD-ROM driver Revision: 3.20
sr 0:0:0:0: Attached scsi CD-ROM sr0
  alloc irq_desc for 24 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:03.0: irq 24 for MSI/MSI-X
  alloc irq_desc for 25 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:03.0: irq 25 for MSI/MSI-X
 vda: vda1 vda2
  alloc irq_desc for 26 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 26 for MSI/MSI-X
  alloc irq_desc for 27 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 27 for MSI/MSI-X
  alloc irq_desc for 28 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 28 for MSI/MSI-X
  alloc irq_desc for 29 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:05.0: irq 29 for MSI/MSI-X
scsi2 : Virtio SCSI HBA
scsi 2:0:0:0: Direct-Access     QEMU     QEMU HARDDISK    1.5. PQ: 0 ANSI: 5
sd 2:0:0:0: [sda] 62914560 512-byte logical blocks: (32.2 GB/30.0 GiB)
sd 2:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sda] Mode Sense: 63 00 00 08
sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2
sd 2:0:0:0: [sda] Attached SCSI disk
dracut: Scanning devices sda2 vda2  for LVM logical volumes vg_unused/lv_swap vg_unused/lv_root 
dracut: Found duplicate PV KR0iAw7ZIxbzalquR0vjeyAEWGK3uRGh: using /dev/vda2 not /dev/sda2
dracut: inactive '/dev/vg_unused/lv_root' [26.51 GiB] inherit
dracut: inactive '/dev/vg_unused/lv_swap' [3.00 GiB] inherit
dracut: Found duplicate PV KR0iAw7ZIxbzalquR0vjeyAEWGK3uRGh: using /dev/vda2 not /dev/sda2
EXT4-fs (dm-0): mounted filesystem with ordered data mode. Opts: 
dracut: Mounted root filesystem /dev/mapper/vg_unused-lv_root
dracut: Loading SELinux policy
type=1404 audit(1384857784.518:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295
SELinux: 2048 avtab hash slots, 277800 rules.
SELinux: 2048 avtab hash slots, 277800 rules.
SELinux:  9 users, 12 roles, 3917 types, 217 bools, 1 sens, 1024 cats
SELinux:  81 classes, 277800 rules
SELinux:  Completing initialization.
SELinux:  Setting up existing superblocks.
SELinux: initialized (dev dm-0, type ext4), uses xattr
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev selinuxfs, type selinuxfs), uses genfs_contexts
SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs
SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses transition SIDs
SELinux: initialized (dev devpts, type devpts), uses transition SIDs
SELinux: initialized (dev inotifyfs, type inotifyfs), uses genfs_contexts
SELinux: initialized (dev anon_inodefs, type anon_inodefs), uses genfs_contexts
SELinux: initialized (dev pipefs, type pipefs), uses task SIDs
SELinux: initialized (dev debugfs, type debugfs), uses genfs_contexts
SELinux: initialized (dev sockfs, type sockfs), uses task SIDs
SELinux: initialized (dev devtmpfs, type devtmpfs), uses transition SIDs
SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs
SELinux: initialized (dev proc, type proc), uses genfs_contexts
SELinux: initialized (dev bdev, type bdev), uses genfs_contexts
SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts
SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts
type=1403 audit(1384857785.074:3): policy loaded auid=4294967295 ses=4294967295
dracut: 
dracut: Switching root
SELinux: initialized (dev usbfs, type usbfs), uses genfs_contexts
udev: starting version 147
piix4_smbus 0000:00:01.3: SMBus Host Controller at 0xb100, revision 0
  alloc irq_desc for 30 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:06.0: irq 30 for MSI/MSI-X
  alloc irq_desc for 31 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:06.0: irq 31 for MSI/MSI-X
  alloc irq_desc for 32 on node -1
  alloc kstat_irqs on node -1
virtio-pci 0000:00:06.0: irq 32 for MSI/MSI-X
sr 0:0:0:0: Attached scsi generic sg0 type 5
sd 2:0:0:0: Attached scsi generic sg1 type 0
platform microcode: firmware: requesting amd-ucode/microcode_amd.bin
microcode: CPU0: AMD CPU family 0x6 not supported
microcode: CPU1: AMD CPU family 0x6 not supported
microcode: CPU2: AMD CPU family 0x6 not supported
microcode: CPU3: AMD CPU family 0x6 not supported
Microcode Update Driver: v2.00 <tigran.co.uk>, Peter Oruba
EXT4-fs (sda1): mounted filesystem with ordered data mode. Opts: 
SELinux: initialized (dev sda1, type ext4), uses xattr
Adding 3145720k swap on /dev/mapper/vg_unused-lv_swap.  Priority:-1 extents:1 across:3145720k 
SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses genfs_contexts
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
ip6_tables: (C) 2000-2006 Netfilter Core Team
nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
ip_tables: (C) 2000-2006 Netfilter Core Team
802.1Q VLAN Support v1.8 Ben Greear <greearb>
All bugs added by David S. Miller <davem>
8021q: adding VLAN 0 to HW filter on device eth0
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
SELinux: initialized (dev autofs, type autofs), uses genfs_contexts
eth0: no IPv6 routers present
virtio-pci 0000:00:03.0: PCI INT A disabled
Buffer I/O error on device dm-0, logical block 1607374
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1617923
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 10590
lost page write due to I/O error on dm-0
JBD2: Detected IO errors while flushing file data on dm-0-8
Aborting journal on device dm-0-8.
Buffer I/O error on device dm-0, logical block 3178496
lost page write due to I/O error on dm-0
JBD2: I/O error detected when updating journal superblock for dm-0-8.
EXT4-fs error (device dm-0): ext4_journal_start_sb: Detected aborted journal
EXT4-fs (dm-0): Remounting filesystem read-only
Buffer I/O error on device dm-0, logical block 5244523
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048592
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048639
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048643
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1048646
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1056781
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1572866
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1572896
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1573263
lost page write due to I/O error on dm-0
Buffer I/O error on device dm-0, logical block 1581858
lost page write due to I/O error on dm-0

Comment 6 Jun Li 2013-11-19 04:12:13 UTC
Hi Paolo,

Also test with local data disk, no this issue.
<cli> ... -drive file=/root/test.img,if=none,id=drive-virtio0-0-0,media=disk,werror=stop,rerror=stop,cache=none,format=qcow2 -device virtio-blk-pci,bus=pci.0,drive=drive-virtio0-0-0,id=virtio0-0-0  ...

Best Regards,

Jun Li

Comment 7 Paolo Bonzini 2014-02-21 15:03:27 UTC
Might be caused by bug 1066623.

Comment 14 Yang Meng 2017-01-09 09:41:51 UTC
1.host info:
kernel-2.6.32-681.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.499.el6.x86_64
2. loginto iscsi server
iscsiadm --mode discovery --type sendtargets --portal 10.73.72.148:3260
Starting iscsid:                                           [  OK  ]
10.73.72.148:3260,1 iqn.2017-01.com.redhat:unplug

iscsiadm --mode node --targetname iqn.2017-01.com.redhat:unplug --portal 10.73.72.148 --login
Logging in to [iface: default, target: iqn.2017-01.com.redhat:unplug, portal: 10.73.72.148,3260] (multiple)
Login to [iface: default, target: iqn.2017-01.com.redhat:unplug, portal: 10.73.72.148,3260] successful.

3. boot up guest with the disk

MALLOC_PERTURB_=1  /usr/libexec/qemu-kvm \
    -S  \
    -name 'avocado-vt-vm1' \
    -machine rhel6.6.0  \
    -nodefaults  \
    -vga std \
    -chardev socket,id=qmp_id_qmpmonitor1,path=/var/tmp/monitor-qmpmonitor1-20170109-005801-DFs5kBqh,server,nowait \
    -mon chardev=qmp_id_qmpmonitor1,mode=control  \
    -chardev socket,id=qmp_id_catch_monitor,path=/var/tmp/monitor-catch_monitor-20170109-005801-DFs5kBqh,server,nowait \
    -mon chardev=qmp_id_catch_monitor,mode=control \
    -device pvpanic,ioport=0x505,id=idl4FEEf  \
    -chardev socket,id=serial_id_serial0,path=/var/tmp/serial-serial0-20170109-005801-DFs5kBqh,server,nowait \
    -device isa-serial,chardev=serial_id_serial0  \
    -chardev socket,id=seabioslog_id_20170109-005801-DFs5kBqh,path=/var/tmp/seabios-20170109-005801-DFs5kBqh,server,nowait \
    -device isa-debugcon,chardev=seabioslog_id_20170109-005801-DFs5kBqh,iobase=0x402 \
    -device ich9-usb-ehci1,id=usb1,addr=1d.7,multifunction=on,bus=pci.0 \
    -device ich9-usb-uhci1,id=usb1.0,multifunction=on,masterbus=usb1.0,addr=1d.0,firstport=0,bus=pci.0 \
    -device ich9-usb-uhci2,id=usb1.1,multifunction=on,masterbus=usb1.0,addr=1d.2,firstport=2,bus=pci.0 \
    -device ich9-usb-uhci3,id=usb1.2,multifunction=on,masterbus=usb1.0,addr=1d.4,firstport=4,bus=pci.0 \
    -device virtio-scsi-pci,id=virtio_scsi_pci0,bus=pci.0,addr=04 \
    -drive id=drive_image1,if=none,snapshot=off,aio=native,cache=none,format=qcow2,file=/usr/share/avocado/data/avocado-vt/images/win7-64-sp1-virtio-scsi.qcow2 \
    -device scsi-hd,id=image1,drive=drive_image1,bootindex=0 \
    -device virtio-net-pci,mac=9a:84:85:86:87:88,id=idVfEcMA,vectors=4,netdev=idTAGAwM,bus=pci.0,addr=05  \
    -netdev tap,id=idTAGAwM,vhost=on \
    -m 8192  \
    -smp 4,maxcpus=4,cores=2,threads=1,sockets=2  \
    -cpu 'Westmere',hv_relaxed \
    -drive id=drive_cd1,if=none,snapshot=off,aio=native,cache=none,media=cdrom,file=/usr/share/avocado/data/avocado-vt/isos/windows/winutils.iso \
    -device scsi-cd,id=cd1,drive=drive_cd1 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1  \
    -vnc :0  \
    -rtc base=localtime,clock=host,driftfix=slew  \
    -boot order=cdn,once=c,menu=off,strict=off \
    -enable-kvm \
    -monitor stdio \
    -device virtio-scsi-pci,bus=pci.0,id=scsi0 \
    -device scsi-block,drive=drive-data-disk,bus=scsi0.0,scsi-id=0,lun=0,id=data_disk \
    -drive file=/dev/sdc,if=none,id=drive-data-disk,media=disk,werror=stop,rerror=stop,cache=none,format=raw \

4.unplug the disk

[root@hp-dl388g8-16 home]# nc -U /var/tmp/monitor-qmpmonitor1-20170109-005801-DFs5kBqh
{"QMP": {"version": {"qemu": {"micro": 1, "minor": 12, "major": 0}, "package": "(qemu-kvm-rhev-0.12.1.2-2.499.el6)"}, "capabilities": []}}
{"execute":"qmp_capabilities"}
{"return": {}}
{ "execute": "device_del", "arguments": { "id":"data_disk" } }
{"timestamp": {"seconds": 1483954849, "microseconds": 564119}, "event": "DEVICE_DELETED", "data": {"device": "data_disk"}}
{"return": {}}

{ "execute": "device_del", "arguments": { "id":"scsi0" } }
{"return": {}}
{"timestamp": {"seconds": 1483954856, "microseconds": 615673}, "event": "DEVICE_DELETED", "data": {"device": "scsi0"}}

5. no error ,no crash.

Comment 15 Gu Nini 2017-01-12 10:17:05 UTC
Meng,

It seems the bug had been reported on qemu-kvm of RHEL7, we should reproduce it on the corresponding versions.

Thanks!
Nini

Comment 16 Yang Meng 2017-01-13 02:13:30 UTC
will try on rhel7.4 host using qemu-kvm , will update when i am done

Comment 17 Yang Meng 2017-01-13 05:51:53 UTC
tryied on both qemu-kvm-rhev & qemu-kvm

qemu-kvm-rhev: qemu-kvm-rhev-2.8.0-1.el7.x86_64
qemu-kvm: qemu-kvm-1.5.3-126.el7_3.3.x86_64

and the steps is exactly like #comment 14.

and the results is ok, no error, no coredump , no crash.

Comment 19 Ademar Reis 2017-06-14 20:14:01 UTC
(In reply to Yang Meng from comment #17)
> tryied on both qemu-kvm-rhev & qemu-kvm
> 
> qemu-kvm-rhev: qemu-kvm-rhev-2.8.0-1.el7.x86_64
> qemu-kvm: qemu-kvm-1.5.3-126.el7_3.3.x86_64
> 
> and the steps is exactly like #comment 14.
> 
> and the results is ok, no error, no coredump , no crash.

Given these results and the fact this BZ is very old, I'm closing it. Please reopen if it ever reproduces. Thanks.