Bug 1024684

Summary: Report error if there are too many open file descriptors
Product: Red Hat Enterprise Linux 6 Reporter: Sibiao Luo <sluo>
Component: qemu-kvmAssignee: Fam Zheng <famz>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 6.5CC: chayang, juzhang, michen, mkenneth, pbonzini, qzhang, rbalakri, rpacheco, virt-maint, xfu
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: qemu-kvm-0.12.1.2-2.453.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-07-22 06:04:25 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On: 1005016, 1196955    
Bug Blocks:    

Description Sibiao Luo 2013-10-30 09:03:09 UTC
Description of problem:
when i tried to random test with 1000 disks assigned to random controllers, targets and LUNs, but guest fail to boot up if assigned more than 992 disks.
BWT, it can do hotplug with 1024 number virtio-scsi disk to guest successfully and they can be detected in guest successfully.

Version-Release number of selected component (if applicable):
host info:
# uname -r && rpm -q qemu-kvm
2.6.32-425.el6.x86_64
qemu-kvm-0.12.1.2-2.415.el6.x86_64
seabios-0.6.1.2-28.el6.x86_64
guest info:
2.6.32-425.el6.x86_64

How reproducible:
100%

Steps to Reproduce:
1.assigned more than 992 disks to random controllers, targets and LUNs.
# sh cli-random-disk.sh 512
Script Example:
# cat cli-random-disk.sh
ulimit -n 40960
cli="/usr/libexec/qemu-kvm -S -M pc -m 24G -smp 12 -cpu SandyBridge -vnc :1 -monitor stdio -boot menu=on -monitor unix:/tmp/monitor,server,nowait -drive file=/home/RHEL-6.5-Snapshot-4-Server-x86_64.qcow2,if=none,id=blk1 -device virtio-blk-pci,scsi=off,drive=blk1,id=blk-disk1,bootindex=0 -netdev tap,id=netdev1,vhost=on,script=/etc/qemu-ifup -device virtio-net-pci,netdev=netdev1,mac=02:03:04:05:06:00,id=net-pci1 -global PIIX4_PM.disable_s3=0 -global PIIX4_PM.disable_s4=0 -serial unix:/tmp/ttyS0,server,nowait"
cli="$cli -device virtio-scsi-pci,id=scsi0"
cli="$cli -device virtio-scsi-pci,id=scsi1"
count=$((${1:-1}-1))
for i in $(seq 0 $count)
do
j=$((2*$i))
echo $j
 cli="$cli -drive file=/home/disk/disk$j,if=none,id=disk$j"
 cli="$cli -device scsi-hd,bus=scsi0.0,drive=disk$j,id=target$j,scsi-id=$(($i%256)),lun=$(($i/256))"
 cli="$cli -drive file=/home/disk/disk$(($j+1)),if=none,id=disk$(($j+1))"
 cli="$cli -device scsi-hd,bus=scsi1.0,drive=disk$(($j+1)),id=target$(($j+1)),scsi-id=$(($i%256)),lun=$(($i/256))"
done
$cli
2.
3.

Actual results:
after step 1, guest fail to boot up if assigned more than 992 disks, just hang there before read SeaBIOS.

Numbers Status
 300*2----ok
 400*2----ok
 480*2----ok
 490*2----ok
 495*2----ok
 496*2----ok
 496*2+1----fail <-----------
 497*2----fail
 498*2----fail
 500*2----fail
 510*2----fail
 512*2----fail

Expected results:
it can boot up successfully and all the disk work well in guest.

Additional info:

Comment 3 RHEL Program Management 2013-11-03 08:35:06 UTC
This request was not resolved in time for the current release.
Red Hat invites you to ask your support representative to
propose this request, if still desired, for consideration in
the next release of Red Hat Enterprise Linux.

Comment 5 Fam Zheng 2015-01-26 10:52:36 UTC
qemu-kvm main loop won't work with > 1024 fds due to the limitation of underlying select(2) syscall interface, we should document that and also error out when it exceeds the limit.

Fam

Comment 6 Jeff Nelson 2015-02-18 17:53:51 UTC
Fix included in qemu-kvm-0.12.1.2-2.453.el6

Comment 8 Sibiao Luo 2015-02-27 06:53:31 UTC
(In reply to Fam Zheng from comment #5)
> qemu-kvm main loop won't work with > 1024 fds due to the limitation of
> underlying select(2) syscall interface, we should document that and also
> error out when it exceeds the limit.
> 
This fixed patch brought a new QEMU core dumped(Bug 1196955), re-assign it to fix it again, please correct me if any mistake, thanks.

Comment 9 Fam Zheng 2015-02-27 06:58:46 UTC
The only change here is the added assertion to avoid silent memory corruption. The limit will be fixed by either bz 1005016 or bz 1125735. Please verify this and don't duplicated more (1024 fd) bugs of them.

Comment 10 Sibiao Luo 2015-02-27 07:08:08 UTC
(In reply to Fam Zheng from comment #9)
> The only change here is the added assertion to avoid silent memory
> corruption. The limit will be fixed by either bz 1005016 or bz 1125735.
> Please verify this and don't duplicated more (1024 fd) bugs of them.

OK, thanks for your kindly reminds.

host info:
# uname -r && rpm -q qemu-kvm
2.6.32-538.el6.x86_64
qemu-kvm-0.12.1.2-2.454.el6.x86_64

(qemu) qemu-kvm: /builddir/build/BUILD/qemu-kvm-0.12.1.2/vl.c:4042: main_loop_wait: Assertion `ioh->fd < 1024' failed.
cli-random-disk.sh: line 15: 38376 Aborted                 (core dumped) $cli

Base on above and comment #9, this silent memory corruption issue has been fixed. Move to VERIFIED status, please correct me if any mistake, thanks.

Best Regards,
sluo

Comment 13 errata-xmlrpc 2015-07-22 06:04:25 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-1275.html