Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1371932

Summary: Can not attach 1024 scsi disks by libvirt although execute "ulimit -n 20000"
Product: Red Hat Enterprise Linux 7 Reporter: Jingjing Shao <jishao>
Component: libvirtAssignee: Libvirt Maintainers <libvirt-maint>
Status: CLOSED NOTABUG QA Contact: Virtualization Bugs <virt-bugs>
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.3CC: dyuan, jdenemar, rbalakri, xuzhang
Target Milestone: rc   
Target Release: ---   
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2016-08-31 14:17:12 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Jingjing Shao 2016-08-31 13:00:35 UTC
Description:
Can not attach 1024 scsi disks by libvirt although execute "ulimit -n 20000"

Version:
libvirt-2.0.0-6.el7.x86_64
qemu-kvm-rhev-2.6.0-22.el7.x86_64

setup:


Steps:
1. Prepare a shutdown guest,such as vm-jishao
# virsh list --all
 Id    Name                           State
----------------------------------------------------
 -     vm-jishao                      shut off

# virsh dumpxml vm-jishao
<domain type='kvm'>
  <name>vm-jishao</name>
  <uuid>9a077ad4-2de9-4c13-9c3c-abffc325b825</uuid>
  <memory unit='KiB'>8048576</memory>
  <currentMemory unit='KiB'>8048576</currentMemory>
  <vcpu placement='static'>4</vcpu>
....


2.Prepare below two templets
# cat scsi-disk-unit_1.xml
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/var/lib/libvirt/images/#IMAGE#'/>
<target dev='sd#ID#' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='#UNIT#'/>
</disk>

# cat scsi-disk-unit_2.xml
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/var/lib/libvirt/images/#IMAGE#'/>
<target dev='sd#ID#' bus='scsi'/>
<address type='drive' controller='1' bus='0' target='0' unit='#UNIT#'/>


3. Prepare below shell scripts add_disk.sh
#! /bin/sh -
for i in {1..512}
do
qemu-img create /var/lib/libvirt/images/disk_1-$i 10M
sed -e "s/#IMAGE#/disk_1-$i/g" -e "s/#UNIT#/$i/g" -e "s/#ID#/$i/g" scsi-disk-unit_1.xml > scsi-disks_1-$i.xml
virsh attach-device vm-jishao scsi-disks_1-$i.xml --persistent
done
for i in {1..512}
do
qemu-img create /var/lib/libvirt/images/disk_2-$i 10M
sed -e "s/#IMAGE#/disk_2-$i/g" -e "s/#UNIT#/$i/g" -e "s/#ID#/$(($i+512))/g" scsi-disk-unit_2.xml > scsi-disks_2-$i.xml
virsh attach-device vm-jishao scsi-disks_2-$i.xml --persistent
done

4.
# ulimit -n 20000
# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 514470
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 20000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 514470
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

5.Execute the shell scripts
# sh add_disk.sh

6.# virsh start vm-jishao
error: Failed to start domain vm-jishao
error: internal error: qemu unexpectedly closed the monitor: e-scsi1-0-0-390,id=scsi1-0-0-390 -drive file=/var/lib/libvirt/images/disk_2-391,format=raw,if=none,id=drive-scsi1-0-0-391 -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=391,drive=drive-scsi1-0-0-391,id=scsi1-0-0-391 -drive file=/var/lib/libvirt/images/disk_2-392,format=raw,if=none,id=drive-scsi1-0-0-392 -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=392,drive=drive-scsi1-0-0-392,id=scsi1-0-0-392 -drive file=/var/lib/libvirt/images/disk_2-393,format=raw,if=none,id=drive-scsi1-0-0-393 -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=393,drive=drive-scsi1-0-0-393,id=scsi1-0-0-393 -drive file=/var/lib/libvirt/images/disk_2-394,format=raw,if=none,id=drive-scsi1-0-0-394 -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=394,drive=drive-scsi1-0-0-394,id=scsi1-0-0-394 -drive file=/var/lib/libvirt/images/disk_2-395,format=raw,if=none,id=drive-scsi1-0-0-395 -device scsi-hd,bus=scsi1.0,channel=0,scsi-id=0,lun=395,drive=drive-scsi1-0-0-395,id=scsi1-0-0-395 -drive file

7.Check the /var/log/libvirt/qemu/vm-jishao.log
2016-08-31T12:46:14.880814Z qemu-kvm: -drive file=/var/lib/libvirt/images/disk_2-492,format=raw,if=none,id=drive-scsi1-0-0-492: Could not open '/var/lib/libvirt/images/disk_2-492': Too many open files
2016-08-31 12:46:14.924+0000: shutting down


Actual results
As the step 6 and 7

Expected results:
the guest should start successfully.

Additional info:
1.start the guest and hotplug disks 
it will get error when hotplug 925 disk
error: Failed to attach device from scsi-disks_2-413.xml
error: internal error: unable to execute QEMU command '__com.redhat_drive_add': Device 'drive-scsi1-0-0-413' could not be initialized


Check the /var/log/libvirt/qemu/vm-jishao.log
((null):33693): Spice-Warning **: reds.c:3041:reds_accept: accept failed, Too many open files

2.check it with qemu commend line 
it can attach 1024 scsi disks successfully

Comment 1 Jiri Denemark 2016-08-31 14:17:12 UTC
Setting limits in a shell has no impact on libvirtd, see /etc/sysconfig/libvirtd for steps you need to take to change the limits of libvrtd.

Comment 2 Jingjing Shao 2016-09-01 03:14:54 UTC
(In reply to Jiri Denemark from comment #1)
> Setting limits in a shell has no impact on libvirtd, see
> /etc/sysconfig/libvirtd for steps you need to take to change the limits of
> libvrtd.


Thanks Jiri, it works.
But I still have another question as below, If it should give a more clearly error info instead of guest'hang ?

1.Do not setting limits and keep it as default.

2. Prepare a guest as below 
# virsh dumpxml vm-jishao
<domain type='kvm'>
  <name>vm-jishao</name>
  <uuid>9a077ad4-2de9-4c13-9c3c-abffc325b825</uuid>
  <memory unit='KiB'>2048576</memory>
  <currentMemory unit='KiB'>2048576</currentMemory>
  <vcpu placement='static'>2</vcpu>
....
  <video>
      <model type='cirrus' vram='16384' heads='1' primary='yes'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
....

3.Prepare below two templets
# cat scsi-disk-unit_1.xml
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/var/lib/libvirt/images/#IMAGE#'/>
<target dev='sd#ID#' bus='scsi'/>
<address type='drive' controller='0' bus='0' target='0' unit='#UNIT#'/>
</disk>

# cat scsi-disk-unit_2.xml
<disk type='file' device='disk'>
<driver name='qemu' type='raw'/>
<source file='/var/lib/libvirt/images/#IMAGE#'/>
<target dev='sd#ID#' bus='scsi'/>
<address type='drive' controller='1' bus='0' target='0' unit='#UNIT#'/>


4. Prepare below shell scripts add_disk.sh
#! /bin/sh -
for i in {1..512}
do
qemu-img create /var/lib/libvirt/images/disk_1-$i 10M
sed -e "s/#IMAGE#/disk_1-$i/g" -e "s/#UNIT#/$i/g" -e "s/#ID#/$i/g" scsi-disk-unit_1.xml > scsi-disks_1-$i.xml
virsh attach-device vm-jishao scsi-disks_1-$i.xml --persistent
done
for i in {1..487}
do
qemu-img create /var/lib/libvirt/images/disk_2-$i 10M
sed -e "s/#IMAGE#/disk_2-$i/g" -e "s/#UNIT#/$i/g" -e "s/#ID#/$(($i+512))/g" scsi-disk-unit_2.xml > scsi-disks_2-$i.xml
virsh attach-device vm-jishao scsi-disks_2-$i.xml --persistent
done

4.Execute the shell scripts
# sh add_disk.sh

5.# # virsh start vm-jishao
^C


# virsh list --all
 Id    Name                           State
----------------------------------------------------
 1570  vm-jishao                      paused


6.Check the /var/log/libvirt/qemu/vm-jishao.log
it does not print anything

Comment 3 Jiri Denemark 2016-09-05 11:22:47 UTC
It probably should fail and the domain should remain running, but it's hard to see what happened without any logs...