Bug 1322661

Summary: The max number of vCPUs KVM supports is 256 currently, so libvirt needs to increase the open file descriptor limit
Product: Red Hat Enterprise Linux 7 Reporter: yalzhang <yalzhang>
Component: doc-Virtualization_Tuning_and_Optimization_GuideAssignee: Jiri Herrmann <jherrman>
Status: CLOSED CURRENTRELEASE QA Contact: ecs-bugs
Severity: unspecified Docs Contact:
Priority: unspecified    
Version: 7.2CC: dyuan, ecs-bugs, jasowang, jdenemar, jherrman, jsuchane, juzhang, laine, mst, mzhan, rbalakri, rhel-docs, yalzhang
Target Milestone: rcKeywords: Documentation
Target Release: ---Flags: jherrman: needinfo-
Hardware: Unspecified   
OS: Unspecified   
Whiteboard:
Fixed In Version: Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: 1308395 Environment:
Last Closed: 2016-11-07 17:22:20 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Bug Depends On: 1308395    
Bug Blocks:    

Comment 2 yalzhang@redhat.com 2016-04-07 03:29:48 UTC 
Sorry, I cloned this bug and did not modify it. Modified it as below and change the component to libvirt:
+++ This bug was initially created as a clone of Bug #1308395 +++

Version-Release number of selected component (if applicable):
Linux work 3.10.0-342.el7.x86_64 #1 SMP Thu Jan 14 18:52:04 EST 2016 x86_64 x86_64 x86_64 GNU/Linux
libvirt-1.3.3-1.el7.x86_64
qemu-kvm-rhev-2.3.0-31.el7_2.5.x86_64

Describe the issue: 
For virt-io multiqueue support, current kernel supports up to 256 rx queues, but when set to the "queues=251" or more, the guest fail to start. May be libvirt needs to increasethe open file descriptor limit (also known as "nofile" limit) for qemu by the number of queues.

How reproducible:
100%

Steps to Reproduce:
1.
# virsh dumpxml rhel7.2 | grep /interface -B4
    ...
      <driver name='vhost' queues='251'/>
      <address type='pci' domain='0x0000' bus='0x01' slot='0x01' function='0x0'/>
    </interface>
# virsh start rhel7.2
error: Failed to start domain rhel7.2
error: internal error: early end of file from monitor, possible problem: ((null):14879): SpiceDispatcher-ERROR **: dispatcher.c:260:dispatcher_init: socketpair failed Too many open files

when I set to 255, the guest can not start.
# virsh start rhel7.2
error: Failed to start domain rhel7.2
error: internal error: early end of file from monitor, possible problem: 2016-03-30T11:40:53.794624Z qemu-kvm: -netdev tap,fds=25:27:28:29:30:31:32:33:34:35:36:37:38:39:40:41:42:43:44:45:46:47:48:49:50:51:52:53:54:55:56:57:58:59:60:61:62:63:64:65:66:67:68:69:70:71:72:73:74:75:76:77:78:79:80:81:82:83:84:85:86:87:88:89:90:91:92:93:94:95:96:97:98:99:100:101:102:103:104:105:106:107:108:109:110:111:112:113:114:115:116:117:118:119:120:121:122:123:124:125:126:127:128:129:130:131:132:133:134:135:136:137:138:139:140:141:142:143:144:145:146:147:148:149:150:151:152:153:154:155:156:157:158:159:160:161:162:163:164:165:166:167:168:169:170:171:172:173:174:175:176:177:178:180:179:181:182:183:184:185:186:187:188:189:190:191:193:192:194:195:196:197:198:199:200:201:202:203:204:205:206:207:208:209:210:211:212:213:214:215:216:218:217:219:220:221:222:223:224:225:226:227:228:229:230:231:232:233:234:235:236:237:238:239:240:241:242:243:244:245:246:247:248:249:250:251:252:253:254:255:256:257:258:259:260:261:262:263:264:265:266:267:268:269:270:271:272:273:
Comment 3 Jaroslav Suchanek 2016-04-07 14:25:06 UTC 
Well, I moved it under documentation component, as I believe that number of open files is controlled by max_files option in the /etc/libvirt/qemu.conf configuration file and therefore it makes sense to document this specific use case in the related chapter.

Jirka, please confirm. Thanks.
Comment 4 Jiri Denemark 2016-04-07 15:05:40 UTC 
Yeah, I think documenting it is probably the right way to address this issue at this point.

Another thing is we could possibly change the limit ourselves based on domain XML, but that's a more complex task not limited to multiqueue. For example, we don't do it for disks either.