Bug 1316420 - libvirtd crashed if set vcpusched vcpus over maxvcpu
Summary: libvirtd crashed if set vcpusched vcpus over maxvcpu
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.3
Hardware: x86_64
OS: Linux
high
high
Target Milestone: rc
: ---
Assignee: Peter Krempa
QA Contact: Virtualization Bugs
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2016-03-10 07:58 UTC by Luyao Huang
Modified: 2021-01-13 10:22 UTC (History)
3 users (show)

Fixed In Version: libvirt-1.3.3-1.el7
Doc Type: Bug Fix
Doc Text:
Clone Of:
Environment:
Last Closed: 2016-11-03 18:39:12 UTC
Target Upstream Version:

Attachments (Terms of Use)
Add an attachment (proposed patch, testcase, etc.)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2016:2577 normal SHIPPED_LIVE Moderate: libvirt security, bug fix, and enhancement update 2016-11-03 12:07:06 UTC

Description Luyao Huang 2016-03-10 07:58:57 UTC 
Description of problem:
libvirtd sometimes crashed if set vcpusched vcpus over maxvcpu

Version-Release number of selected component (if applicable):
libvirt-1.3.2-1.el7.x86_64

How reproducible:
80%

Steps to Reproduce:
1. edit xml like this:
  <vcpu placement='auto' current='3'>4</vcpu>
  <cputune>
    <vcpupin vcpu='2' cpuset='1'/>
    <vcpusched vcpus='0-4' scheduler='batch'/>
  </cputune>

2. try it again and again
# virsh edit  rhel7.0-rhel
error: Disconnected from qemu:///system due to I/O error
error: End of file while reading data: Input/output error
Failed. Try again? [y,n,i,f,?]: 

3.

Actual results:
libvirtd crashed when vcpusched vcpus over maxvcpu

Expected results:
not crashed

Additional info:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f543aa73700 (LWP 380)]
0x00007f544a447d1c in _int_malloc () from /lib64/libc.so.6
(gdb) bt
#0  0x00007f544a447d1c in _int_malloc () from /lib64/libc.so.6
#1  0x00007f544a44987c in malloc () from /lib64/libc.so.6
#2  0x00007f544be9fd55 in xmlXPathNewCompExpr () from /lib64/libxml2.so.2
#3  0x00007f544be9ffeb in xmlXPathTryStreamCompile () from /lib64/libxml2.so.2
#4  0x00007f544beb2f8b in xmlXPathEvalExpr () from /lib64/libxml2.so.2
#5  0x00007f544beb3082 in xmlXPathEval () from /lib64/libxml2.so.2
#6  0x00007f544d17e5d2 in virXPathNodeSet (xpath=xpath@entry=0x7f544d336c6a "./cputune/iothreadsched", ctxt=ctxt@entry=0x7f5420005770, list=list@entry=0x7f543aa72870) at util/virxml.c:586
#7  0x00007f544d1b1f36 in virDomainDefParseXML (xml=xml@entry=0x7f54200027d0, root=root@entry=0x7f5420005930, ctxt=ctxt@entry=0x7f5420005770, caps=caps@entry=0x7f54281de1d0, xmlopt=xmlopt@entry=0x7f54281ea5a0, 
    flags=flags@entry=642) at conf/domain_conf.c:15195
#8  0x00007f544d1b7910 in virDomainDefParseNode (xml=xml@entry=0x7f54200027d0, root=0x7f5420005930, caps=caps@entry=0x7f54281de1d0, xmlopt=xmlopt@entry=0x7f54281ea5a0, flags=flags@entry=642)
    at conf/domain_conf.c:16567
#9  0x00007f544d1b7a28 in virDomainDefParse (
    xmlStr=xmlStr@entry=0x7f5420002e30 "<domain type='kvm'>\n  <name>rhel7.0-rhel</name>\n  <uuid>67c7a123-5415-4136-af62-a2ee098ba6cd</uuid>\n  <memory unit='KiB'>4048000</memory>\n  <currentMemory unit='KiB'>4048000</currentMemory>\n  <vcpu pl"..., filename=filename@entry=0x0, caps=caps@entry=0x7f54281de1d0, xmlopt=0x7f54281ea5a0, flags=flags@entry=642) at conf/domain_conf.c:16514
#10 0x00007f544d1b7a70 in virDomainDefParseString (
    xmlStr=xmlStr@entry=0x7f5420002e30 "<domain type='kvm'>\n  <name>rhel7.0-rhel</name>\n  <uuid>67c7a123-5415-4136-af62-a2ee098ba6cd</uuid>\n  <memory unit='KiB'>4048000</memory>\n  <currentMemory unit='KiB'>4048000</currentMemory>\n  <vcpu pl"..., caps=caps@entry=0x7f54281de1d0, xmlopt=<optimized out>, flags=flags@entry=642) at conf/domain_conf.c:16529
#11 0x00007f54342ced8c in qemuDomainDefineXMLFlags (conn=0x7f54240009a0, 
    xml=0x7f5420002e30 "<domain type='kvm'>\n  <name>rhel7.0-rhel</name>\n  <uuid>67c7a123-5415-4136-af62-a2ee098ba6cd</uuid>\n  <memory unit='KiB'>4048000</memory>\n  <currentMemory unit='KiB'>4048000</currentMemory>\n  <vcpu pl"..., flags=<optimized out>) at qemu/qemu_driver.c:7386
#12 0x00007f544d215c1a in virDomainDefineXMLFlags (conn=0x7f54240009a0, 
    xml=0x7f5420002e30 "<domain type='kvm'>\n  <name>rhel7.0-rhel</name>\n  <uuid>67c7a123-5415-4136-af62-a2ee098ba6cd</uuid>\n  <memory unit='KiB'>4048000</memory>\n  <currentMemory unit='KiB'>4048000</currentMemory>\n  <vcpu pl"..., flags=1) at libvirt-domain.c:6430
#13 0x00007f544de5af3a in remoteDispatchDomainDefineXMLFlags (server=0x7f544eb6eea0, msg=0x7f544eb87560, ret=0x7f5420001370, args=0x7f5420002a90, rerr=0x7f543aa72c30, client=0x7f544eb875d0)
    at remote_dispatch.h:3894
#14 remoteDispatchDomainDefineXMLFlagsHelper (server=0x7f544eb6eea0, client=0x7f544eb875d0, msg=0x7f544eb87560, rerr=0x7f543aa72c30, args=0x7f5420002a90, ret=0x7f5420001370) at remote_dispatch.h:3872
#15 0x00007f544d27f1f2 in virNetServerProgramDispatchCall (msg=0x7f544eb87560, client=0x7f544eb875d0, server=0x7f544eb6eea0, prog=0x7f544eb83440) at rpc/virnetserverprogram.c:437
#16 virNetServerProgramDispatch (prog=0x7f544eb83440, server=server@entry=0x7f544eb6eea0, client=0x7f544eb875d0, msg=0x7f544eb87560) at rpc/virnetserverprogram.c:307
#17 0x00007f544d27a41d in virNetServerProcessMsg (msg=<optimized out>, prog=<optimized out>, client=<optimized out>, srv=0x7f544eb6eea0) at rpc/virnetserver.c:135
#18 virNetServerHandleJob (jobOpaque=<optimized out>, opaque=0x7f544eb6eea0) at rpc/virnetserver.c:156
#19 0x00007f544d172df5 in virThreadPoolWorker (opaque=opaque@entry=0x7f544eb51160) at util/virthreadpool.c:145
#20 0x00007f544d172318 in virThreadHelper (data=<optimized out>) at util/virthread.c:206
#21 0x00007f544a799dc5 in start_thread () from /lib64/libpthread.so.0
#22 0x00007f544a4c01cd in clone () from /lib64/libc.so.6


==1050== Invalid read of size 4
==1050==    at 0x552364E: virDomainThreadSchedParseHelper (domain_conf.c:14603)
==1050==    by 0x554DF05: virDomainVcpuThreadSchedParse (domain_conf.c:14626)
==1050==    by 0x554DF05: virDomainDefParseXML (domain_conf.c:15190)
==1050==    by 0x555390F: virDomainDefParseNode (domain_conf.c:16567)
==1050==    by 0x5553A27: virDomainDefParse (domain_conf.c:16514)
==1050==    by 0x1E891D8B: qemuDomainDefineXMLFlags (qemu_driver.c:7386)
==1050==    by 0x55B1C19: virDomainDefineXMLFlags (libvirt-domain.c:6430)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlags (remote_dispatch.h:3894)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlagsHelper (remote_dispatch.h:3872)
==1050==    by 0x561B1F1: virNetServerProgramDispatchCall (virnetserverprogram.c:437)
==1050==    by 0x561B1F1: virNetServerProgramDispatch (virnetserverprogram.c:307)
==1050==    by 0x561641C: virNetServerProcessMsg (virnetserver.c:135)
==1050==    by 0x561641C: virNetServerHandleJob (virnetserver.c:156)
==1050==    by 0x550EDF4: virThreadPoolWorker (virthreadpool.c:145)
==1050==    by 0x550E317: virThreadHelper (virthread.c:206)
==1050==    by 0x805ADC4: start_thread (in /usr/lib64/libpthread-2.17.so)
==1050==  Address 0x2360f9b0 is 16 bytes after a block of size 96 alloc'd
==1050==    at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==1050==    by 0x4C2BACB: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==1050==    by 0x54AAEAF: virReallocN (viralloc.c:245)
==1050==    by 0x54AAF79: virExpandN (viralloc.c:294)
==1050==    by 0x552B4C1: virDomainDefSetVcpusMax (domain_conf.c:1308)
==1050==    by 0x554BBCE: virDomainVcpuParse (domain_conf.c:14675)
==1050==    by 0x554BBCE: virDomainDefParseXML (domain_conf.c:15028)
==1050==    by 0x555390F: virDomainDefParseNode (domain_conf.c:16567)
==1050==    by 0x5553A27: virDomainDefParse (domain_conf.c:16514)
==1050==    by 0x1E891D8B: qemuDomainDefineXMLFlags (qemu_driver.c:7386)
==1050==    by 0x55B1C19: virDomainDefineXMLFlags (libvirt-domain.c:6430)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlags (remote_dispatch.h:3894)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlagsHelper (remote_dispatch.h:3872)
==1050==    by 0x561B1F1: virNetServerProgramDispatchCall (virnetserverprogram.c:437)
==1050==    by 0x561B1F1: virNetServerProgramDispatch (virnetserverprogram.c:307)
==1050== 
==1050== Invalid write of size 4
==1050==    at 0x5523658: virDomainThreadSchedParseHelper (domain_conf.c:14610)
==1050==    by 0x554DF05: virDomainVcpuThreadSchedParse (domain_conf.c:14626)
==1050==    by 0x554DF05: virDomainDefParseXML (domain_conf.c:15190)
==1050==    by 0x555390F: virDomainDefParseNode (domain_conf.c:16567)
==1050==    by 0x5553A27: virDomainDefParse (domain_conf.c:16514)
==1050==    by 0x1E891D8B: qemuDomainDefineXMLFlags (qemu_driver.c:7386)
==1050==    by 0x55B1C19: virDomainDefineXMLFlags (libvirt-domain.c:6430)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlags (remote_dispatch.h:3894)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlagsHelper (remote_dispatch.h:3872)
==1050==    by 0x561B1F1: virNetServerProgramDispatchCall (virnetserverprogram.c:437)
==1050==    by 0x561B1F1: virNetServerProgramDispatch (virnetserverprogram.c:307)
==1050==    by 0x561641C: virNetServerProcessMsg (virnetserver.c:135)
==1050==    by 0x561641C: virNetServerHandleJob (virnetserver.c:156)
==1050==    by 0x550EDF4: virThreadPoolWorker (virthreadpool.c:145)
==1050==    by 0x550E317: virThreadHelper (virthread.c:206)
==1050==    by 0x805ADC4: start_thread (in /usr/lib64/libpthread-2.17.so)
==1050==  Address 0x2360f9b0 is 16 bytes after a block of size 96 alloc'd
==1050==    at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==1050==    by 0x4C2BACB: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==1050==    by 0x54AAEAF: virReallocN (viralloc.c:245)
==1050==    by 0x54AAF79: virExpandN (viralloc.c:294)
==1050==    by 0x552B4C1: virDomainDefSetVcpusMax (domain_conf.c:1308)
==1050==    by 0x554BBCE: virDomainVcpuParse (domain_conf.c:14675)
==1050==    by 0x554BBCE: virDomainDefParseXML (domain_conf.c:15028)
==1050==    by 0x555390F: virDomainDefParseNode (domain_conf.c:16567)
==1050==    by 0x5553A27: virDomainDefParse (domain_conf.c:16514)
==1050==    by 0x1E891D8B: qemuDomainDefineXMLFlags (qemu_driver.c:7386)
==1050==    by 0x55B1C19: virDomainDefineXMLFlags (libvirt-domain.c:6430)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlags (remote_dispatch.h:3894)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlagsHelper (remote_dispatch.h:3872)
==1050==    by 0x561B1F1: virNetServerProgramDispatchCall (virnetserverprogram.c:437)
==1050==    by 0x561B1F1: virNetServerProgramDispatch (virnetserverprogram.c:307)
==1050== 
==1050== Invalid write of size 4
==1050==    at 0x552365A: virDomainThreadSchedParseHelper (domain_conf.c:14611)
==1050==    by 0x554DF05: virDomainVcpuThreadSchedParse (domain_conf.c:14626)
==1050==    by 0x554DF05: virDomainDefParseXML (domain_conf.c:15190)
==1050==    by 0x555390F: virDomainDefParseNode (domain_conf.c:16567)
==1050==    by 0x5553A27: virDomainDefParse (domain_conf.c:16514)
==1050==    by 0x1E891D8B: qemuDomainDefineXMLFlags (qemu_driver.c:7386)
==1050==    by 0x55B1C19: virDomainDefineXMLFlags (libvirt-domain.c:6430)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlags (remote_dispatch.h:3894)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlagsHelper (remote_dispatch.h:3872)
==1050==    by 0x561B1F1: virNetServerProgramDispatchCall (virnetserverprogram.c:437)
==1050==    by 0x561B1F1: virNetServerProgramDispatch (virnetserverprogram.c:307)
==1050==    by 0x561641C: virNetServerProcessMsg (virnetserver.c:135)
==1050==    by 0x561641C: virNetServerHandleJob (virnetserver.c:156)
==1050==    by 0x550EDF4: virThreadPoolWorker (virthreadpool.c:145)
==1050==    by 0x550E317: virThreadHelper (virthread.c:206)
==1050==    by 0x805ADC4: start_thread (in /usr/lib64/libpthread-2.17.so)
==1050==  Address 0x2360f9b4 is 20 bytes after a block of size 96 alloc'd
==1050==    at 0x4C29BFD: malloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==1050==    by 0x4C2BACB: realloc (in /usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so)
==1050==    by 0x54AAEAF: virReallocN (viralloc.c:245)
==1050==    by 0x54AAF79: virExpandN (viralloc.c:294)
==1050==    by 0x552B4C1: virDomainDefSetVcpusMax (domain_conf.c:1308)
==1050==    by 0x554BBCE: virDomainVcpuParse (domain_conf.c:14675)
==1050==    by 0x554BBCE: virDomainDefParseXML (domain_conf.c:15028)
==1050==    by 0x555390F: virDomainDefParseNode (domain_conf.c:16567)
==1050==    by 0x5553A27: virDomainDefParse (domain_conf.c:16514)
==1050==    by 0x1E891D8B: qemuDomainDefineXMLFlags (qemu_driver.c:7386)
==1050==    by 0x55B1C19: virDomainDefineXMLFlags (libvirt-domain.c:6430)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlags (remote_dispatch.h:3894)
==1050==    by 0x120F39: remoteDispatchDomainDefineXMLFlagsHelper (remote_dispatch.h:3872)
==1050==    by 0x561B1F1: virNetServerProgramDispatchCall (virnetserverprogram.c:437)
==1050==    by 0x561B1F1: virNetServerProgramDispatch (virnetserverprogram.c:307)
==1050==
Comment 1 Peter Krempa 2016-03-10 13:03:59 UTC 
Fixed upstream:

commit 8c7b7c4b0bb0d58dfb2e3dcdf1855a7dc9c858d0
Author: Peter Krempa <pkrempa@redhat.com>
Date:   Thu Mar 10 09:46:53 2016 +0100

    conf: Fix off-by-one in virDomainDefGetVcpu
    
    Cpus are indexed starting from '0' so the check was invalid.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1316384
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1316420

v1.3.2-101-g8c7b7c4
Comment 2 Mike McCune 2016-03-28 23:19:36 UTC 
This bug was accidentally moved from POST to MODIFIED via an error in automation, please see mmccune@redhat.com with any questions
Comment 4 Luyao Huang 2016-08-09 02:45:16 UTC 
verify this bug with libvirt-2.0.0-4.el7.x86_64:

1. open a terminal to run libvirtd under valgrind:

# valgrind --leak-check=full libvirtd
==1352== Memcheck, a memory error detector
==1352== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==1352== Using Valgrind-3.11.0 and LibVEX; rerun with -h for copyright info
==1352== Command: libvirtd

2. edit guest xml in another terminal:

# virsh edit r7

  <vcpu placement='auto' current='6'>10</vcpu>
  <cputune>
    <vcpupin vcpu='2' cpuset='1'/>
    <vcpusched vcpus='0-10' scheduler='batch'/>
  </cputune>

error: unsupported configuration: vCPU '10' is not present in domain definition
Failed. Try again? [y,n,i,f,?]: 

3. no invalid memory access in valgrind report and libvirtd not crash
Comment 6 errata-xmlrpc 2016-11-03 18:39:12 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-2577.html
Note You need to log in before you can comment on or make changes to this bug.