Bug 1599545 - libvirtd crashed randomly during start
Summary: libvirtd crashed randomly during start
Keywords:
Status: CLOSED ERRATA
Alias: None
Product: Red Hat Enterprise Linux 7
Classification: Red Hat
Component: libvirt
Version: 7.6
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: rc
: ---
Assignee: Michal Privoznik
QA Contact: Luyao Huang
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2018-07-10 03:57 UTC by Luyao Huang
Modified: 2018-10-30 09:58 UTC (History)
7 users (show)

Fixed In Version: libvirt-4.5.0-3.el7
Doc Type: If docs needed, set a value
Doc Text:
Clone Of:
Environment:
Last Closed: 2018-10-30 09:57:31 UTC
Target Upstream Version:


Attachments (Terms of Use)


Links
System ID Priority Status Summary Last Updated
Red Hat Product Errata RHSA-2018:3113 None None None 2018-10-30 09:58:30 UTC

Description Luyao Huang 2018-07-10 03:57:59 UTC
Description of problem:
libvirtd crashed randomly during start

Version-Release number of selected component (if applicable):
libvirt-4.5.0-2.el7.x86_64

How reproducible:
40%

Steps to Reproduce:
1. restart libvirtd on a host which have heavy workload
2.
3.

Actual results:

libvirtd crashed during start

Expected results:

No invalid memory access

Additional info:

valgrind:

==11847== For counts of detected and suppressed errors, rerun with: -v
==11847== ERROR SUMMARY: 23 errors from 23 contexts (suppressed: 0 from 0)
==11781== Thread 19:
==11781== Conditional jump or move depends on uninitialised value(s)
==11781==    at 0x4C2AC70: free (vg_replace_malloc.c:530)
==11781==    by 0x536F159: virFree (viralloc.c:582)
==11781==    by 0x54A1C33: virDomainConfNWFilterInstantiate (domain_nwfilter.c:122)
==11781==    by 0x321D4C05: qemuProcessFiltersInstantiate (qemu_process.c:3028)
==11781==    by 0x321D4C05: qemuProcessReconnect (qemu_process.c:7658)
==11781==    by 0x54208A4: virThreadHelper (virthread.c:206)
==11781==    by 0x8429DD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==11781==    by 0x873BEAC: clone (in /usr/lib64/libc-2.17.so)
==11781== 
==11781== Invalid free() / delete / delete[] / realloc()
==11781==    at 0x4C2ACBD: free (vg_replace_malloc.c:530)
==11781==    by 0x536F159: virFree (viralloc.c:582)
==11781==    by 0x54A1C33: virDomainConfNWFilterInstantiate (domain_nwfilter.c:122)
==11781==    by 0x321D4C05: qemuProcessFiltersInstantiate (qemu_process.c:3028)
==11781==    by 0x321D4C05: qemuProcessReconnect (qemu_process.c:7658)
==11781==    by 0x54208A4: virThreadHelper (virthread.c:206)
==11781==    by 0x8429DD4: start_thread (in /usr/lib64/libpthread-2.17.so)
==11781==    by 0x873BEAC: clone (in /usr/lib64/libc-2.17.so)
==11781==  Address 0x2bc7 is not stack'd, malloc'd or (recently) free'd
==11781== 

GDB:

#0  0x00007ffff428d59c in free () from /lib64/libc.so.6
#1  0x00007ffff721314a in virFree (ptrptr=ptrptr@entry=0x7fffc67f1b00) at util/viralloc.c:582
#2  0x00007ffff7345ac4 in virDomainConfNWFilterInstantiate (vmname=<optimized out>, vmuuid=vmuuid@entry=0x7fffc017e898 "߉\237\\۔H\262\206z\340\302f\265\233z", net=<optimized out>, 
    ignoreExists=ignoreExists@entry=true) at conf/domain_nwfilter.c:122
#3  0x00007fffca5a77f6 in qemuProcessFiltersInstantiate (ignoreExists=true, def=0x7fffc017e890) at qemu/qemu_process.c:3028
#4  qemuProcessReconnect (opaque=<optimized out>) at qemu/qemu_process.c:7653
#5  0x00007ffff72c4895 in virThreadHelper (data=<optimized out>) at util/virthread.c:206
#6  0x00007ffff45dcdd5 in start_thread () from /lib64/libpthread.so.0
#7  0x00007ffff4305ead in clone () from /lib64/libc.so.6

Comment 1 Luyao Huang 2018-07-10 04:03:50 UTC
This upstream commit should fix this issue:

commit d7557f5f6f51264c1b5623e97f7b4d5259d4d3b6
Author: Luyao Huang <lhuang@redhat.com>
Date:   Thu Jul 5 12:34:11 2018 +0800

    virDomainConfNWFilterInstantiate: initialize @xml to avoid random crash
    
    If the code jump to the cleanup before assigning value to @xml
    libvirtd may crash when it tries to free an uninitialized pointer.
    
    backtrace:
    
    0  0x00007ffff428d59c in free () from /lib64/libc.so.6
    1  0x00007ffff721314a in virFree (ptrptr=ptrptr@entry=0x7fffc67f1b00) at util/viralloc.c:582
    2  0x00007ffff7345ac4 in virDomainConfNWFilterInstantiate (vmname=<optimized out>,
       vmuuid=vmuuid@entry=0x7fffc0181ca8 "߉\237\\۔H\262\206z\340\302f\265\233z", net=<optimized out>,
       ignoreExists=ignoreExists@entry=true) at conf/domain_nwfilter.c:122
    3  0x00007fffca5a77f6 in qemuProcessFiltersInstantiate (ignoreExists=true, def=0x7fffc0181ca0) at qemu/qemu_process.c:3028
    4  qemuProcessReconnect (opaque=<optimized out>) at qemu/qemu_process.c:7653
    5  0x00007ffff72c4895 in virThreadHelper (data=<optimized out>) at util/virthread.c:206
    6  0x00007ffff45dcdd5 in start_thread () from /lib64/libpthread.so.0
    7  0x00007ffff4305ead in clone () from /lib64/libc.so.6
    
    Signed-off-by: Luyao Huang <lhuang@redhat.com>

v4.5.0-44-gd7557f5

Comment 5 Luyao Huang 2018-08-14 06:53:51 UTC
Verify this bug with libvirt-4.5.0-6.el7.x86_64:

1. use valgrind to start libvirtd cannot find invalid memory free error

2. restart libvirtd in a heavy workload host several times, no libvirtd crash happened

Comment 7 errata-xmlrpc 2018-10-30 09:57:31 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHSA-2018:3113


Note You need to log in before you can comment on or make changes to this bug.