Note: This bug is displayed in read-only format because the product is no longer active in Red Hat Bugzilla.
RHEL Engineering is moving the tracking of its product development work on RHEL 6 through RHEL 9 to Red Hat Jira (issues.redhat.com). If you're a Red Hat customer, please continue to file support cases via the Red Hat customer portal. If you're not, please head to the "RHEL project" in Red Hat Jira and file new tickets here. Individual Bugzilla bugs in the statuses "NEW", "ASSIGNED", and "POST" are being migrated throughout September 2023. Bugs of Red Hat partners with an assigned Engineering Partner Manager (EPM) are migrated in late September as per pre-agreed dates. Bugs against components "kernel", "kernel-rt", and "kpatch" are only migrated if still in "NEW" or "ASSIGNED". If you cannot log in to RH Jira, please consult article #7032570. That failing, please send an e-mail to the RH Jira admins at rh-issues@redhat.com to troubleshoot your issue as a user management inquiry. The email creates a ServiceNow ticket with Red Hat. Individual Bugzilla bugs that are migrated will be moved to status "CLOSED", resolution "MIGRATED", and set with "MigratedToJIRA" in "Keywords". The link to the successor Jira issue will be found under "Links", have a little "two-footprint" icon next to it, and direct you to the "RHEL project" in Red Hat Jira (issue links are of type "https://issues.redhat.com/browse/RHEL-XXXX", where "X" is a digit). This same link will be available in a blue banner at the top of the page informing you that that bug has been migrated.

Bug 1116754

Summary: [RHEL6.6]libvirtd will crash after create two network by virsh command
Product: Red Hat Enterprise Linux 6 Reporter: Luyao Huang <lhuang>
Component: libvirtAssignee: Ján Tomko <jtomko>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: high Docs Contact:
Priority: high    
Version: 6.6CC: dyuan, honzhang, jiahu, jtomko, lhuang, mzhan, pkrempa, rbalakri, zpeng
Target Milestone: rc   
Target Release: 6.6   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-0.10.2-41.el6 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2014-10-14 04:23:02 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Attachments:
Description Flags
libvirt debug log none

Description Luyao Huang 2014-07-07 08:50:49 UTC 
Created attachment 916000 [details]
libvirt debug log

Description of problem:
libvirtd will crash after create two network by virsh command

Version-Release number of selected component (if applicable):
libvirt-0.10.2-40.el6.x86_64
Kernel-2.6.32-486.el6.x86_64


How reproducible:
100%

Steps to Reproduce:
1.prepare two XML files like this.
#vi net4.xml
<network>
  <name>net4</name>
  <forward mode='nat'/>
  <bridge name='virbr13' stp='on' delay='0' />
  <ip address='192.168.13.13' netmask='255.255.255.0'>
  </ip>
</network>
#vi net3.xml
<network>
  <name>net3</name>
  <forward mode='nat'/>
  <bridge name='virbr388' stp='on' delay='0' />
  <ip address='192.168.13.1' netmask='255.255.255.0'>
  </ip>
</network>

2.use net-create command :
[root@lhuang1 ~]# virsh net-create net3.xml
Network net3 created from net3.xml

[root@lhuang1 ~]# virsh net-create net4.xml
error: Failed to create network from net4.xml
error: End of file while reading data: Input/output error
error: One or more references were leaked after disconnect from the hypervisor
error: Failed to reconnect to the hypervisor

3.check libvirtd status
[root@lhuang1 ~]# service libvirtd status
libvirtd dead but pid file exists


Actual results:
libvirtd crashed

Expected results:
libvirtd should not crash and report some error like:
error: Failed to create network from net4.xml
error: internal error: Network is already in use by interface virbr13


Additional info:
Attachment is the libvirt debug log
Comment 2 Peter Krempa 2014-07-07 08:53:06 UTC 
Please attach the backtrace of all threads of the crashed libvirt daemon.
Comment 3 Luyao Huang 2014-07-07 09:34:14 UTC 
(gdb) thread apply all bt

Thread 11 (Thread 0x7fffec317700 (LWP 4255)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#1  0x0000003da3864246 in virCondWait (c=<value optimized out>, 
    m=<value optimized out>) at util/threads-pthread.c:117
#2  0x0000003da3864813 in virThreadPoolWorker (opaque=<value optimized out>)
    at util/threadpool.c:103
#3  0x0000003da3864069 in virThreadHelper (data=<value optimized out>)
    at util/threads-pthread.c:161
#4  0x00000036636079d1 in start_thread (arg=0x7fffec317700)
    at pthread_create.c:301
#5  0x00000036632e8b6d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 10 (Thread 0x7fffecd18700 (LWP 4254)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#1  0x0000003da3864246 in virCondWait (c=<value optimized out>, 
    m=<value optimized out>) at util/threads-pthread.c:117
#2  0x0000003da3864813 in virThreadPoolWorker (opaque=<value optimized out>)
    at util/threadpool.c:103
#3  0x0000003da3864069 in virThreadHelper (data=<value optimized out>)
---Type <return> to continue, or q <return> to quit---
    at util/threads-pthread.c:161
#4  0x00000036636079d1 in start_thread (arg=0x7fffecd18700)
    at pthread_create.c:301
#5  0x00000036632e8b6d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 9 (Thread 0x7fffed719700 (LWP 4253)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#1  0x0000003da3864246 in virCondWait (c=<value optimized out>, 
    m=<value optimized out>) at util/threads-pthread.c:117
#2  0x0000003da3864813 in virThreadPoolWorker (opaque=<value optimized out>)
    at util/threadpool.c:103
#3  0x0000003da3864069 in virThreadHelper (data=<value optimized out>)
    at util/threads-pthread.c:161
#4  0x00000036636079d1 in start_thread (arg=0x7fffed719700)
    at pthread_create.c:301
#5  0x00000036632e8b6d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 8 (Thread 0x7fffee11a700 (LWP 4252)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
---Type <return> to continue, or q <return> to quit---
#1  0x0000003da3864246 in virCondWait (c=<value optimized out>, 
    m=<value optimized out>) at util/threads-pthread.c:117
#2  0x0000003da3864813 in virThreadPoolWorker (opaque=<value optimized out>)
    at util/threadpool.c:103
#3  0x0000003da3864069 in virThreadHelper (data=<value optimized out>)
    at util/threads-pthread.c:161
#4  0x00000036636079d1 in start_thread (arg=0x7fffee11a700)
    at pthread_create.c:301
#5  0x00000036632e8b6d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 7 (Thread 0x7fffeeb1b700 (LWP 4251)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#1  0x0000003da3864246 in virCondWait (c=<value optimized out>, 
    m=<value optimized out>) at util/threads-pthread.c:117
#2  0x0000003da3864813 in virThreadPoolWorker (opaque=<value optimized out>)
    at util/threadpool.c:103
#3  0x0000003da3864069 in virThreadHelper (data=<value optimized out>)
    at util/threads-pthread.c:161
#4  0x00000036636079d1 in start_thread (arg=0x7fffeeb1b700)
    at pthread_create.c:301
#5  0x00000036632e8b6d in clone ()
---Type <return> to continue, or q <return> to quit---
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 6 (Thread 0x7fffef51c700 (LWP 4250)):
#0  0x00000000004f24ca in networkShutdownNetwork (driver=0x7fffe40008c0, 
    network=0x7fffcc000920) at network/bridge_driver.c:2529
#1  0x00000000004f4e47 in networkStartNetwork (driver=0x7fffe40008c0, 
    network=0x7fffcc000920) at network/bridge_driver.c:2515
#2  0x00000000004f5c70 in networkCreate (conn=0x7fffd4000c00, 
    xml=<value optimized out>) at network/bridge_driver.c:2949
#3  0x0000003da38ed386 in virNetworkCreateXML (conn=0x7fffd4000c00, 
    xmlDesc=0x7fffcc000a10 "<network>\n  <name>net5</name>\n  <forward mode='nat'/>\n  <bridge name='virbr155' stp='on' delay='0' />\n  <ip address='192.168.155.13' netmask='255.255.255.0'>\n  </ip>\n</network>\n\n") at libvirt.c:10463
#4  0x000000000043df7e in remoteDispatchNetworkCreateXML (
    server=<value optimized out>, client=0x7a77f0, msg=<value optimized out>, 
    rerr=0x7fffef51bb80, args=0x7fffcc000970, ret=0x7fffcc0008c0)
    at remote_dispatch.h:8717
#5  remoteDispatchNetworkCreateXMLHelper (server=<value optimized out>, 
    client=0x7a77f0, msg=<value optimized out>, rerr=0x7fffef51bb80, 
    args=0x7fffcc000970, ret=0x7fffcc0008c0) at remote_dispatch.h:8697
#6  0x0000003da3942a82 in virNetServerProgramDispatchCall (prog=0x7a1a30, 
    server=0x79f320, client=0x7a77f0, msg=0x7a7e30)
    at rpc/virnetserverprogram.c:431
---Type <return> to continue, or q <return> to quit---
#7  virNetServerProgramDispatch (prog=0x7a1a30, server=0x79f320, 
    client=0x7a77f0, msg=0x7a7e30) at rpc/virnetserverprogram.c:304
#8  0x0000003da3944fde in virNetServerProcessMsg (srv=<value optimized out>, 
    client=0x7a77f0, prog=<value optimized out>, msg=0x7a7e30)
    at rpc/virnetserver.c:170
#9  0x0000003da394567c in virNetServerHandleJob (
    jobOpaque=<value optimized out>, opaque=0x79f320) at rpc/virnetserver.c:191
#10 0x0000003da386477c in virThreadPoolWorker (opaque=<value optimized out>)
    at util/threadpool.c:144
#11 0x0000003da3864069 in virThreadHelper (data=<value optimized out>)
    at util/threads-pthread.c:161
#12 0x00000036636079d1 in start_thread (arg=0x7fffef51c700)
    at pthread_create.c:301
#13 0x00000036632e8b6d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 5 (Thread 0x7fffeff1d700 (LWP 4249)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#1  0x0000003da3864246 in virCondWait (c=<value optimized out>, 
    m=<value optimized out>) at util/threads-pthread.c:117
#2  0x0000003da3864813 in virThreadPoolWorker (opaque=<value optimized out>)
    at util/threadpool.c:103
---Type <return> to continue, or q <return> to quit---
#3  0x0000003da3864069 in virThreadHelper (data=<value optimized out>)
    at util/threads-pthread.c:161
#4  0x00000036636079d1 in start_thread (arg=0x7fffeff1d700)
    at pthread_create.c:301
#5  0x00000036632e8b6d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 4 (Thread 0x7ffff091e700 (LWP 4248)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#1  0x0000003da3864246 in virCondWait (c=<value optimized out>, 
    m=<value optimized out>) at util/threads-pthread.c:117
#2  0x0000003da3864813 in virThreadPoolWorker (opaque=<value optimized out>)
    at util/threadpool.c:103
#3  0x0000003da3864069 in virThreadHelper (data=<value optimized out>)
    at util/threads-pthread.c:161
#4  0x00000036636079d1 in start_thread (arg=0x7ffff091e700)
    at pthread_create.c:301
#5  0x00000036632e8b6d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 3 (Thread 0x7ffff131f700 (LWP 4247)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
---Type <return> to continue, or q <return> to quit---
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#1  0x0000003da3864246 in virCondWait (c=<value optimized out>, 
    m=<value optimized out>) at util/threads-pthread.c:117
#2  0x0000003da3864813 in virThreadPoolWorker (opaque=<value optimized out>)
    at util/threadpool.c:103
#3  0x0000003da3864069 in virThreadHelper (data=<value optimized out>)
    at util/threads-pthread.c:161
#4  0x00000036636079d1 in start_thread (arg=0x7ffff131f700)
    at pthread_create.c:301
#5  0x00000036632e8b6d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 2 (Thread 0x7ffff1d20700 (LWP 4246)):
#0  pthread_cond_wait@@GLIBC_2.3.2 ()
    at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:183
#1  0x0000003da3864246 in virCondWait (c=<value optimized out>, 
    m=<value optimized out>) at util/threads-pthread.c:117
#2  0x0000003da3864813 in virThreadPoolWorker (opaque=<value optimized out>)
    at util/threadpool.c:103
#3  0x0000003da3864069 in virThreadHelper (data=<value optimized out>)
    at util/threads-pthread.c:161
#4  0x00000036636079d1 in start_thread (arg=0x7ffff1d20700)
    at pthread_create.c:301
---Type <return> to continue, or q <return> to quit---
#5  0x00000036632e8b6d in clone ()
    at ../sysdeps/unix/sysv/linux/x86_64/clone.S:115

Thread 1 (Thread 0x7ffff7fd3860 (LWP 4243)):
#0  0x00000036632df343 in __poll (fds=<value optimized out>, 
    nfds=<value optimized out>, timeout=<value optimized out>)
    at ../sysdeps/unix/sysv/linux/poll.c:87
#1  0x0000003da3851bac in virEventPollRunOnce () at util/event_poll.c:615
#2  0x0000003da3850de7 in virEventRunDefaultImpl () at util/event.c:247
#3  0x0000003da394481d in virNetServerRun (srv=0x79f320)
    at rpc/virnetserver.c:748
#4  0x0000000000424237 in main (argc=<value optimized out>, 
    argv=<value optimized out>) at libvirtd.c:1229





I am sorry that i have forgotten backtrace.
Comment 4 Ján Tomko 2014-07-07 11:08:01 UTC 
Fixed upstream by:
commit 947230fb56325c86d318215a3e6c9cdb379dc966
Author:     Peter Krempa <pkrempa>
CommitDate: 2012-11-02 13:28:39 +0100

    conf: net: Fix helper for applying new network definition
    
    When there's no new definition the helper overwrote the old one with
    NULL.

git describe: v1.0.0-5-g947230f contains: v1.0.1-rc1~312
Comment 7 Hu Jianwei 2014-07-18 06:54:01 UTC 
I can not reproduce it on below version.

Version:
libvirt-0.10.2-41.el6.x86_64
qemu-kvm-rhev-0.12.1.2-2.428.el6.x86_64

[root@rhel6 ~]# cat net3.xml 
<network>
<name>net3</name>
<forward mode='nat'/>
<bridge name='virbr388' stp='on' delay='0' />
<ip address='192.168.13.1' netmask='255.255.255.0'>
         <dhcp>
             <range start='192.168.13.128' end='192.168.13.254'/>
          </dhcp>
</ip>
</network>
[root@rhel6 ~]# cat net4.xml 
<network>
<name>net4</name>
<forward mode='nat'/>
<bridge name='virbr13' stp='on' delay='0' />
<ip address='192.168.13.30' netmask='255.255.255.0'>
         <dhcp>
              <range start='192.168.13.12' end='192.168.13.25'/>
          </dhcp>
</ip>
</network>

[root@rhel6 ~]# virsh net-create net3.xml
Network net3 created from net3.xml

[root@rhel6 ~]# virsh net-create net4.xml
error: Failed to create network from net4.xml
error: internal error Network is already in use by interface virbr388

[root@rhel6 ~]# service libvirtd status
libvirtd (pid  14389) is running...

[root@rhel6 ~]# virsh net-list --all
Name                 State      Autostart     Persistent
--------------------------------------------------
default              active     yes           yes
net3                 active     no            no

[root@rhel6 ~]# virsh list --all
 Id    Name                           State
----------------------------------------------------
 6     r7                             running

Libvirtd is running as usual, so changed to Verified.
Comment 9 errata-xmlrpc 2014-10-14 04:23:02 UTC 
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

http://rhn.redhat.com/errata/RHBA-2014-1374.html