Bug 1299700 - libvirtd crashes when defining a network with incorrect forward mode
libvirtd crashes when defining a network with incorrect forward mode
Status: CLOSED ERRATA
Product: Red Hat Enterprise Linux 6
Classification: Red Hat
Component: libvirt (Show other bugs)
6.8
x86_64 Unspecified
medium Severity medium
: rc
: ---
Assigned To: Ján Tomko
Virtualization Bugs
:
Depends On:
Blocks:
  Show dependency treegraph
 
Reported: 2016-01-18 22:47 EST by Jingjing Shao
Modified: 2016-05-10 15:25 EDT (History)
5 users (show)

See Also:
Fixed In Version: libvirt-0.10.2-56.el6
Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of:
Environment:
Last Closed: 2016-05-10 15:25:54 EDT
Type: Bug
Regression: ---
Mount Type: ---
Documentation: ---
CRM:
Verified Versions:
Category: ---
oVirt Team: ---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: ---


Attachments (Terms of Use)

  None (edit)
Description Jingjing Shao 2016-01-18 22:47:46 EST
PKG
 OS-rhel6.7
 libvirt-0.10.2-54.el6_7.3  or libvirt-0.10.2-55.el6
 qemu-kvm-rhev-0.12.1.2-2.481.el6.x86_64

 Setup
 1. modprobe -r kvm_intel
 2. modprobe -r kvm
 3. modprobe kvm allow_unsafe_assigned_interrupts=1
 4. modprobe kvm_intel

 Steps to Reproduce:
 1.[root@sriov1 jishao]# lspci | grep 82576
 03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
 03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
 03:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
 03:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
 03:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
 03:10.3 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)

 2.[root@sriov1 jishao]# virsh nodedev-list --tree
 computer
   |
   +- net_lo_00_00_00_00_00_00
   +- net_virbr1_nic_52_54_00_3c_df_9b
   +- net_virbr2_nic_52_54_00_3d_09_3d
   +- pci_0000_00_00_0
   +- pci_0000_00_01_0
   |   |
   |   +- pci_0000_03_00_0
   |   |   |
   |   |   +- net_eth0_00_1b_21_39_8b_18
   |   |    
   |   +- pci_0000_03_00_1
   |   |   |
   |   |   +- net_eth1_00_1b_21_39_8b_19
   |   |    
   |   +- pci_0000_03_10_0
   |   |   |
   |   |   +- net_eth5_7e_53_1e_e7_47_fc
   |   |    
   |   +- pci_0000_03_10_1
   |   |   |
   |   |   +- net_eth3_3a_d5_75_ca_f0_86

 3.[root@sriov1 jishao]# virsh nodedev-dumpxml pci_0000_03_00_0
 <device>
   <name>pci_0000_03_00_0</name>
   <parent>pci_0000_00_01_0</parent>
   <driver>
     <name>igb</name>
   </driver>
   <capability type='pci'>
     <domain>0</domain>
     <bus>3</bus>
     <slot>0</slot>
     <function>0</function>
     <product id='0x10c9'>82576 Gigabit Network Connection</product>
     <vendor id='0x8086'>Intel Corporation</vendor>
     <capability type='virt_functions'>
       <address domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
       <address domain='0x0000' bus='0x03' slot='0x10' function='0x2'/>
     </capability>
   </capability>
 </device>


 4.[root@sriov1 jishao]# cat hostdev.xml
 <network>
   <name>hostdev</name>
    <ip address='10.66.4.135' prefix='24'/>
       <forward mod='hostdev' managed='yes'>
       <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
    </forward>
 </network>


 5.#virsh net-define hostdev.xml

 Then the libvirtd crash


 Additional info:
 the gdb log 
Program received signal SIGSEGV, Segmentation fault.
0x0000003701b336ef in __strlen_sse42 () from /lib64/libc.so.6
(gdb) t a a bt

Thread 11 (Thread 0x7fef79571700 (LWP 13208)):
#0  0x0000003701e0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00000032e3e65846 in virCondWait () from /usr/lib64/libvirt.so.0
#2  0x00000032e3e65e13 in ?? () from /usr/lib64/libvirt.so.0
#3  0x00000032e3e65669 in ?? () from /usr/lib64/libvirt.so.0
#4  0x0000003701e07a51 in start_thread () from /lib64/libpthread.so.0
#5  0x0000003701ae896d in clone () from /lib64/libc.so.6

Thread 10 (Thread 0x7fef78b70700 (LWP 13209)):
#0  0x0000003701e0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00000032e3e65846 in virCondWait () from /usr/lib64/libvirt.so.0
#2  0x00000032e3e65e13 in ?? () from /usr/lib64/libvirt.so.0
#3  0x00000032e3e65669 in ?? () from /usr/lib64/libvirt.so.0
#4  0x0000003701e07a51 in start_thread () from /lib64/libpthread.so.0
#5  0x0000003701ae896d in clone () from /lib64/libc.so.6

Thread 9 (Thread 0x7fef7816f700 (LWP 13210)):
#0  0x0000003701b336ef in __strlen_sse42 () from /lib64/libc.so.6
#1  0x00000032e3e48f11 in virBufferEscapeString () from /usr/lib64/libvirt.so.0
#2  0x00000032e3eb4ec9 in virNetworkDefFormatBuf () from /usr/lib64/libvirt.so.0
#3  0x00000032e3eb5caa in virNetworkDefFormat () from /usr/lib64/libvirt.so.0
#4  0x00000032e3eb5d91 in virNetworkSaveConfig () from /usr/lib64/libvirt.so.0
#5  0x00000000004f4a42 in ?? ()
#6  0x00000032e3ef01f6 in virNetworkDefineXML () from /usr/lib64/libvirt.so.0
#7  0x000000000043e0ee in ?? ()
#8  0x00000032e3f45e12 in virNetServerProgramDispatch () from /usr/lib64/libvirt.so.0
#9  0x00000032e3f470fe in ?? () from /usr/lib64/libvirt.so.0
#10 0x00000032e3f4779c in ?? () from /usr/lib64/libvirt.so.0
#11 0x00000032e3e65d7c in ?? () from /usr/lib64/libvirt.so.0
---Type <return> to continue, or q <return> to quit---
#12 0x00000032e3e65669 in ?? () from /usr/lib64/libvirt.so.0
#13 0x0000003701e07a51 in start_thread () from /lib64/libpthread.so.0
#14 0x0000003701ae896d in clone () from /lib64/libc.so.6

Thread 8 (Thread 0x7fef7776e700 (LWP 13211)):
#0  0x0000003701e0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00000032e3e65846 in virCondWait () from /usr/lib64/libvirt.so.0
#2  0x00000032e3e65e13 in ?? () from /usr/lib64/libvirt.so.0
#3  0x00000032e3e65669 in ?? () from /usr/lib64/libvirt.so.0
#4  0x0000003701e07a51 in start_thread () from /lib64/libpthread.so.0
#5  0x0000003701ae896d in clone () from /lib64/libc.so.6

Thread 7 (Thread 0x7fef76d6d700 (LWP 13212)):
#0  0x0000003701e0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00000032e3e65846 in virCondWait () from /usr/lib64/libvirt.so.0
#2  0x00000032e3e65e13 in ?? () from /usr/lib64/libvirt.so.0
#3  0x00000032e3e65669 in ?? () from /usr/lib64/libvirt.so.0
#4  0x0000003701e07a51 in start_thread () from /lib64/libpthread.so.0
#5  0x0000003701ae896d in clone () from /lib64/libc.so.6

Thread 6 (Thread 0x7fef7636c700 (LWP 13213)):
#0  0x0000003701e0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00000032e3e65846 in virCondWait () from /usr/lib64/libvirt.so.0
#2  0x00000032e3e65e13 in ?? () from /usr/lib64/libvirt.so.0
#3  0x00000032e3e65669 in ?? () from /usr/lib64/libvirt.so.0
#4  0x0000003701e07a51 in start_thread () from /lib64/libpthread.so.0
#5  0x0000003701ae896d in clone () from /lib64/libc.so.6

Thread 5 (Thread 0x7fef7596b700 (LWP 13214)):
#0  0x0000003701e0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
---Type <return> to continue, or q <return> to quit---
#1  0x00000032e3e65846 in virCondWait () from /usr/lib64/libvirt.so.0
#2  0x00000032e3e65e13 in ?? () from /usr/lib64/libvirt.so.0
#3  0x00000032e3e65669 in ?? () from /usr/lib64/libvirt.so.0
#4  0x0000003701e07a51 in start_thread () from /lib64/libpthread.so.0
#5  0x0000003701ae896d in clone () from /lib64/libc.so.6

Thread 4 (Thread 0x7fef74f6a700 (LWP 13215)):
#0  0x0000003701e0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00000032e3e65846 in virCondWait () from /usr/lib64/libvirt.so.0
#2  0x00000032e3e65e13 in ?? () from /usr/lib64/libvirt.so.0
#3  0x00000032e3e65669 in ?? () from /usr/lib64/libvirt.so.0
#4  0x0000003701e07a51 in start_thread () from /lib64/libpthread.so.0
#5  0x0000003701ae896d in clone () from /lib64/libc.so.6

Thread 3 (Thread 0x7fef74569700 (LWP 13216)):
#0  0x0000003701e0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00000032e3e65846 in virCondWait () from /usr/lib64/libvirt.so.0
#2  0x00000032e3e65e13 in ?? () from /usr/lib64/libvirt.so.0
#3  0x00000032e3e65669 in ?? () from /usr/lib64/libvirt.so.0
#4  0x0000003701e07a51 in start_thread () from /lib64/libpthread.so.0
#5  0x0000003701ae896d in clone () from /lib64/libc.so.6

Thread 2 (Thread 0x7fef73b68700 (LWP 13217)):
#0  0x0000003701e0b63c in pthread_cond_wait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1  0x00000032e3e65846 in virCondWait () from /usr/lib64/libvirt.so.0
#2  0x00000032e3e65e13 in ?? () from /usr/lib64/libvirt.so.0
#3  0x00000032e3e65669 in ?? () from /usr/lib64/libvirt.so.0
#4  0x0000003701e07a51 in start_thread () from /lib64/libpthread.so.0
#5  0x0000003701ae896d in clone () from /lib64/libc.so.6

---Type <return> to continue, or q <return> to quit---
Thread 1 (Thread 0x7fef7f620860 (LWP 13207)):
#0  0x0000003701adf143 in poll () from /lib64/libc.so.6
#1  0x00000032e3e531ac in virEventPollRunOnce () from /usr/lib64/libvirt.so.0
#2  0x00000032e3e523e7 in virEventRunDefaultImpl () from /usr/lib64/libvirt.so.0
#3  0x00000032e3f4693d in virNetServerRun () from /usr/lib64/libvirt.so.0
#4  0x00000000004244b7 in ?? ()
#5  0x0000003701a1ed5d in __libc_start_main () from /lib64/libc.so.6
#6  0x0000000000423389 in ?? ()
#7  0x00007ffc40263458 in ?? ()
#8  0x000000000000001c in ?? ()
#9  0x0000000000000002 in ?? ()
#10 0x00007ffc40263f4f in ?? ()
#11 0x00007ffc40263f58 in ?? ()
#12 0x0000000000000000 in ?? ()
Comment 2 Ján Tomko 2016-01-20 03:00:20 EST
Fixed upstream by:
commit 4cf1c3fab138462fc9c014aee853fa17f278c5df
Author:     Peter Krempa <pkrempa@redhat.com>
CommitDate: 2014-08-21 15:55:07 +0200

    conf: net: Correctly switch how to format address fields
    
    When formatting the forward mode addresses or interfaces the switch was
    done based on the type of the network rather than of the type of the
    individual <interface>/<address> element. In case a user would specify
    an incorrect network type ("passhtrough") with <address> elements,
    libvirtd would crash as it would attempt to format an <interface>.
    
    Use the type of the individual element to format the XML.
    
    Resolves: https://bugzilla.redhat.com/show_bug.cgi?id=1132347

git describe: v1.2.7-192-g4cf1c3f contains: v1.2.8-rc1~54

Downstream patch:
http://post-office.corp.redhat.com/archives/rhvirt-patches/2016-January/msg00250.html
Comment 4 Jingjing Shao 2016-01-21 21:29:36 EST
Verify it as follows. The result is expected. Move its status to VERIFIED.

1.[root@sriov1 jishao]# rpm -q libvirt
libvirt-0.10.2-56.el6.x86_64

2.[root@sriov1 jishao]# lspci | grep 82576
03:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
03:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
03:10.1 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
03:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
03:10.3 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)

3.[root@sriov1 jishao]# virsh nodedev-list --tree
computer
  |
  +- net_lo_00_00_00_00_00_00
  +- net_virbr1_nic_52_54_00_3c_df_9b
  +- net_virbr2_nic_52_54_00_3d_09_3d
  +- pci_0000_00_00_0
  +- pci_0000_00_01_0
  |   |
  |   +- pci_0000_03_00_0
  |   |   |
  |   |   +- net_eth0_00_1b_21_39_8b_18
  |   |
  |   +- pci_0000_03_00_1
  |   |   |
  |   |   +- net_eth1_00_1b_21_39_8b_19
  |   |
  |   +- pci_0000_03_10_0
  |   |   |
  |   |   +- net_eth5_7e_53_1e_e7_47_fc
  |   |
  |   +- pci_0000_03_10_1
  |   |   |


4.[root@sriov1 jishao]# virsh nodedev-dumpxml pci_0000_03_00_0
<device>
  <name>pci_0000_03_00_0</name>
  <parent>pci_0000_00_01_0</parent>
  <driver>
    <name>igb</name>
  </driver>
  <capability type='pci'>
    <domain>0</domain>
    <bus>3</bus>
    <slot>0</slot>
    <function>0</function>
    <product id='0x10c9'>82576 Gigabit Network Connection</product>
    <vendor id='0x8086'>Intel Corporation</vendor>
    <capability type='virt_functions'>
      <address domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
      <address domain='0x0000' bus='0x03' slot='0x10' function='0x2'/>
    </capability>
  </capability>
</device>

5.[root@sriov1 jishao]# cat hostdev.xml
<network>
  <name>hostdev</name>
   <ip address='10.66.4.135' prefix='24'/>
      <forward mod='hostdev' managed='yes'>
      <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
   </forward>
</network>

6.[root@sriov1 jishao]# virsh net-define hostdev.xml
Network hostdev defined from hostdev.xml

7.[root@sriov1 jishao]# service libvirtd status
libvirtd (pid  5353) is running...

8.[root@sriov1 jishao]# virsh net-list --all
Name                 State      Autostart     Persistent
--------------------------------------------------
default              inactive   no            yes
hostdev              inactive   no            yes
vlan                 active     yes           yes

9.[root@sriov1 jishao]# virsh net-dumpxml hostdev
<network>
  <name>hostdev</name>
  <uuid>9c4a2fe5-c8ee-09ed-8062-fefc5bdc9f23</uuid>
  <forward mode='nat'>
    <address type='pci' domain='0x0000' bus='0x03' slot='0x10' function='0x0'/>
  </forward>
  <bridge name='virbr2' stp='on' delay='0' />
  <mac address='52:54:00:2F:D6:16'/>
  <ip address='10.66.4.135' prefix='24'>
  </ip>
</network>

10.[root@sriov1 jishao]# virsh net-start hostdev
Network hostdev started

11.[root@sriov1 jishao]# ifconfig
......
virbr2    Link encap:Ethernet  HWaddr 52:54:00:2F:D6:16
          inet addr:10.66.4.135  Bcast:10.66.4.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
....
Comment 6 errata-xmlrpc 2016-05-10 15:25:54 EDT
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2016-0738.html

Note You need to log in before you can comment on or make changes to this bug.