Bug 1211436

Summary: non-root libvirtd crashed when do nwfilter-list in user mode
Product: Red Hat Enterprise Linux 7 Reporter: Luyao Huang <lhuang>
Component: libvirtAssignee: Michal Privoznik <mprivozn>
Status: CLOSED ERRATA QA Contact: Virtualization Bugs <virt-bugs>
Severity: medium Docs Contact:
Priority: medium    
Version: 7.2CC: dyuan, fjin, honzhang, mzhan, rbalakri, shyu
Target Milestone: rcKeywords: Upstream
Target Release: ---   
Hardware: x86_64   
OS: Linux   
Whiteboard:
Fixed In Version: libvirt-1.2.15-1.el7 Doc Type: Bug Fix
Doc Text:
Story Points: ---
Clone Of: Environment:
Last Closed: 2015-11-19 06:28:16 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:

Description Luyao Huang 2015-04-14 02:01:40 UTC
Description of problem:
non-root libvirtd crashed when do nwfilter-list in user mode

Version-Release number of selected component (if applicable):
libvirt-1.2.14-1.el7.x86_64
qemu-kvm-rhev-2.2.0-8.el7.x86_64

How reproducible:
100%

Steps to Reproduce:
0. login a non-root user:
$ id
uid=1000(lhaung) gid=1000(lhaung) groups=1000(lhaung) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

$ virsh uri
qemu:///session

1. $ virsh list
 Id    Name                           State
----------------------------------------------------

2. $ ps aux|grep libvirtd
root     11658  0.0  0.3 1166784 25196 ?       Ssl  09:31   0:00 /usr/sbin/libvirtd
lhaung   24241  2.8  0.2 730584 17724 ?        Sl   17:31   0:00 /usr/sbin/libvirtd --timeout=30
lhaung   24264  0.0  0.0 112644   960 pts/18   S+   17:31   0:00 grep --color=auto libvirtd

3. $ virsh nwfilter-list
error: Failed to list node filters
error: End of file while reading data: Input/output error

4. $ ps aux|grep libvirtd
root     11658  0.0  0.3 1166784 25196 ?       Ssl  09:31   0:00 /usr/sbin/libvirtd
lhaung   24285  0.0  0.0 112644   960 pts/18   S+   17:31   0:00 grep --color=auto libvirtd

Actual results:
non-root libvirtd crashed when do nwfilter-list

Expected results:
fix it

infomation:

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f040f937700 (LWP 24703)]
__GI___pthread_mutex_lock (mutex=0x0) at pthread_mutex_lock.c:50
50          unsigned int type = PTHREAD_MUTEX_TYPE (mutex);
(gdb) bt
#0  __GI___pthread_mutex_lock (mutex=0x0) at pthread_mutex_lock.c:50
#1  0x00007f0417fe6cc5 in virMutexLock (m=<optimized out>) at util/virthread.c:89
#2  0x00007f0406410951 in nwfilterDriverLock () at nwfilter/nwfilter_driver.c:75
#3  nwfilterConnectListAllNWFilters (conn=0x7f03f80009a0, filters=0x7f040f936b60, flags=0) at nwfilter/nwfilter_driver.c:491
#4  0x00007f041809e0a8 in virConnectListAllNWFilters (conn=0x7f03f80009a0, filters=0x7f040f936b60, flags=0) at libvirt-nwfilter.c:98
#5  0x00007f0418b19e55 in remoteDispatchConnectListAllNWFilters (server=0x7f041964a710, msg=0x7f0419662400, ret=0x7f03f40008e0, args=0x7f03f40008c0, rerr=0x7f040f936c70, client=<optimized out>) at remote.c:4998
#6  remoteDispatchConnectListAllNWFiltersHelper (server=0x7f041964a710, client=<optimized out>, msg=0x7f0419662400, rerr=0x7f040f936c70, args=0x7f03f40008c0, ret=0x7f03f40008e0) at remote_dispatch.h:1294
#7  0x00007f04180eb152 in virNetServerProgramDispatchCall (msg=0x7f0419662400, client=0x7f04196621d0, server=0x7f041964a710, prog=0x7f041965f250) at rpc/virnetserverprogram.c:437
#8  virNetServerProgramDispatch (prog=0x7f041965f250, server=server@entry=0x7f041964a710, client=0x7f04196621d0, msg=0x7f0419662400) at rpc/virnetserverprogram.c:307
#9  0x00007f0418b43efd in virNetServerProcessMsg (msg=<optimized out>, prog=<optimized out>, client=<optimized out>, srv=0x7f041964a710) at rpc/virnetserver.c:172
#10 virNetServerHandleJob (jobOpaque=<optimized out>, opaque=0x7f041964a710) at rpc/virnetserver.c:193
#11 0x00007f0417fe7615 in virThreadPoolWorker (opaque=opaque@entry=0x7f0419637d50) at util/virthreadpool.c:145
#12 0x00007f0417fe6b38 in virThreadHelper (data=<optimized out>) at util/virthread.c:206
#13 0x00007f041541fdf5 in start_thread (arg=0x7f040f937700) at pthread_create.c:308
#14 0x00007f04151461ad in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Additional info:

I think this issue was introduced in 1.2.13, maybe commit 55ea7be7.

Comment 1 Michal Privoznik 2015-04-16 08:45:46 UTC
Patch proposed upstream:

https://www.redhat.com/archives/libvir-list/2015-April/msg00720.html

Comment 2 Michal Privoznik 2015-04-17 13:04:05 UTC
And I've pushed the patch upstream:

commit 77d92e2e77f0dae8fc9e1eb5fa0db9fc9f2818bd
Author:     Michal Privoznik <mprivozn>
AuthorDate: Thu Apr 16 09:59:22 2015 +0200
Commit:     Michal Privoznik <mprivozn>
CommitDate: Fri Apr 17 10:04:05 2015 +0200

    nwfilter: Partly initialize driver even for non-privileged users
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1211436
    
    This reverts commit b7829f959b33c6e32422222a9ed745c0da7dc696.
    
    The previous fix was not correct. Like everywhere else, a driver is a
    global variable allocated in stateInitialize function (or something
    similar for stateless drivers). Later, when a driver API is called,
    it's possible that the global variable is accessed and dereferenced.
    Now, some drivers require root privileges because they undertake some
    actions reserved only for the system admin (e.g. manipulating host
    firewall). And here's the trouble, the NWFilter state initializer
    exited too early when finding out it's running unprivileged, leaving
    the global NWFilter driver variable uninitialized. Any subsequent
    API call that tried to lock the driver resulted in dereferencing the
    driver and thus crash.
    
    On the other hand, in order to not resurrect the bug the original
    commit was fixing, Let's forbid the nwfilter define in session mode.
    
    Signed-off-by: Michal Privoznik <mprivozn>
    
    Conflicts:
        src/nwfilter/nwfilter_driver.c: Context. Code changed a bit
            since 2013.

v1.2.14-214-g77d92e2

Comment 4 Fangge Jin 2015-07-01 09:43:00 UTC
I can reproduce this bug on build:
libvirt-1.2.14-1.el7.x86_64

Verify this bug on build:
libvirt-1.2.16-1.el7.x86_64

Verify steps:
0. login as a non-root user:
$ id
uid=1000(fjin) gid=1000(fjin) groups=1000(fjin) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

$ virsh uri
qemu:///session

1.
$ virsh list
 Id    Name                           State
----------------------------------------------------

2.
$ ps aux|grep libvirtd
root     17777  0.0  0.0 906768 24644 ?        Ssl  Jun30   0:00 /usr/sbin/libvirtd --listen
fjin     31314 15.0  0.0 803156 16568 ?        Sl   15:48   0:00 /usr/sbin/libvirtd --timeout=30
fjin     31349  0.0  0.0 112640   964 pts/0    S+   15:48   0:00 grep --color=auto libvirtd

3.
$ virsh nwfilter-list
 UUID                                  Name                 
------------------------------------------------------------------

4.
$ ps aux|grep libvirtd
root     17777  0.0  0.0 906768 24644 ?        Ssl  Jun30   0:00 /usr/sbin/libvirtd --listen
fjin     31314  0.8  0.0 868692 16588 ?        Sl   15:48   0:00 /usr/sbin/libvirtd --timeout=30
fjin     31353  0.0  0.0 112640   964 pts/0    S+   15:48   0:00 grep --color=auto libvirtd

5.
$ cat disallow-arp.xml 
<filter name='disallow-arp' chain='arp'>
  <rule action='drop' direction='inout' priority='500'/>
</filter>

$ virsh nwfilter-define disallow-arp.xml 
error: Failed to define network filter from disallow-arp.xml
error: Requested operation is not valid: Can't define NWFilters in session mode

6.
$ ps aux|grep libvirtd
root     17777  0.0  0.0 906768 24644 ?        Ssl  Jun30   0:00 /usr/sbin/libvirtd --listen
fjin     31314  0.8  0.0 868692 16588 ?        Sl   15:48   0:00 /usr/sbin/libvirtd --timeout=30
fjin     31353  0.0  0.0 112640   964 pts/0    S+   15:48   0:00 grep --color=auto libvirtd


The bug has been fixed, so move to verified.

Comment 6 errata-xmlrpc 2015-11-19 06:28:16 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHBA-2015-2202.html