Bug 1621626
| Summary: | [OVS} OVS daemon got segfault when virsh start <guest> | ||
|---|---|---|---|
| Product: | Red Hat Enterprise Linux Fast Datapath | Reporter: | Jean-Tsung Hsiao <jhsiao> |
| Component: | openvswitch2.11 | Assignee: | Open vSwitch development team <ovs-team> |
| Status: | CLOSED ERRATA | QA Contact: | Jean-Tsung Hsiao <jhsiao> |
| Severity: | low | Docs Contact: | |
| Priority: | low | ||
| Version: | FDP 19.C | CC: | ctrautma, fleitner, jhsiao, kfida, kzhang, maxime.coquelin, nsaddler, qding, ralongi, tredaelli |
| Target Milestone: | --- | ||
| Target Release: | --- | ||
| Hardware: | x86_64 | ||
| OS: | Linux | ||
| Whiteboard: | |||
| Fixed In Version: | openvswitch2.10-2.10.0-49.el7fdn | Doc Type: | If docs needed, set a value |
| Doc Text: | Story Points: | --- | |
| Clone Of: | Environment: | ||
| Last Closed: | 2019-08-19 16:20:36 UTC | Type: | Bug |
| Regression: | --- | Mount Type: | --- |
| Documentation: | --- | CRM: | |
| Verified Versions: | Category: | --- | |
| oVirt Team: | --- | RHEL 7.3 requirements from Atomic Host: | |
| Cloudforms Team: | --- | Target Upstream Version: | |
| Embargoed: | |||
| Bug Depends On: | 1713698 | ||
| Bug Blocks: | |||
|
Description
Jean-Tsung Hsiao
2018-08-23 17:14:41 UTC
Hi Jean-Tsung, Could you please share the OVS logs so that I can check the vhost-user request received? Thanks, Maxime (In reply to Maxime Coquelin from comment #3) > Hi Jean-Tsung, > > Could you please share the OVS logs so that I can check the vhost-user > request received? > > Thanks, > Maxime Hi Maxime, Just reproduced it. Got the following piece of log when "virsh start <guest>"; and, the daemon got segfaulted. 2018-08-23T23:57:31.175Z|00108|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on vxlan_sys_4789 device failed: No such device 2018-08-23T23:57:31.216Z|00109|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on vxlan_sys_4789 device failed: No such device 2018-08-23T23:57:31.471Z|00001|dpdk|INFO|VHOST_CONFIG: /tmp/vhost0: connected 2018-08-23T23:57:31.471Z|00002|dpdk|INFO|VHOST_CONFIG: new device, handle is 0 2018-08-23T23:57:31.495Z|00001|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_GET_FEATURES 2018-08-23T23:57:31.495Z|00002|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_GET_PROTOCOL_FEATURES 2018-08-23T23:57:31.495Z|00003|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_PROTOCOL_FEATURES 2018-08-23T23:57:31.495Z|00004|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_GET_QUEUE_NUM 2018-08-23T23:57:31.495Z|00005|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_SLAVE_REQ_FD 2018-08-23T23:57:31.495Z|00006|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_OWNER 2018-08-23T23:57:31.495Z|00007|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_GET_FEATURES 2018-08-23T23:57:31.496Z|00008|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL 2018-08-23T23:57:31.496Z|00009|dpdk|INFO|VHOST_CONFIG: vring call idx:0 file:81 2018-08-23T23:57:31.496Z|00010|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_VRING_CALL 2018-08-23T23:57:31.496Z|00011|dpdk|INFO|VHOST_CONFIG: vring call idx:1 file:82 2018-08-23T23:57:31.503Z|00110|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on vxlan_sys_4789 device failed: No such device 2018-08-23T23:57:31.548Z|00111|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on vxlan_sys_4789 device failed: No such device 2018-08-23T23:57:33.197Z|00112|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on vxlan_sys_4789 device failed: No such device 2018-08-23T23:57:35.196Z|00113|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on vxlan_sys_4789 device failed: No such device 2018-08-23T23:57:35.242Z|00114|netdev_linux|INFO|ioctl(SIOCGIFINDEX) on vxlan_sys_4789 device failed: No such device 2018-08-23T23:57:40.691Z|00012|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE 2018-08-23T23:57:40.693Z|00013|dpdk|INFO|VHOST_CONFIG: set queue enable: 1 to qp idx: 0 2018-08-23T23:57:40.693Z|00014|netdev_dpdk|INFO|State of queue 0 ( tx_qid 0 ) of vhost device '/tmp/vhost0'changed to 'enabled' 2018-08-23T23:57:40.693Z|00015|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE 2018-08-23T23:57:40.693Z|00016|dpdk|INFO|VHOST_CONFIG: set queue enable: 1 to qp idx: 1 2018-08-23T23:57:40.693Z|00017|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE 2018-08-23T23:57:40.693Z|00018|dpdk|INFO|VHOST_CONFIG: set queue enable: 1 to qp idx: 0 2018-08-23T23:57:40.693Z|00019|netdev_dpdk|INFO|State of queue 0 ( tx_qid 0 ) of vhost device '/tmp/vhost0'changed to 'enabled' 2018-08-23T23:57:40.693Z|00020|dpdk|INFO|VHOST_CONFIG: read message VHOST_USER_SET_VRING_ENABLE 2018-08-23T23:57:40.693Z|00021|dpdk|INFO|VHOST_CONFIG: set queue enable: 1 to qp idx: 1 (In reply to Jean-Tsung Hsiao from comment #4) > (In reply to Maxime Coquelin from comment #3) > > Hi Jean-Tsung, > > > > Could you please share the OVS logs so that I can check the vhost-user > > request received? > > > > Thanks, > > Maxime > > Hi Maxime, > Just reproduced it. Got the following piece of log when "virsh start > <guest>"; and, the daemon got segfaulted. Thanks Jean-Tsung, It seems the trace wasn't fully evacuated into the journal. Any chance I can connect to your setup? So I managed to reproduce on Jean's setup.
The crash happens because mmap of guest memory fails due to SElinux policy:
ovs-vswitchd[60356]: ovs|00024|dpdk|ERR|VHOST_CONFIG: mmap region 0 failed.
and:
type=AVC msg=audit(1535118626.626:515): avc: denied { map } for pid=60356 comm="ovs-vswitchd" path=2F6465762F6875676570616765732F6C6962766972742F71656D752F332D6D712D7668752D74756E6E656C2D7365727665722F71656D755F6261636B5F6D656D2E5F6F626A656374735F72616D2D6E6F6465302E79427A6E3162202864656C6574656429 dev="hugetlbfs" ino=4125881 scontext=system_u:system_r:openvswitch_t:s0 tcontext=system_u:object_r:svirt_image_t:s0 tclass=file permissive=0
Jean already filed a Bz for this SELinux policy issue (Bz1620257).
However, the vhost-user library should be more robust, so that it handles
gracefully SELinux policy issues.
I'll keep this Bz to make vhost-user more robust, but lower the priority.
Hi Aaron, See Comment #6 for the same AVC listed in bug 1620257. Thanks! Jean *** Bug 1601134 has been marked as a duplicate of this bug. *** Patch posted upstream: http://patches.dpdk.org/patch/48346/ The patch is part of v18.11.1 upstream. So making it depending on Bz1713698 (Rebase to 18.11.2) and set FDP release to 19.E. Have verified and passed the fix with OVS-2.11.0-18. Since the problem described in this bug report should be resolved in a recent advisory, it has been closed with a resolution of ERRATA. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHBA-2019:2522 |