Bug 1643071

Summary: [OVS 2.9] ofproto-dpif-xlate: Avoid deadlock on multicast snooping recursion.
Product: Red Hat Enterprise Linux 7 Reporter: Flavio Leitner <fleitner>
Component: openvswitchAssignee: Timothy Redaelli <tredaelli>
Status: CLOSED ERRATA QA Contact: Rick Alongi <ralongi>
Severity: medium Docs Contact:
Priority: high    
Version: 7.5CC: aguetta, amuller, apevec, atelang, atragler, bshephar, chrisw, ctrautma, fleitner, jmaxwell, ovs-qe, ovs-team, qding, ragiman, rhos-maint, tredaelli
Target Milestone: rc   
Target Release: ---   
Hardware: All   
OS: Linux   
Whiteboard:
Fixed In Version: openvswitch-2.9.0-79.el7fdn Doc Type: If docs needed, set a value
Doc Text:
Story Points: ---
Clone Of: 1643065 Environment:
Last Closed: 2019-01-02 17:54:40 UTC Type: Bug
Regression: --- Mount Type: ---
Documentation: --- CRM:
Verified Versions: Category: ---
oVirt Team: --- RHEL 7.3 requirements from Atomic Host:
Cloudforms Team: --- Target Upstream Version:
Embargoed:
Bug Depends On:    
Bug Blocks: 1640875    

Description Flavio Leitner 2018-10-25 12:59:10 UTC
+++ This bug was initially created as a clone of Bug #1640875 +++

Description of problem:
Issue: https://github.com/openvswitch/ovs-issues/issues/153
Copied from the github issue: https://github.com/openvswitch/ovs/commit/ed56f8bd6a668df3664f68b00c5396c268aa1821

Until now, OVS did multicast snooping outputs holding the read-lock on
the mcast_snooping object.  This could recurse via a patch port to try to
take the write-lock on the same object, which deadlocked.  This patch fixes
the problem, by releasing the read-lock before doing any outputs.

It would probably be better to use RCU for mcast_snooping.  That would be
a bigger patch and less suitable for backporting.

----

Upstream branch-2.9 patch:
https://github.com/openvswitch/ovs/commit/2ed1c95e9c80a21a5d81cb760c872b62b32b0733

Comment 7 Rick Alongi 2018-11-27 15:27:20 UTC
SanityOnly Verification:

[ralongi@ralongi openvswitch]$ rhpkg switch-branch fast-datapath-rhel-7
Branch fast-datapath-rhel-7 set up to track remote branch fast-datapath-rhel-7 from origin.

[ralongi@ralongi openvswitch]$ git log --oneline --grep=1643071
ba0148b Rebase to openvswitch-2.9.0-81.el7fdn

....
   commit 4fe6a4014fc6aa0b18203909657afa5ca0ed120c
    Author: Timothy Redaelli <tredaelli>
    Date:   Tue Nov 6 17:03:48 2018 +0100
    
        Backport "ofproto-dpif-xlate: Avoid deadlock on multicast snooping recursion"
    
        Resolves: #1643071
....

[ralongi@ralongi openvswitch]$ git show 80be318 | grep snoop
fatal: ambiguous argument '80be318': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
[ralongi@ralongi openvswitch]$ git show ba0148b | grep snoop
        Backport "ofproto-dpif-xlate: Avoid deadlock on multicast snooping recursion"
diff --git a/0001-ofproto-dpif-xlate-Avoid-deadlock-on-multicast-snoop.patch b/0001-ofproto-dpif-xlate-Avoid-deadlock-on-multicast-snoop.patch
+++ b/0001-ofproto-dpif-xlate-Avoid-deadlock-on-multicast-snoop.patch
+Subject: [PATCH] ofproto-dpif-xlate: Avoid deadlock on multicast snooping
+Until now, OVS did multicast snooping outputs holding the read-lock on
+the mcast_snooping object.  This could recurse via a patch port to try to
+It would probably be better to use RCU for mcast_snooping.  That would be
+@@ -2554,6 +2556,53 @@ update_mcast_snooping_table(const struct xlate_ctx *ctx,
++ * mcast_snooping's rwlock for reading to iterate through the port lists and
++ * same mcast_snooping and attempt to take the write lock (see
+                               struct mcast_snooping *ms OVS_UNUSED,
+                                  struct mcast_snooping *ms,
+                                struct mcast_snooping *ms,
+                                struct mcast_snooping *ms,
+             if (mcast_snooping_is_membership(flow->tp_src)) {
+                 /* RFC4541: section 2.1.1, item 1: A snooping switch should
+             grp = mcast_snooping_lookup4(ms, flow->nw_dst, vlan);
+             grp = mcast_snooping_lookup(ms, &flow->ipv6_dst, vlan);
+             if (mcast_snooping_flood_unreg(ms)) {
+Patch330: 0001-ofproto-dpif-xlate-Avoid-deadlock-on-multicast-snoop.patch
+- Backport "ofproto-dpif-xlate: Avoid deadlock on multicast snooping recursion" (#1643071)

Regression test job link: https://beaker.engineering.redhat.com/jobs/3119374

Comment 9 errata-xmlrpc 2019-01-02 17:54:40 UTC
Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0014