Bug 1643065

Summary:	[OVS 2.10] ofproto-dpif-xlate: Avoid deadlock on multicast snooping recursion.
Product:	Red Hat Enterprise Linux 7	Reporter:	Flavio Leitner <fleitner>
Component:	openvswitch2.10	Assignee:	Timothy Redaelli <tredaelli>
Status:	CLOSED ERRATA	QA Contact:	Rick Alongi <ralongi>
Severity:	medium	Docs Contact:
Priority:	high
Version:	7.5	CC:	aguetta, amuller, apevec, atelang, atragler, bshephar, chrisw, ctrautma, fleitner, igkioka, ovs-team, qding, ragiman, ralongi, rhos-maint, tredaelli
Target Milestone:	rc
Target Release:	---
Hardware:	All
OS:	Linux
Whiteboard:
Fixed In Version:	openvswitch2.10-2.10.0-25.el7fdn	Doc Type:	Bug Fix
Doc Text:	Consequence: Open vSwitch goes into deadlocked state when receiving an ICMPv6 multicast listener report v2 packet on a system port and trying to send it back on the IN_PORT after modifying the VLAN tag Fix: This patch fixes the problem, by releasing the read-lock before doing any outputs.	Story Points:	---
Clone Of:	1640875
Clones:	1643071 (view as bug list)		Environment:
Last Closed:	2019-01-07 08:51:12 UTC	Type:	Bug
Regression:	---	Mount Type:	---
Documentation:	---	CRM:
Verified Versions:		Category:	---
oVirt Team:	---	RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---	Target Upstream Version:
Embargoed:

Description Flavio Leitner 2018-10-25 12:54:34 UTC

Bug to avoid regression.

+++ This bug was initially created as a clone of Bug #1640875 +++

Description of problem:
Issue: https://github.com/openvswitch/ovs-issues/issues/153
Copied from the github issue: https://github.com/openvswitch/ovs/commit/ed56f8bd6a668df3664f68b00c5396c268aa1821

Until now, OVS did multicast snooping outputs holding the read-lock on
the mcast_snooping object.  This could recurse via a patch port to try to
take the write-lock on the same object, which deadlocked.  This patch fixes
the problem, by releasing the read-lock before doing any outputs.

It would probably be better to use RCU for mcast_snooping.  That would be
a bigger patch and less suitable for backporting.

----

Upstream branch-2.10 patch:
https://github.com/openvswitch/ovs/commit/9006428ea5f3ac4526aefd3f66ef6f11d1dbe1cd

Comment 2 Flavio Leitner 2018-10-25 14:16:30 UTC

*** Bug 1640930 has been marked as a duplicate of this bug. ***

Comment 5 Rick Alongi 2018-11-27 15:21:55 UTC

SanityOnly verification:

[ralongi@ralongi openvswitch2.10]$ rhpkg switch-branch fast-datapath-rhel-7
Switched to branch 'fast-datapath-rhel-7'

[ralongi@ralongi openvswitch2.10]$ git log --oneline --grep=1643065
80be318 Rebase to openvswitch2.10-2.10.0-28.el7fdn

[ralongi@ralongi openvswitch2.10]$ git show 80be318

.....

    commit 97ee068715c1597f430ba8f664917c306628aeb5
    Author: Timothy Redaelli <tredaelli>
    Date:   Tue Nov 6 15:22:36 2018 +0100
    
        Backport "ofproto-dpif-xlate: Avoid deadlock on multicast snooping recursion"
    
        Resolves: #1643065

.....

[ralongi@ralongi openvswitch2.10]$ git show 80be318 | grep snoop
        Backport "ofproto-dpif-xlate: Avoid deadlock on multicast snooping recursion"
diff --git a/0001-ofproto-dpif-xlate-Avoid-deadlock-on-multicast-snoop.patch b/0001-ofproto-dpif-xlate-Avoid-deadlock-on-multicast-snoop.patch
+++ b/0001-ofproto-dpif-xlate-Avoid-deadlock-on-multicast-snoop.patch
+Subject: [PATCH] ofproto-dpif-xlate: Avoid deadlock on multicast snooping
+Until now, OVS did multicast snooping outputs holding the read-lock on
+the mcast_snooping object.  This could recurse via a patch port to try to
+It would probably be better to use RCU for mcast_snooping.  That would be
+@@ -2662,6 +2664,53 @@ update_mcast_snooping_table(const struct xlate_ctx *ctx,
++ * mcast_snooping's rwlock for reading to iterate through the port lists and
++ * same mcast_snooping and attempt to take the write lock (see
+                               struct mcast_snooping *ms OVS_UNUSED,
+                                  struct mcast_snooping *ms,
+                                struct mcast_snooping *ms,
+                                struct mcast_snooping *ms,
+             if (mcast_snooping_is_membership(flow->tp_src)) {
+                 /* RFC4541: section 2.1.1, item 1: A snooping switch should
+             grp = mcast_snooping_lookup4(ms, flow->nw_dst, vlan);
+             grp = mcast_snooping_lookup(ms, &flow->ipv6_dst, vlan);
+             if (mcast_snooping_flood_unreg(ms)) {
+Patch170: 0001-ofproto-dpif-xlate-Avoid-deadlock-on-multicast-snoop.patch
+- Backport "ofproto-dpif-xlate: Avoid deadlock on multicast snooping recursion" (#1643065


Regression test job link: https://beaker.engineering.redhat.com/jobs/3119375

Comment 9 errata-xmlrpc 2019-01-07 08:51:12 UTC

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://access.redhat.com/errata/RHBA-2019:0029