Bug 1293230 – rt: netpoll: live lock with NAPI polling and busy polling on realtime kernel

Bugzilla will be upgraded to version 5.0. The upgrade date is tentatively scheduled for 2 December 2018, pending final testing and feedback.

Bug 1293230 - rt: netpoll: live lock with NAPI polling and busy polling on realtime kernel

Summary:	rt: netpoll: live lock with NAPI polling and busy polling on realtime kernel

Status:	CLOSED ERRATA

Aliases:	None

Product:	Red Hat Enterprise Linux 7
Classification:	Red Hat
Component:	kernel-rt (Show other bugs)
Sub Component:
Version:	7.3
Hardware:	x86_64 Linux

Priority	high Severity high
Target Milestone:	rc
Target Release:	---
Assigned To:	Luis Claudio R. Goncalves
QA Contact:	Zhenjie Chen
Docs Contact:

URL:
Whiteboard:
Keywords:	ZStream

Depends On:	1292902
Blocks:	1295884
	Show dependency tree / graph

Reported:	2015-12-21 02:06 EST by Jan Kurik
Modified:	2016-08-15 18:54 EDT (History)
CC List:	7 users (show)

See Also:
Fixed In Version:
Doc Type:	Bug Fix
Doc Text:	Cause: As the realtime kernel replaces most of the spinlocks by rtmutexes, in order to maximize the preemption points inside the kernel, the locking scheme used in both NAPI polling and busy polling could become out of synchronization with the State Machine they protected. That could be observed when a machine with faster NICs (10g or 40g) was subject to a heavy pressure receiving network packets. Consequence: The system could experience a livelock that would either slowly degrade the system performance up to a halt or disable the network adapter and halt. Fix: the locking schemes on NAPI polling and busy polling routines were hardened to enforce the State machine sanity. Result: the system works fine under pressure.
Story Points:	---
Clone Of:	1292902
Clones:	1295884 (view as bug list)
Environment:
Last Closed:	2016-02-16 05:50:02 EST
Type:	---
Regression:	---
Mount Type:	---
Documentation:	---
CRM:
Verified Versions:
Category:	---
oVirt Team:	---
RHEL 7.3 requirements from Atomic Host:
Cloudforms Team:	---

Attachments	(Terms of Use)
netpoll: Always take poll_lock when doing polling (2.17 KB, patch) 2016-01-06 22:55 EST, Luis Claudio R. Goncalves	no flags	Details \| Diff
Revert "ixgbevf: Prevent livelock spinning grabbing ixgbevf_qv_lock" (3.18 KB, patch) 2016-01-06 22:57 EST, Luis Claudio R. Goncalves	no flags	Details \| Diff
revert "ixgbe: Prevent livelock spinning grabbing ixgbe_qv_lock" (3.53 KB, patch) 2016-01-06 22:58 EST, Luis Claudio R. Goncalves	no flags	Details \| Diff
revert "sfc: Prevent livelock spinning grabbing efx_channel_lock" (3.07 KB, patch) 2016-01-06 22:58 EST, Luis Claudio R. Goncalves	no flags	Details \| Diff
Setup notes for hp-dl388g8-09.rhts.eng.nay.redhat.com (2.05 KB, text/plain) 2016-02-02 09:20 EST, Clark Williams	no flags	Details
Setup notes for rhelrt-17.farm.hsv.redhat.com (2.11 KB, text/plain) 2016-02-02 09:21 EST, Clark Williams	no flags	Details
Add an attachment (proposed patch, testcase, etc.)

External Trackers
Tracker	ID	Priority	Status	Summary	Last Updated
Red Hat Product Errata	RHSA-2016:0212	normal	SHIPPED_LIVE	Important: kernel-rt security, bug fix, and enhancement update	2016-02-16 10:46:40 EST

Groups: None (edit)

Description Jan Kurik 2015-12-21 02:06:59 EST

This bug has been copied from bug #1292902 and has been proposed
to be backported to 7.2 z-stream (EUS).

Comment 3 Luis Claudio R. Goncalves 2016-01-06 22:55 EST

Created attachment 1112330 [details]
netpoll: Always take poll_lock when doing polling

Comment 4 Luis Claudio R. Goncalves 2016-01-06 22:57 EST

Created attachment 1112331 [details]
Revert "ixgbevf: Prevent livelock spinning grabbing ixgbevf_qv_lock"

Comment 5 Luis Claudio R. Goncalves 2016-01-06 22:58 EST

Created attachment 1112332 [details]
revert "ixgbe: Prevent livelock spinning grabbing ixgbe_qv_lock"

Comment 6 Luis Claudio R. Goncalves 2016-01-06 22:58 EST

Created attachment 1112333 [details]
revert "sfc: Prevent livelock spinning grabbing efx_channel_lock"

Comment 11 Clark Williams 2016-02-02 09:20 EST

Created attachment 1120448 [details]
Setup notes for hp-dl388g8-09.rhts.eng.nay.redhat.com

Notes for reproducing this bug on an HP DL388 with SolarFlare (sfc) NIC

Comment 12 Clark Williams 2016-02-02 09:21 EST

Created attachment 1120449 [details]
Setup notes for rhelrt-17.farm.hsv.redhat.com

setup notes for test box in HSV lab

Comment 16 errata-xmlrpc 2016-02-16 05:50:02 EST

Since the problem described in this bug report should be
resolved in a recent advisory, it has been closed with a
resolution of ERRATA.

For information on the advisory, and where to find the updated
files, follow the link below.

If the solution does not work for you, open a new bug report.

https://rhn.redhat.com/errata/RHSA-2016-0212.html

Note You need to log in before you can comment on or make changes to this bug.